Cópia De Shu Lin, Daniel J. Costello - Error Control Coding (2nd Edition) (2004, Prentice Hall).pdf

Vice President and Editorial Director, ECS: Marcia J. Horton Vice President and Director of Production and Manufacturing, ESM: David W. Riccardi Editorial Assistant: Carole Synder Executive Managing Editor: Vince O'Brien Managing Editor: David A. George Production Editor: Kevin Bradley Director of Creative Services: Paul Belfanti Art Director: Jayne Conte Cover Designer: Bruce Kenselaar Art Editor: Greg Dulles Manufacturing Manager: Trudy Pisciotti Manufacturing Buyer: Lynda Castillo Senior Marketing Manager: Holly Stark

About the Cover: Image courtesy of the Burstein Collection/Corbis. © 1983, 2004 by Pearson Education, Inc. Pearson Prentice Hall Pearson Education, Inc. Upper Saddle River, NJ 07458

All rights reserved. No part of this book may be reproduced, in any form or by any means, without permission in writing from the publisher. Pearson Prentice Hall® is a trademark of Pearson Education, Inc. The author and publisher of this book have used their best efforts in preparing this book. These efforts include the development, research, and testing of the theories and programs to determine their effectiveness. The author and publisher make no warranty of any kind, expressed or implied with regard to these programs or the documentation contained in this book. The author and publisher shall not be liable in any event for incidental or consequential damages in connection with or arising out of the furnishing, performance or use of these programs. If you purchased this book within the United States or Canada you should be aware that it has been

wrongfully imported without the approval of the Publisher or Author. Printed in the United States of America 1098765432

ISBN D-13-017973-6 Pearson Education Ltd., London Pearson Education Australia Pty. Ltd., Sydney Pearson Education Singapore, Pte. Ltd. Pearson Education North Asia Ltd., Hong Kong Pearson Education Canada, Inc., Toronto Pearson Educaci6n de Mexico, S.A. de C.V. Pearson Education-Japan, Tokyo Pearson Education Malaysia, Pte. Ltd. Pearson Education, Inc., Upper Saddle River, New Jersey

]_

([)Olidlii1111g [l[J)lf ]Ril:)Ilii21lbJilie Jl])n!l;Ilfall 1I'nllll11§lill11Il§§Ill[J)Il11 !11111lidl §fomge

]_

1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8

1 3 5

Introduction . . . . . . . Types of Codes . . . . . . . . . . Modulation and Coding . . . . . Maximum Likelihood Decoding Types of Errors . . . . . Error Control Strategies Performance Measures Coded Modulation Bibliography . . .

Z llrrntl1rnidllllldlll[J)llll t{!]) Ail!§;il:)lblr!ll

2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8

3

13 14

15 21 23

25 25 31 37 42

47 54 55

61 63 65

Urrne!llr Bfotelk (Cl[J)rdJte§

66

3.1 3.2 3.3 3.4

66

3.5 3.6 3.7

~

Groups . . . . . . Fields . . . . . . . Binary Field Arithmetic . Construction of Galois Field G F(2111 ) Basic Properties of a Galois Field GF(2111 ) Computations Using Galois Field GF(2 111 ) Arithmetic . Vector Spaces Matrices .. Problems .. Bibliography

10

Introduction to Linear Block Codes . Syndrome and Error Detection ... The Minimum Distance of a Block Code Error-Detecting and Error-Correcting Capabilities of a Block Code . . . . . . . . . . . . . . . . . . . . . . . . . . . Standard Array and Syndrome Decoding . . . . . . . Probability of an Undetected Error for Linear Codes over a BSC . . . . . . . . . . . . . . . . . . . . . . . . Single-Parity-Check Codes, Repetition Codes, and Self-Dual Codes . . . . Problems .. Bibliography

lIHllllJP1@irftrull1ltl ILiirrne211r Bfoirlk OoJ«l!te§

4.1 4.2 4.3 4.4

Hamming Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . A Class of Single-Error-Correcting and Double-Error-Detecting Codes . . . . . . . . . . . . . . . . . . . . . . . Reed-Muller Codes. . . . . . . . . . . . . . . Other Constructions for Reed-Muller Codes

72

76 78 82

90 94 95 97 99

100 102 105 114

iv

Contents

4.5 4.6 4.7 4.8

The Squaring Construction of Codes The (24, 12) Golay Code Product Codes . . Interleaved Codes Problems .. Bibliography . . .

119 125 128 131 132 134

5 Cydic Codes 5.1 Description of Cyclic Codes . . . . . . . . . . . . . . . Generator and Parity-Check Matrices of Cyclic Codes 5.2 5.3 Encoding of Cyclic Codes . . . . . . . . . . . . 5.4 Syndrome Computation and Error Detection 5.5 Decoding of Cyclic Codes 5.6 Cyclic Hamming Codes . . . . . . . . 5.7 Error-Trapping Decoding . . . . . . 5.8 Improved Error-Trapping Decoding 5.9 The (23, 12) Golay Code 5.10 Shortened Cyclic Codes . 5.11 Cyclic Product Codes 5.12 Quasi-Cyclic Codes Problems . . Bibliography . . . .

136 136 143 146 150 155 162 166 173 175 179 184 185 188 192

6 Bi1rmry BCH Codes 6.1 Binary Primitive BCH Codes . . . . . . . . . . . . . 6.2 Decoding of BCH Codes . . . . . . . . . . . . . . . 6.3 Iterative Algorithm for Finding the Error-Location Polynomial u (X) . . . . . . . . . . . . . . . . . . . . 6.4 Simplified Iterative Algorithm for Finding the Error-Location Polynomial u(X) . . . . . . . . . . . . . . . . . . . . . . . . . 6.5 Finding the Error-Location Numbers and Error Correction 6.6 Correction of Errors and Erasures . . . . . . 6.7 Implementation of Galois Field Arithmetic . . . . . . . . . . 6.8 Implementation of Error Correction . . . . . . . . . . . . . 6.9 Weight Distribution and Error Detection of Binary BCH Codes. 6.10 Remarks . . Problems . . Bibliography

194 194 205

7 Nrnrubinary BCH Codes, Reed-Sollomoira Codes, and Decoding Algorithms 7.1 q-ary Linear Block Codes . . . . . 7.2 Primitive BCH Codes over GF(q) . . . . . . . . . . . . . . . . Reed-Solomon Codes . . . . . . . . . . . . . . . . . . . . . . 7.3 7.4 Decoding of Nonbinary BCH and RS Codes: The Berlekamp Algorithm . . . . . . . . . . . . . . . . . 7.5 Decoding with the Euclidean Algorithm 7.6 Frequency-Domain Decoding . . . 7.7 Correction of Errors and Erasures . . . .

234 234 236 237

209 212 215 217 217 224 227 230 230 231

241 248 255 263

Contents Problems .. Bibliography ~

'!lJ

v 269

270

!Mhnj(f])Irn1ly JLl[J)11;nir ][])eir(f])«ll1il!lbillie !llrrn«ll lFnrrnihte
Z'TJ

8.1 8.2 8.3 8.4 8.5 8.6 8. 7 8.8 8.9

273

0

One-Step Majority-Logic Decoding . . . . . . . . A Class of One-Step Majority-Logic Decodable Codes Other One-Step Majority-Logic Decodable Codes Multiple-Step Majority-Logic Decoding Euclidean Geometry . . . . Eudidean Codes . . . . . . . Twofold EG Codes . . . . . . . . . . . . Projective Geometry and Projective Geometry Codes Remarks .. Problems .. Bibliography

282

290 296

304 309 319

325 331 332

335

1I'Irelllln§®§ foir J1nll1l®ID!i' JIBil(!J)teJk (C(f])«Jl®§

9.1 9.2 9.3 9.4 9.5 9.6 9.7 9.8 9.9

Finite-State Machine Model and Trellis Representation of a Code . . . . . . . . . . . . . . . . . . . . . . . . Bit-Level Trellises for Binary Linear Block Codes . State Labeling . . . . . . . . . . . . . . . . . . . . . Structural Properties of Bit-Level Trellises . . . . . State Labeling and Trellis Construction Based on the Parity-Check Matrix . . . . . . . . . . . . . . . . . . . Trellis Complexity and Symmetry . . . . . . . . . . . Trellis Sectionalization and Parallel Decomposition . Low-Weight Subtrellises Cartesian Product Problems .. Bibliography . . .

].([D lR?.eilnmlbiiiiln1ly lIBID§®«Jl §(!])[1l ][])eirfon([J)Il11 ][])eic(f])@Illl111); Ail!li([J)Ii'Il1llhlrrrrn§ foir lLilll1l®IDir lIBil(f])irllz (C([J)@®§ QCl[}W1JdbllJJted by Mf{}JrrC !Po C lF@§§l[}Tr#err) 0

10.1 10.2

338

342 351 354 360 367 374 380 382 390 391

0

Soft-Decision Decoding . . . . . . . . . . . . . . . . . . . . Reliability Measures and General Reliability-Based Decoding Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3 Sufficient Conditions on the Optimality of a Decoded Codeword . . . . . . . . . . . . . . . . . . . . . . . . . 10.4 Generalized Minimum Distance and Chase Decoding Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . 10.5 Weighted Erasure Decoding . . . . . . . . . . . . . . . 10.6 A Maximum Likelihood Decoding Algorithm Based on Iterative Processing of the Least Reliable Positions . . . . . . . . . . . 10.7 Reduced List Syndrome Decoding Algorithm . . . . . . . . . 10.8 Most Reliable Independent Position Reprocessing Decoding Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.9 Weighted Majority-Logic Decoding . . . . . . . . . . . . . . . 10.10 Iterative Reliability-Based Decoding of One-Step Majority-Logic Decodable Codes . . . . . . . . . . . . . . . . . . . . . . . . . . .

395 395

400 402 407 413

417 419 422 439

442

vi

Contents

Problems Bibliography

447 448

11 Convohntional Codes 11.1 Encoding of Convolutional Codes . . . . . . . 11.2 Structural Properties of Convolutional Codes 11.3 Distance Properties of Convolutional Codes Problems . . Bibliography . . . . . . . . . . . . . .

453 454 486 506 510 513

12 Optimum Decoding of Convolutional Codes 12.1 The Viterbi Algorithm . . . . . . . . . . . . . . 12.2 Performance Bounds for Convolutional Codes . 12.3 Construction of Good Convolutional Codes . . 12.4 Implementation and Performance of the Viterbi Algorithm 12.5 The Soft-Output Viterbi Algorithm (SOVA) . . 12.6 The BCJR algorithm . . . . . . . . . . . . . . . 12.7 Punctured and Tail-Biting Convolutional Codes Problems . . Bibliography . . . . . . . . . . . . . . . . . . . .

515 516 525 538 544 558 563 582 598 602

13 Suboptimum Decoding of Convoiutional Codes 13.1 The ZJ (Stack) Sequential Decoding Algorithm 13.2 The Fano Sequential Decoding Algorithm . . . 13.3 Performance Characteristics of Sequential Decoding 13.4 Code Construction for Sequential Decoding . . . . . 13.5 Majority-Logic Decoding . . . . . . . . . . . . . . . . 13.6 Performance Characteristics of Majority-Logic Decoding. 13.7 Code Construction for Majority-Logic Decoding Problems . . Bibliography . . . . . . . . . . . . . . . . . .

605 606 620 626 640 645 670 677 685 688

14 Trellis-Based Soft-Decision Decoding Algorithms 14.1 The Viterbi Decoding Algorithm . . . . . . . . . . . . . . 14.2 A Recursive Maximum Likelihood Decoding Algorithm . 14.3 A Suboptimum Iterative Decoding Algorithm Based on a Low-Weight Subtrellis . . . . . . . . . . . . . . . 14.4 The MAP Decoding Algorithm . . . . . . . . . . 14.5 MAP Decoding Based on a Sectionaliz@fi Trellis . 14.6 Max-log-MAP Decoding Algorithm . Problems . . Bibliography . . . . . . . . . . . . . .

691 691 695 704 711 718 726 734 735

15 Concatenated Coding, Code Decomposition, and Multistage Decoding 15.1 Single-Level Concatenated Codes . . . 15.2 Multilevel Concatenated Codes . . . . 15.3 A Soft-Decision Multistage Decoding . 15.4 Decomposition of Codes . . . . . . . . 15.5 An Iterative Multistage MLD Algorithm

739 739 743 748 750 754

Contents

15.6

Concatenated Coding Schemes with Convolutional Inner Codes . . . . . . . . . . Binary Concatenation . Problems .. Bibliography . . . . . .

15.7

]_(l\i

vii

760 761 763 764

1rumrlb)I[]) C
'J(fii({ii

16.1 16.2 16.3 16.4 16.5

767 783 807 814 826 844 847

Introduction to Turbo Coding . . . . . Distance Properties of Turbo Codes .. Performance Analysis of Turbo Codes Design of Turbo Codes . . . . . . . Herative Decoding of Turbo Codes Problems .. Bibliography . . . . . . . . . .

]_'/ ]Ll[J)W lIJJerm§n1ly lP'&11irntly°Cllnedk: (Cl[J)tdle§ 0

17.1 17.2 17 .3 17.4 17.5 17.6 17.7 17.8 17.9 17.10 17.11 17 .12

Introduction to LDPC Codes. Tanner Graphs for Linear Block Codes ... A Geometric Construction of LD PC Codes EG-LDPC Codes . . . . . PG-LDPC Codes . . . . . . . . . . . . . . . Decoding of LDPC Codes . . . . . . . . . . Code Construction by Column and Row Splitting Breaking Cycles in Tanner Graphs . . . . Shortened Finite-Geometry LDPC Codes Construction of Gallager LDPC Codes .. Masked EG-Gallager LDPC Codes . . . . Construction of Quasi-Cyclic Codes by Circulant Decomposition . . . . . . . . . . . . . . . . . . . . Construction of LDPC Codes Based on Finite Geometries over GF(p 5 ) • • • • • • Random LDPC Codes . . . . . Irregular LDPC Codes . . . . . Graph-Theoretic LDPC Codes. Construction of LDPC Codes Based on Balanced Incomplete Block Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . Construction of LDPC Codes Based on Shortened RS Codes with Two Information Symbols . . . . . . . . Concatenations with LDPC and Turbo Codes Problems .. Bibliography . . . . . . . . . . . . . . . . .

17.13 17.14 17.15 17.16 17.17 17.18 17.19

]_~ 1fireililfa
18.1 18.2 18.3 18.4 18.5

Introduction to Trellis-Coded Modulation TCM Code Construction . . TCM Performance Analysis Rotationally Invariant TCM Multidimensional TCM . . .

~§]_

852 855 858 860 866 871 885 892 898 902 906 912 917 920 922

929 935 938 944 945 947 i\ȤZ

953 980 992 998 1015

viii

Contents

Problems . . . . . . Bibliography . . . .

1056 1059

19 Block Coded Modulation 19.1 Distance Concepts . . . . . . . . . . . . . . . . . 19.2 Multilevel Block Modulation Codes . . . . . . . 19.3 Multistage Decoding of Multilevel BCM Codes 19.4 Concatenated Coded Modulation . . . . . . . . 19.5 Product Coded Modulation . . . . . . . . . . . . 19.6 Multilevel Coded Modulation for Unequal Error Protection . Problems . . . . . . . . Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1063 1063 1064 1075 1081 1088 1090 1100 1101

20 Bmrst-Error-Correcting Codes 20.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 20.2 Decoding of Single-Burst-Error-Correcting Cyclic Codes . 20.3 Single-Burst-Error-Correcting Codes . . . . 20.4 Phased-Burst-Error-Correcting Codes . . . 20.5 Burst-and-Random-Error-Correcting Codes Problems . . . . . . . . . . . . . . . . . Bibliography . . . . . . . . . . . . . . . . . .

1104 1104 1105 1107 1118 1119 1124 1125

21 Burst-Error-Corredb11g Convohlltionai Codes 21.1 Bounds on Burst-Error-Correcting Capability 21.2 Burst-Error-Correcting Convolutional Codes 21.3 Interleaved Convolutional Codes . . . . . . . 21.4 Burst-and-Random-Error-Correcting Convolutional Codes Problems . . . . . . . . . . . . . Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . .

1127 1127 1128 1139 1142 1153 1154

22 Automatic-Repeat-Request Strategies 22.1 Basic ARQ Schemes . . . . . . . . . . . . . . . . . . . . . . 22.2 Selective-Repeat ARQ System with Finite Receiver Buffer 22.3 ARQ Schemes with Mixed Modes of Retransmission 22.4 Hybrid ARQ Schemes . . . . . . . . . . . . . . . . . . . . . 22.5 A Class of Half-Rate Invertible Codes . . . . . . . . . . . . 22.6 Type-II Hybrid Selective-Repeat ARQ with Finite Receiver Buffer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22.7 Hybrid ARQ Systems Using Convolutional Codes . . . . 22.8 A Concatenated Coded Modulation Hybrid ARQ System Problems . . . . Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . .

1156 1156 1163 1171 1174 1178

A Tables of Gafofa Fields

1204

B Minimal Polynomials of Elements in GF(2m)

1227

C Generator Polynomials of Binary Primitive BCH Codes of Length up to 2111 - 1

1231

Index

1181 1190 1192 1197 1198

1249

This book owes its beginnings to the pioneering work of Claude Shannon in 1948 on reliable communication over noisy transmission channels. Shannon's central theme was that if the signaling rate of the system is less than the channel capacity, reliable communication can be achieved if one chooses proper encoding and decoding techniques. The design of good codes and of efficient decoding initiated by Hamming, Golay, and others in the late 1940s, has since occupied the energies of many researchers. Much of this work is highly mathematical in nature, and a thorough understanding requires an extensive background in modern algebra and probability theory. This requirement has impeded many engineers and computer scientists who are interested in applying these techniques to practical systems. One of the purposes of this book is to present the essentials of this highly complex material in such a manner that it can be understood and applied with only a minimum of mathematical background. Coding research in the 1950s and 1960s was devoted primarily to developing the theory of efficient encoders and decoders. In 1970 the first author published a book entitled An Introduction to Error-Correcting Codes, which presented the fundamentals of the previous two decades of work covering both block and convolutional codes. The approach was to explain the material in an easily understood manner, with a minimum of mathematical rigor. Then, in 1983, the authors published the first edition of this book. The approach again stressed the fundamentals of coding. In addition, new material on many of the practical applications of coding developed during the 1970s was introduced. Other major additions included a comprehensive treatment of the error-detecting capabilities of block codes and an emphasis on soft decoding methods for convolutional codes. In the 1980s and 1990s, the coding field exploded with new theoretical developments, several of which have had significant practical consequences. Three of these new developments stand out in particular: the application of binary convolutional and block codes to expanded (nonbinary) modulation alphabets, the development of practical soft decoding methods for block codes, and the discovery of soft-input, softoutput iterative decoding techniques for block and convolutional codes. These new developments have revolutionized the way coding is applied to practical systems, affecting the design of high-speed data modems, digital mobile cellular telephony, satellite and space communications, and high-density data storage, to name just a few. A total of seven new chapters covering these topics have been added to this edition: two chapters on trellis- and block-coded modulation techniques, three chapters on soft decoding methods for block codes, and two chapters on turbo and low-density parity-check codes and iterative decoding. Because of the significant amount of new material and the need to maintain an emphasis on coding fundamentals, it was not possible to include certain topics in this edition. For example, the new developments in algebraic geometry codes and erasure correcting codes are not covered. Also, although the codes developed in the book can be applied to data storage systems, the specific peculiarities of the storage channel are not directly addressed. Similarly, it was not possible to give a comprehensive treatment of coding for fading channels. In addition to the new

x

Preface

chapters noted, all the chapters in the first edition have been thoroughly revised and updated in the second edition. A brief description of each chapter follows, highlighting changes from the first edition. Chapter 1 presents an overview of coding for error control in data transmission and storage systems. A brief discussion of modulation and demodulation serves to place coding in the context of a complete system. Two new sections, introducing the concepts of coded modulation, coding gain, and the Shannon limit, have been added. Chapter 2 develops those concepts from modern algebra that are necessary to understand the material in later chapters. The presentation is at a level that can be understood by students in the senior year as well as by practicing engineers and computer scientists. Chapters 3 through 10 cover in detail the fundamentals of block codes. Linear block codes are presented in Chapter 3. Included is material on the error detection capability of linear codes. Several important classes of linear codes are introduced in Chapter 4. New material on Reed-Muller codes has been added to this chapter. The basic structure and properties of cyclic codes are presented in Chapter 5, and syndrome-based decoding methods are introduced. The important class of BCH codes is presented in detail in Chapter 6. A discussion of hardware and software implementation of BCH decoders is included, as well as the use of BCH codes for error detection. Chapter 7 includes an expanded coverage of Reed-Solomon codes. New material on the Euclidean algorithm and frequency-domain decoding has been added. Chapter 8 provides detailed coverage of majority-logic decodable codes, including the important classes of Euclidean and projective geometry codes. Chapters 9 and 10 are both completely new. Chapter 9 develops the theory of the trellis structure of block codes, laying the groundwork for the introduction of trellis-based soft decoding algorithms in Chapter 14. Chapter 10, written by Professor Marc Fossorier, presents comprehensive coverage of reliability-based soft decoding methods for block codes and includes an introduction to iterative decoding techniques. Chapters 11 through 13 are devoted to the presentation of the fundamentals of convolutional codes. Convolutional codes are introduced in Chapter 11, with the encoder state diagram serving as the basis for studying code structure and distance properties. New material on feedback encoders and input-output weight enumerating functions has been added. Chapter 12 covers optimum decoding methods for convolutional codes, with an emphasis on the (maximum likelihood) Viterbi decoding algorithm for both hard and soft demodulator decisions. New material has been added on the soft-output Viterbi algorithm, the (maximum a posteriori probability) BCJR algorithm, and the code modification techniques of puncturing and tail-biting. A detailed performance analysis based on encoder weight enumerating functions is also included. Chapter 13 covers suboptimum decoding methods for convolutional codes, with an emphasis on sequential decoding, using both the ZJ (stack) and Fano algorithms, and majority-logic decoding. The analytically difficult problem of the computational performance of sequential decoding is discussed without including detailed proofs, and new material on soft-decision versions of sequential and majority-logic decoding has been added.

Preface

,ii

Chapter 14 extends the soft decoding methods introduced for convolutional codes in Chapter 12 to block codes. This completely new chapter makes extensive use of the block code trellis structures introduced in Chapter 9. Chapters 15 through 19 cover the important advances in the field since the publication of the first edition. Chapter 15 discusses the important concepts of code concatenation, multistage decoding, and code decomposition. These topics lay the groundwork for the new coding techniques presented in the next four chapters. Chapters 16 19 are completely new. Chapter 16 introduces the area of or turbo coding, and its related iterative decoding techniques based on the BCJR algorithm presented in Chapter 12. Performance analysis based on the uniform interleaver technique and the EXIT chart concept is included. 17 presents a thorough coverage of parity-check codes based on algebraic, random, and combinatoric construction methods. Several decoding methods are discussed, and a complete development of soft-decision belief propagation decoding is included. The area of coded modulation is covered in Chapters 18 and 19. The fundamentals of trellis-coded modulation are presented in Chapter 18. Sections on rotationally invariant codes and multidimensional signal sets are included. Block-coded modulation is covered in Chapter 19. The important concepts of multilevel modulation and multistage decoding are included. The book concludes with three chapters on burst-error correction and automatic-repeat-request (ARO) strategies. Chapters 20 and 21 cover methods for correcting the burst errors and combinations of burst and random errors commonly encountered on fading channels. Both block (Chapter 20) and convolutional (Chapter 21) burst-error-correcting codes are included. Chapter 22 is devoted to the ARQ error control schemes used on two-way communication channels. Both pure ARQ (error detection with retransmission) and ARQ (a combination of error correction and error detection with retransmission) are discussed. Several additional features make the book useful both as a classroom text and as a comprehensive reference for engineers and computer scientists involved in the design of error control systems. Three appendices include the algebraic background used in the construction of most block codes. Many tables of the best known block and convolutional codes for various decoding methods are presented throughout the book. These tables can be used by system designers to select the best code for a given application. In this edition, a consistent octal notation has been adopted for the generator and parity-check polynomials that define the codes listed in these tables. Many examples of codes used in practical applications and computersimulated performance curves for specific coding systems are also included. A set of homework problems is given at the end of each chapter. Most of these problems are relatively straightforward applications of material covered in the text, although some more advanced problems are included. Instructors can obtain solutions to selected problems from the publisher. References are also given at the end of each chapter. Although no attempt was made to compile a complete bibliography on coding, the references listed serve to provide additional detail on topics covered in the book. The book can be used as a text for an introductory course on coding at the senior or beginning graduate level or a more comprehensive full-year graduate course. H also can be used as a self-study guide for engineers and computer scientists

xii

Preface

in industry who want to learn the fundamentals of coding and how they can be applied to the design of error control systems. As a text the book can be used as the basis for a two-semester sequence in coding theory, with Chapters 1-10 on the fundamentals of block codes covered in one semester and the remaining chapters on convolutional codes and advanced block code topics in a second semester. Another possibility is to cover Chapters 1-8 and 11-13, which include the fundamentals of both block and convolutional codes, in one semester, followed by a second semester devoted to advanced topics. Alternatively portions of the book can be covered in a one-semester course. A course on block codes comprise Chapters 1-7 plus selected topics from Chapters 8-10, 14-15, 17, 19-20, and 22, whereas Chapters 1, 11-13, 16, 18, and 21 provide a thorough coverage of convolutional codes. We would like to express our sincere appreciation to Professor Marc Fossorier, who, in addition to writing Chapter 10, spent many long hours reading and rereading drafts of various chapters. We also wish to thank the many graduate students and postdoctoral associates who offered their comments and helped in the preparation of the book, including running computer simulations, drafting figures, compiling tables, and converting the manuscript to LaTeX format. These persons include Yu Kou, Cathy Liu, Rose Shao, Diana Stojanovic, Jun Xu, Lei Chen, Oscar Takeshita, Gil Shamir, Adrish Banerjee, Arvind Sridharan, Ching He, Wei Zhang, and Ali Pusane. In particular, Yu Kou, Cathy Liu, and Adrish Banerjee deserve special mention for overseeing the preparation of the final version of the manuscript. We are grateful to the National Science Foundation and to the National Aeronautics and Space Administration for their continuing support of our research in the coding field. Without their assistance, our interest in coding could never have developed to the point of writing this book. We also thank the University of Hawaii, Manoa, the University of California, Davis, the University of Notre Dame, and the Humboldt Foundation of Germany for their support of our efforts in writing this book and for providing facilities. Finally, we would like to give special thanks to our wives, children, and grandchildren for their continuing love and affection throughout this project.

SHU LIN

University of California, Davis University of Hawaii,.Manoa DANIEL J. COSTELLO, JR. University of Notre Dame

cC@@BuD(9] "tf@u ~®~D@~Q)~® [D)o(9Joit@~ lu@uD~rno~~o@uD @uUCQJ §lt@u@(9]® In recent years, there has been an increasing demand for efficient and reliable digital data transmission and storage systems. This demand has been accelerated by the emergence of large-scale, high-speed data networks for the exchange, processing, and storage of digital information in the commercial, governmental, and military spheres. A merging of communications and computer technology is required in the design of these systems. A major concern of the system designer is the control of errors so that the data can be reliably reproduced. fo 1948, Shannon [1] demonstrated in a landmark paper that, by proper encoding of the information, errors induced by a noisy channel or storage medium can be reduced to any desired level without sacrificing the rate of information transmission or storage, as long as the information rate is less than the capacity of the channel. Since Shannon's work much effort has been expended on the problem of devising efficient encoding and decoding methods for error control in a noisy environment. Recent developments have contributed toward achieving the reliability required by today's high-speed digital systems, and the use of coding for error control has become an integral part in the design of modern communication and digital storage systems. The transmission and storage of digital information have much in common. Both processes transfer data from an information source to a destination (or user). A typical transmission (or storage) system may be represented by the block diagram shown in Figure 1.1. The information source can be either a person or a machine-for example, a digital computer, or a data terminal. The source output, which is to be communicated to the destination, can be either a continuous waveform or a sequence of discrete symbols. The source encoder transforms the source output into a sequence of binary digits (bits) called the information sequence ll!l. fo the case of a continuous source, this process involves analog-to-digital (A/D) conversion. The source encoder is ideally designed so that (1) the number of bits per unit time required to represent the source output is minimized; and (2) the source output can be unambiguously reconstructed from the information sequence ll!l. The subject of source coding is not discussed in this book. For a thorough treatment of this important topic, see references [2], [3], [4] and [5]. The channel encoder transforms the information sequence ll!l into a discrete encoded sequence wcalled a codeword. In most instances w is also a binary sequence, although in some applications nonbinary codes have been used. The design and implementation of channel encoders to combat the noisy environment in which codewords must be transmitted or stored is one of the major topics of this book.

2

Chapter 1

Coding for Reliable Digital Transmission and Storage

Information source

Source encoder

Channel encoder

Noise----

Destination

FIGURE 1.1:

Source decoder

Channel decoder

Modulator (writing unit)

Channel (storage medium)

Demodulator (reading unit)

Block diagram of a typical data transmission or storage system.

Discrete symbols are not suitable for transmission over a physical channel or recording on a digital storage medium. The modulator (or writing unit) transforms each output symbol of the channel encoder into a waveform of duration T seconds that is suitable for transmission (or recording). This waveform enters the channel (or storage medium) and is corrupted by noise. Typical transmission channels include telephone lines, mobile cellular telephony, high-frequency (HF) radio, telemetry, microwave and satellite links, optical fiber cables, and so on. Typical storage media include core and semiconductor memories, magnetic tapes, drums and discs, compact discs, optical memory units, and so on. Each of these examples is subject to various types of noise disturbances. On a telephone line, the disturbance may come from switching impulse noise, thermal noise, or crosstalk from other lines. On magnetic discs (or compact discs), surface defects and dust particles are regarded as noise disturbances. The demodulator (or reading unit) processes each received waveform of duration T and produces either a discrete (quantized) or a continuous (unquantized) output. The sequence of demodulator outputs corresponding to the encoded sequence v is called the received sequence r. The channel decoder transforms the received sequencer into a binary sequence u called the estimated information sequence. The decoding strategy is based on the rules of channel encoding and the noise characteristics of the channel (or storage medium). Ideally, u will be a replica of the information sequence un, although the noise may cause some decoding errors. Another major topic of this book is the design and implementation of channel decoders that minimize the probability of decoding error. The source decoder transforms the estimated information sequence u into an estimate of the source output and delivers this estimate to the destination. When the source is continuous, this process involves digital-to-analog (D/A) conversion. In a well-designed system, the estimate will be a faithful reproduction of the source output except when the channel (or storage medium) is very noisy. To focus attention on the channel encoder and channel decoder, (1) the information source and source encoder can be combined into a digital source with output u; (2) the modulator ( or writing unit), the channel (or storage medium), and

Section 1.2

Types of Codes

3

~D~ig-it-al~~--@--~r-;:~ _ source

[___::::__J

Noise - - - >

Digital sink

FIGURE 1.2:

Coding channel

Decoder

Simplified model of a coded system.

the demodulator (or reading unit) can be combined into a coding channel with input v and output ir; and (3) the source decoder and destination can be combined into a digital sink with input illi. These combinations result in the simplified block diagram shown in Figure 1.2. The major engineering problem that is addressed in this book is to design and implement the channel encoder/decoder pair such that (1) information can be transmitted (or recorded) in a noisy environment as fast (or as densely) as possible; (2) the information can be reliably reproduced at the output of the channel decoder; and (3) the cost of implementing the encoder and decoder faHs within acceptable limits.

Two structurally different types of codes are in common use today: block codes and convolutional codes. The encoder for a block code divides the information sequence into message blocks of k information bits (symbols) each. A message block is represented by the binary k-tuple Uii = (uo, iii,··· , uk-I), called a message. (In block coding, the symbol llil is used to denote a k-bit message rather than the entire information sequence.) There are a total of 2k different possible messages. The encoder transforms each message llil independently into an n-tuple v = (vo, VI,··· , Vn-I) of discrete symbols, called a codeword. (In block coding, the symbol v is used to denote an n-symbol block rather than the entire encoded sequence.) Therefore, corresponding to the 2k different possible messages, there are 2k different possible codewords at the encoder output. This set of 2k codewords of length n is called an (n, k) block code. The ratio R = k/n is called the code rate, and it can be interpreted as the number of information bits entering the encoder per transmitted symbol. Because the n-symbol output codeword depends only on the corresponding k-bit input message; that is, each message is encoded independently, the encoder is memoryless and can be implemented with a combinational logic circuit. In a binary code, each codeword vis also binary. Hence, for a binary code to be useful; that is, to have a different codeword assigned to each message, k < n, or R ::: 1. When k < n, n - k redundant bits are added to each message to form a

4

Chapter 1

Coding for Reliable Digital Transmission and Storage TAB LE 1.1 : A binary block code with k = 4 and n = 7.

Messages

Codewords

(0 0 0 0) (100 0) (0 100) (1100) (0 0 1 0) (101 0) (0 11 0) (l 1 l 0) (0001) (100 1) (0 101) (1101) (0 0 11) (1011) (0111) (1111)

(0 0 0 0 0 0 0) (110100 0) (0 110100) (1011100) (11100 1 0) (0 0 1101 0) (100 0 11 0) (0 1 0 l l l 0) (10100 0 1) (0111001) (1100 1 0 l) (0001101) (0100011) (100 1011) (0010111) (111111 1)

codeword. These redundant bits provide the code with the capability of combating the channel noise. For a fixed code rate R, more redundant bits can be added by increasing the number of message bits k and the block length n of the code while holding the ratio k/n constant. How to choose these redundant bits to transmit information reliably over a noisy channel is the major problem in designing the encoder. An example of a binary block code with k = 4 and n = 7 is shown in Table 1.1. Chapters 3 through 10, 14, 15, 17, 19, and 20 are devoted to the analysis, design, and decoding of block codes for controlling errors in a noisy environment. The encoder for a convolutional code also accepts k-bit blocks of the information sequence u and produces an encoded sequence (code sequence) v of n-symbol blocks. (In convolutional coding, the symbols u and v are used to denote sequences of blocks rather than a single block.) However, each encoded block depends not only on the corresponding k-bit message block at the same time unit but also on 111 previous message blocks. Hence, the encoder has a memory order of 111. The set of all possible encoded output sequences produced by the encoder forms the code. The ratio R = k/n is called the code rate. Because the encoder contains memory, it must be implemented with a sequential logic circuit. In a binary convolutional code, redundant bits for combating the channel noise are added to the information sequence when k < n, or R < 1. Typically, k and n are small integers, and more redundancy is added by increasing the memory order 111 of the code while holding k and n, and hence the code rate R, fixed. How to use the memory to achieve reliable transmission over a noisy channel is the major problem in designing the encoder of a convolutional code. An example of a binary feed-forward convolutional encoder with k = 1, n = 2, and 111 = 2 is shown in Figure 1.3. As an illustration of how codewords are generated, consider the information sequence

Section 1.3

Modulation and Coding

5

,-------u--->[}---> '\ ~~~~~~~~~~~~>{

Shift register stage

+

I--~~~

Multiplexer

EXCLUSIVE-OR gate

FIGURE 1.3: A binary feed-forward convolutional encoder with k

= 1,

n

=

2, and

m =2. = (11 0 1 0 0 0 · · · ) , where the leftmost bit is assumed to enter the encoder first Using the rules of EXCLUSIVE-OR (or X-OR) addition, and assuming that the multiplexer takes the first encoded bit from the top output, it is easy to see that the encoded sequence is v = (11, 10, 10, 00, 01, 11, 00, 00, 00, ... ). Chapters 11 through 13, 18, and 21 are devoted to the design, and decoding of convolutional codes for controlling errors in a noisy environment.

i!Il

The modulator in a communication system must select a waveform of duration T seconds that is suitable for transmission for each encoder output symbol. fo the case of a binary code, the modulator must generate one of two signals, s1 (t) for an encoded "l" or s2(t) for an encoded "O". For a wideband channel, the optimum choice of signals is s1(t)

{2E; = yT cos2nJot,

s2(t)

{2E; = yT cos(2nJot + n)

0::::

t:::: T (1.1)

where the carrier frequency Jo is a multiple of 1/ and Es is the energy of each signal. This form of modulation is called binary-phase-shift-keying (BPSK), since the phase of the carrier cos 2n Jot is shifted between O and n, depending on the encoder output. The BPSK-modulated waveform corresponding to the codeword w = (11 01 0 0 0) in Table 1.1 is shown in Figure 1.4. A common form of noise disturbance present in any communication system is additive white Gaussian noise (AWGN) [2, 6, 7]. H the transmitted signal is

6

Chapter 1

Coding for Reliable Digital Transmission and Storage s(t)

R -R

1

0

1

0

0

0

0

FIGURE 1.4:

BPSK-modulated waveform corresponding to the codeword v

(1101000). s(t)(= s1 (t) or s2(t)), then the received signal is r(t) = s(t) + n(t),

(1.2)

where n(t) is a Gaussian random process with one-sided power spectral density (PSD) No. Other forms of noise are also present in many systems [7]. For example, in a communication system subject to multipath transmission, the received signal is observed to fade (lose strength) during certain time intervals. This fading can be modeled as a multiplicative noise scaling factor on the signal s(t). The demodulator must produce an output corresponding to the received signal in each T -second interval. This output may be a real number or one of a discrete set of preselected symbols depending on the demodulator design. An optimum demodulator always includes a matched filter or correlation detector followed by a switch that samples the output once every T seconds. For BPSK modulation with coherent detection the sampled output is a real number y

=

foT r ( t ) f f cos2nfot dt.

(1.3)

The sequence of unquantized demodulator outputs can be passed on directly to the channel decoder for processing. In this case, the channel decoder must be capable of handling unquantized inputs; that is, it must process real numbers. A much more common approach to decoding is to quantize the real-number detector output y into one of a finite number Q of discrete output symbols. In this case, the channel decoder has discrete inputs; that is, it must process discrete values. Most coded communication systems use some form of discrete processing. To transmit information with M = l1 channel signals, the output sequence of the binary encoder is first segmented into a sequence of /-bit bytes. Each byte is called a symbol, and there are M distinct symbols. Each symbol is then mapped into one of the M signals in a signal set S for transmission. Each signal is a waveform pulse of duration T, which results in M-ary modulation. One example of M-ary modulation is M-ary phase-shift-keying (MPSK), for which the signal set consists of M sinusoidal pulses. These signals have the same energy Es but M different equally spaced phases. Such a signal set is given by s;(t) =

y{iE; T cos(2nfot + ¢;),

0::::

t::::

T,

Section 1.3

Modulation and Coding

71

----0<1c-~~~~~~~~>o--S2 SJ

(a) BPSK

A

(b) QPSK

(c) 8-PSK

FIGURE 1.5: BPSK, QPSK, and 8-PSK signal constellations.

where¢; = 2n(i -1) / M for 1 ::::c i ::::c M. Because the signals have constant envelope, MPSK is also called constant-envelope modulation. For M = 2, M = 4, and M = 8, we have BPSK, 4-PSK (also known as QPSK), and 8-PSK, respectively. These are commonly used modulations for digital communications. Their signal space constellations are depicted in Figure 1.5. Other types of M-ary modulation will be discussed in Chapters 18 and 19. H the detector output in a given interval depends only on the transmitted signal in that interval, and not on any previous transmission, the channel is said to be memoryless. fo this case, the combination of an M-ary input modulator, the physical channel, and a Q-ary output demodulator can be modeled as a discrete memoryless channel (DMC). A DMC is completely described by a set of transition probabilities P(Jli), 0 ::::c i ::::c M -1, 0 ::::c j ::::c Q -1, where i represents a modulator input symbol, j represents a demodulator output symbol, and P(j Ii) is the probability of receiving j given that i was transmitted. As an example, consider a communication system in which (1) binary modulation is used (M = 2), (2) the amplitude distribution of the

8

Chapter 1

Coding for Reliable Digital Transmission and Storage l-p-q 0

l-p 0

0

0

Erasure 1

l-p 1-p-q (a)

1

(b)

0 0

1

Q-1 ( c)

FIGURE 1.6: Transition probability diagrams for

(a) binary symmetric channel (BSC); (b) binary symmetric erasure channel; and (c) binary-input, Q-ary-output discrete memoryless channel. noise is symmetric, and (3) the demodulator output is quantized to Q = 2 levels. In this case a particularly simple and practically important channel model, called the binary symmetric channel (BSC), results. The transition probability diagram for a BSC is shown in Figure l.6(a). Note that the transition probability p completely describes the channel. The transition probability p can be calculated from a knowledge of the signals used, the probability distribution of the noise, and the output quantization threshold of the demodulator. When BPSK modulation is used on an A WGN channel with optimum coherent detection and binary output quantization, the BSC transition probability is just the uncoded BPSK bit error probability for equally likely signals given by (1.4) p = Q()2Es/No), where Q(x) ~ ~ fx°° e-Y 12 dy is the complementary error function (or simply Q-function) of Gaussian statistics. An upper bound on Q(x) that will be used later in evaluating the error performance of codes on a BSC is 2

X '.'.'.:

0.

(1.5)

When binary coding is used, the modulator has only binary inputs (M = 2). Similarly, when binary demodulator output quantization is used (Q = 2), the

Section 1.3

Modulation and Coding

9l

decoder has only binary inputs. fo this case, the demodulator is said to make hard decisions. Many coded digital communication systems, whether block or convolutional, use binary coding with hard-decision decoding owing to the resulting simplicity of implementation; however, when Q > 2 (or the output is left unquantized) the demodulator is said to make soft decisions. fo this case the decoder must accept multilevel (or continuous-valued) inputs. Although this makes the decoder more difficult to implement, soft-decision decoding offers significant performance improvement compared with hard-decision decoding, as is discussed in Chapters 10 and 12-19. A transition probability diagram for a soft-decision DMC with M = 2 and Q > 2 is shown in Figure 1.6(c ). This is the appropriate model for a binary-input A WGN channel with finite output quantization. The transition probabilities can be calculated from a knowledge of the signals used, the probability distribution of the noise, and the output quantization thresholds of the demodulator in a manner similar to the calculation of the BSC transition probability p. The channel model given by Figure 1.6(b) is called the binary symmetric erasure channel (BSEC) where Q = 3 and one of the output symbols is an erasure (correcting errors and erasures will be discussed in Chapters 6 and 7). For a more thorough treatment of the calculation of DMC transition probabilities see references [6] and [7]. Suppose that the input to the modulator consists of symbols selected from a finite and discrete alphabet X = {xo, x1, · · · , XM-d and the output of the demodulator is left unquantized. fo this case, the combination of the modulator, the physical channel, and the demodulator results in a discrete-input, continuous-output channel. The channel output is a random variable that can assume any value y on the real line, i.e., y E {-oo, +oo }. H the physical channel is subject only to A WGN with zero mean and one-sided PSD No, then the channel output is a Gaussian random variable with zero mean and variance (5 2 = No/2. fo this case, we obtain a discrete-input, continuous-output memoryless Gaussian channel. This channel is completely characterized by a set of M conditional (Gaussian) probability dens1ty functions, p(ylx) for x E {O, 1, · · · , M - 1} [7]. For the special case with M = 2 and x E {O, 1}, we obtain a binary-input, continuous-output memoryless Gaussian channel. H BPSK modulation is used, the channel is completely characterized by the following conditional probability density functions: p(ylx

= 0)

p(yJx

= 1)

(1.6)

where Es is the signal energy. Soft-decision decoding algorithms for this channel will be discussed in later chapters. If the detector output in a given interval depends on the transmitted signal in previous intervals as well as the transmitted signal in the present interval, the channel is said to have memory. A fading channel is a good example of a channel with memory, since the multipath transmission destroys the independence from interval to interval. Appropriate models for channels with memory are difficult to construct, and coding for these channels is more of an art than a science. Two important and related parameters in any digital communication system are the speed of information transmission and the bandwidth of the channel. Because one encoded symbol is transmitted every T seconds, the symbol transmission rate

10

Chapter 1

Coding for Reliable Digital Transmission and Storage

(baud rate) is 1/ T. In a coded system, if the code rate is R = k/n, k information bits correspond to the transmission of n symbols, and the information transmission rate (data rate) is R/T bits per second (bps). In addition to signal modification due to the effects of noise, most communication channels are subject to signal distortion owing to bandwidth limitations. To minimize the effect of this distortion on the detection process, the channel should have a bandwidth W of at least T hertz (Hz). 1 In an uncoded system (R = 1), the data rate l/T = 2W is therefore limited by the channel bandwidth. In a binary coded system, with a code rate R < 1, the data rate R / T = 2RW is reduced by the factor R compared with an uncoded system. Hence, to maintain the same data rate as an uncoded system, the coded system requires bandwidth expansion by a factor of 1/ R. It is characteristic of binary coded systems to require some bandwidth expansion to maintain a constant data rate. If no additional bandwidth can be used without causing severe signal distortion, binary coding is not feasible, and bandwidth-efficient means of reliable communication must be sought. This is the subject of Chapters 18 and 19.

!

1.4

MAXIMUM UKEUHOOD DECOrnNG

A block diagram of a coded system on an A WON channel with finite output quantization is shown in Figure 1.7. In a block-coded system, the source output u represents a k-bit message, the encoder output v represents an n-symbol codeword, the demodulator output r represents the corresponding Q-ary received n-tuple, and the decoder output ii represents the le-bit estimate of the encoded message. In a

Digital source

n (t)

---,,..i

AWGN channel

Demodulator r(t) I

Digital sink

Decoder

14---i---,--I

Q-level quantizer

Matched filter detector

I I

: I I I

L--------------------------~

Discrete memory channel FIGURE 1.7:

A coded system on an additive white Gaussian noise channel.

1The exact bandwidth required depends on the shape of the signal waveform, the acceptable limits of distortion, and the definition of bandwidth.

Section 1 A

Maximum likelihood Decoding

n

convolutional-coded 1lll represents a sequence of kh information bits, and w represents a codeword containing N ~ nh + nm = n(h + m) symbols, where kh is the length of the information sequence, and N is the length of the codeword. The additional nm encoded are produced after the last block of information bits has entered the to them-time unit memory of the encoder. This topic is more discussed in Chapter 11. The demodulator output iris a Q-ary received N-tuple, and the decoder 1Ul is a kh-bit estimate of the information sequence. The decoder must produce an estimate 1Ul of the information sequence 1lll based on the received sequence Jr. since there is a one-to-one correspondence between the information sequence ill! and the codeword w. the decoder can produce an estimate v of the codeword w. Clearly, llll = UJJ if and if v = w. A decoding rule is a strategy for choosing an estimated codeword vfor each possible received sequence ir. If the codeword w was a decoding error occurs if and only if v =I=- w. Given that Ir is the conditional error probability of the decoder is defined as P(E\Jr) ~ P(v =/=- w\ir). (1.7) The error probability of the decoder is then given by P(E)

=

L P(E\ir)P(ir).

(1.8)

where P(ir) is the probability of the received sequence ir. P(ir) is independent of the decoding rule since ir is produced prior to decoding. Hence, an optimum decoding rule, that is, one that minimizes P(E), must minimize P(Elir) = P(v =I=- w\ir) for all ir. Because minimizing P(v =I=- w\ir) is equivalent to maximizing P(v = w\ir), P(Elir) is minimized for a given ir by choosing was the codeword w that maximizes P(w\ir) =

P(irlw)P(w) P(ir) ;

(1.9)

that is, v is chosen as the most likely codeword given that rr· is received. H all information sequences, and hence all codewords, are equally likely; that is, P(w) is the same for all w, maximizing (1.9) is equivalent to maximizing P(ir\v). For a DMC, (1.10)

since for a memoryless channel each received symbol depends only on the corresponding transmitted symbol. A decoder that chooses its estimate to maximize (1.10) is called a maximum likelihood decoder (MLD). Because logx is a monotone increasing function of x, maximizing (1.10) is equivalent to maximizing the log-likelihood function, (1.11)

An MLD for a DMC then chooses v as the codeword v that maximizes the sum in (1.11). If the codewords are not equally likely, then an MLD is not necessarily optimum, since the conditional probabilities P(ir\w) must be weighted by

12

Chapter 1

Coding for Reliable Digital Transmission and Storage

the codeword probabilities P(v) to determine which codeword maximizes P(vlr); however, in many cases, the codeword probabilities are not known exactly at the receiver, making optimum decoding impossible, and an MLD then becomes the best feasible decoding rule. Now, consider specializing the MLD decoding rule to the BSC. In this case r is a binary sequence that may differ from the transmitted codeword v in some positions owing to the channel noise. When r; -1- v;, P(r;lv;) = p, and when r; = v;, P(r;lv;) = 1 - p. Let d(r, v), be the distance between rand v, that is, the number of positions in which r and v differ. This distance is called the Hamming distance. For a block code of length n, (1.11) becomes log P(rlv)

= d(r, v) log p + [n - d(r, v)] log(l = d(r, v) log _P_ + n log(l 1-p

p).

p)

(1.12)

!,

(For a convolutional code, n in (1.12) is replaced by N.) Since log l~p < 0 for p < and n Iog(l - p) is a constant for all v, the MLD decoding rule for the BSC chooses v as the codeword v that minimizes the distance d(r, v) between r and v; that is, it chooses the codeword that differs from the received sequence in the fewest number of positions. Hence, an MLD for the BSC is sometimes called a minimum distance decoder. For a discrete-input, continuous-output memoryless channel, the received sequence r is a sequence of real numbers. To compute the conditional probability P(rlv), we simply replace each symbol transition probability p(r;lv;) in the product of (1.10) with the corresponding conditional probability density function, such as those given in (1.6). Then, the MLD decoding rule is to choose the codeword v as the decoded codeword that maximizes the conditional probability P(rlv). The capability of a noisy channel to transmit information reliably was determined by Shannon [1] in his original work. This result, called the noisy channel coding theorem, states that every channel has a capacity C, and that for any rate R < C, there exist codes of rate R that, with maximum likelihood decoding, have an arbitrarily small decoding error probability P(E). In particular, for any R < C, there exist block codes with sufficiently large block length n such that (1.13)

and there exist convolutional codes with sufficiently large memory order m such that P(E) :'.:: r<m+l)nE,(R).

(1.14)

Eb(R) and Ec(R) are positive functions of R for R < C and are completely determined by the channel characteristics. The bound of (1.13) implies that arbitrarily small error probabilities are achievable with block coding for any fixed R < C by increasing the block length n while holding the ratio k/n constant. The bound of (1.14) implies that arbitrarily small error probabilities are also achievable with convolutional coding for any fixed R < C by increasing the memory order m while holding k and n constant. The noisy channel coding theorem is based on an argument called random coding. The bound obtained is actually on the average error probability of the

Section 1.5

Types of Errors

'il3l

ensemble of all codes. Because some codes must perform better than the average, the noisy channel coding theorem guarantees the existence of codes satisfying (1.13) and (1.14), but it does not indicate how to construct these codes. Furthermore, to long block achieve very low error probabilities for block codes of fixed rate R < lengths are needed. This requires that the number of codewords 2k = 211 R must and be very large. Because an MLD must compute log P(rrJv) for each then choose the codeword that gives the maximum, the number of -~,,.,~,~-u that must be performed by an MLD becomes very large as n increases. For convolutional codes, low error probabilities require a large memory order m. As will be seen in Chapter 12, an MLD for convolutional codes requires to decode each block of k information bits. This number, too, 21rn 1 becomes very large as m increases. Hence, it is impractical to achieve very low error probabilities with maximum likelihood decoding. Therefore, two major encountered when designing a coded system to achieve low error probabilities: to construct good long codes whose performance with MLD satisfies (1.13) and (1.14), and (2) to find easily implementable methods of encoding and decoding these codes such that their actual performance is dose to what could be achieved with MLD. The remainder of this book is devoted to finding solutions to these two

On memoryless channels, the noise affects each transmitted symbol independently. As an example, consider the BSC whose transition diagram is shown in Figure 1.6( a). Each transmitted bit has a probability p of being received incorrectly and a independent of the other transmitted probability 1 - p of being received bits. Hence, transmission errors occur randomly in the received sequence, and memoryless channels are called random-error channels. Good examples of randomerror channels are the deep-space channel and many satellite channels. Most line-of-sight transmission facilities, as well, are primarily affected by random errors. Codes designed to correct random errors are called random-error correcting codes. Most of the codes presented in this book are random-error correcting codes. On channels with memory, the noise is not independent from transmission to transmission. A simplified model of a channel with memory is shown in Figure 1.8. This model contains two states, a "good state," in which transmission errors occur infrequently, Pl ~ 0, and a "bad state," in which transmission errors are probable, P2 ~ .5. The channel is in the good state most of the time but on occasion shifts to the bad state owing to a change in the transmission characteristic of the channel, for example, a "deep fade" caused by multipath transmission. As a consequence, transmission errors occur in dusters or bursts because of the high transition probability in the bad state, and channels with memory are called bursterror channels. Examples of burst-error channels are mobile telephony channels, where the error bursts are caused by signal fading owing to multipath transmission; cable transmission, which is affected by impulsive switching noise and crosstalk; and magnetic recording, which is subject to dropouts caused by surface defects and dust partides. Codes designed to correct burst errors are called burst-error correcting codes. Chapters 20 and 21 are devoted to codes of this type and coding techniques for correcting burst errors.

14

Chapter 1

1-q,

Coding for Reliable Digital Transmission and Storage

C, State s 1

0

State s2

0

0

1

FIGURE 1.8: A

simplified model of a channel with memory.

Finally, some channels contain a combination of both random and burst errors. These are called compound channels, and codes designed to correct errors on these channels are called burst-and-random-error correcting codes. Sections 20.5 and 21.4 are devoted to codes of this type. 1.6

ERROR CONTROL STRATEGIES

The block diagram shown in Figure 1.1 represents a one-way communication system. The transmission (or recording) is strictly in one direction, from transmitter to receiver. Error control strategies for a one-way system must use forward error correction (FEC); that is, they employ error-correcting codes that automatically correct errors detected at the receiver. Examples are digital storage systems, in which the information recorded on a storage medium may be replayed weeks or even months after it is recorded; and deep-space communication systems, where the relatively simple encoding equipment can be placed aboard the spacecraft, but the much more complex decoding procedure must be performed on Earth. Most of the coded systems in use today employ some form of FEC, even if the channel is not strictly one-way. This book is devoted mostly to the analysis and design of FEC systems. In some cases, though, a transmission system can be two-way; that is, information can be sent in both directions, and the transmitter also acts as a receiver (a transceiver), and vice versa. Examples of two-way systems are some data networks and satellite communication systems. Error control strategies for a two-way system use error detection and retransmission, called automatic repeat request (ARQ). In an ARQ system, when errors are detected at the receiver, a request is sent for the

Section 1.7

Performance Measures

15

transmitter to repeat the message, and repeat requests continue to be sent until the message is correctly received. There are two types of ARQ systems: stop-and-wait ARQ and continuous ARQ. With stop-and-wait ARQ the transmitter sends a codeword to the receiver and waits for a positive (ACK) or negative (NAK) acknowledgment from the receiver. Han ACK is received (no errors detected), the transmitter sends the next detected), the transmitter resends codeword; however, if a NAK is received the same codeword may be the previous codeword. When the noise is received and retransmitted several times before it is With continuous ARQ, the transmitter sends codewords to the receiver continuously and receives acknowledgments continuously. When a NAK is received, the transmitter begins a retransmission. H may back up to the codeword in error and resend that codeword plus the N - 1 codewords that follow it. This is called go-back-N ARQ. Alternatively, the transmitter may simply resend only those codewords that are negatively acknowledged. This is known as selective-repeat ARQ. Selective-repeat ARQ is more efficient than go-back-N ARQ, but requires more logic and buffering. Continuous ARQ is more efficient than stop-and-wait ARQ, but it is also more expensive to implement. In a satellite communication system, where the transmission rate is high and the round-trip delay is long, continuous ARQ is normally used. Stop-and-wait ARQ is used in systems where the time taken to transmit a codeword is long compared to the time taken to receive an acknowledgment. Stop-and-wait ARQ is designed for use on half-duplex channels (only one-way transmission at a time), whereas continuous ARQ is designed for use on full-duplex channels (simultaneous two-way transmission). The major advantage of ARQ over FEC is that error detection requires much simpler decoding equipment than error correction. Also, ARQ is adaptive in the sense that information is retransmitted only when errors occur. In contrast, when the channel error rate is high, retransmissions must be sent too frequently, and the system throughput-the rate at which newly generated messages are correctly received-is lowered by ARQ. In this situation, a hybrid combination of FEC for the most frequent error patterns along with error detection and retransmission for the less likely error patterns is more efficient than ARQ alone. Various types of ARQ and hybrid ARQ schemes are presented in Chapter 22.

The performance of a coded communication system is in general measured by its probability of decoding error (called the error probability) and its coding gain over an uncoded system that transmits information at the same rate. There are two types of error probability, probability of word (or block) error and probability of bit error. The probability of word error is defined as the probability that a decoded word ( or block) at the output of the decoder is in error. This error probability is commonly called the word-error rate (WER) or block-error rate (BLER). The probability of bit error, also called the bit-error rate (BER), is defined as the probability that a decoded information bit at the output of the decoder is in error. A coded communication system should be designed to keep these two error probabilities as

16

Chapter 1

Coding for Reliable Digital Transmission and Storage

low as possible under certain system constraints, such as power, bandwidth, and decoding complexity. In a coded communication system with code rate R = k/n, the number of transmitted symbols (or signals) per information bit is 1/R. If the energy per transmitted symbol is Es, then the energy-per-information bit is (1.15) The error probability of a coded communication system is commonly expressed in terms of the ratio of energy-per-information bit Eb to the one-sided PSD No of the channel noise. Consider a coded communication system using the (23,12) binary Golay code [8] for error control (see Chapter 5 for a complete description of this code). Each codeword consists of 23 code digits, of which 12 are information bits. Therefore, there are 11 redundant bits, and the code rate is R = 12/23 = 0.5217. Suppose BPSK modulation with coherent detection is used and the channel is an A WGN channel with one-sided PSD No. Let Eb/ No denote the ratio of energy-per-information-bit to noise power spectral density at the input to the receiver, which is called the signal-to-noise ratio (SNR) and is usually expressed in decibels (dBs ). The biterror performance of the (23,12) Golay code with both hard-decision decoding and (unquantized) soft-decision maximum likelihood decoding versus Eb/ No is depicted in Figure 1.9. For comparison, the bit-error performance of an uncoded BPSK system is also included. We see that the coded system, with either hard- or softdecision decoding, provides a lower bit-error probability than the uncoded system for the same SNR when the SNR is above a certain threshold. For hard-decision decoding of the Golay code, this threshold is 3.7 dB. For SNR = 7 dB, the BER of the uncoded BPSK system is 8 x 10- 4 , whereas the coded system achieves a BER of 2.9 x 10-5 . This is a significant improvement in error performance. For SNR = 5 dB, the performance improvement of the Golay-coded system over the uncoded system is small: 2.1 x 10-3 compared to 6.5 x 10-3 ; however, with soft-decision decoding, the coded system achieves a BER of 7 x 10- 5 . The other performance measure of a coded communication system is the coding gain. Coding gain is defined as the reduction in the Eb/ No required to achieve a specific error probability (BER or WER) for a coded communication system compared to an uncoded system. For example, in Figure 1.9, for a BER = 10-5 , the Golay-coded system with hard-decision decoding has a coding gain of 2.15 dB over uncoded BPSK, whereas with soft-decision decoding (MLD), a coding gain of more than 4.0 dB is achieved. This result shows that soft-decision decoding of the Golay code achieves 1.85 dB additional coding gain compared to hard-decision decoding at a BER of 10-5 . This additional coding gain is achieved at the expense of higher decoding complexity. Coding gain is important in communication applications where every decibel of improved performance results in substantial savings in overall system cost. From Figure 1.9, we observe that at sufficiently low values of Eb/ No, the coding gain actually becomes negative. This threshold phenomenon is common to all coding schemes. There always exists an Eb/ No below which the code loses its effectiveness and actually makes the situation worse. This SNR is called the coding threshold. It is important to keep this threshold low and to maintain a coded communication

Performance Measures

Section 1.7 10-1~,I

I + : Uncoded

',, ~ 10-2

H

"''

"

BPSK

o : Hard decision decoding of the (23,12) Golay code

' "'

withBPSK

\\

,, : Soft decision \ decoding of the '\ (23,12) Golay code williBPSK

\

10- 6 '--~-'--~-'--~-'--~-'-~-'-~-'-~-4'!-~-'-~-'-~~ 1 2 3 4 5 6 7 8 9 10 11 Ez,IN0 (dB)

FIGURE 1.9: Bit-error performance of a coded communication system with the

(23,12)

Golay code. system operating at an SNR well above its coding threshold. For the Golay-coded system with hard-decision decoding, the coding threshold is 3.7 dB. Another quantity that is sometimes used as a performance measure for a particular code is the asymptotic coding gain (the coding gain for large SNR). This coding gain depends only on the code rate and the minimum distance dmin of the code, which is defined as the minimum Hamming distance between any two codewords. For an A WGN channel, it follows from (1.4) and (1.5) that for high SNR (large Eb/ No), the bit-error probability of an uncoded BPSK system with coherent detection can be approximated by (1.16) For high SNR with soft-decision (unquantized) MLD, the bit error probability of an (n, k) block code with minimum distance dmin can be approximated by (see Chapter 10) (1.17) where R = k/n is the code rate, and K is a (typically small) constant that depends only on the code, One may compute the reduction in SNR by equating (1.16) and (1.17), which gives (1.18) Taking the logarithm (base e) of both sides and noting that log K and log negligible for large Eb/ No, we obtain (Eb/ No)uncoded (Eb/ No)coded

_ -

Rd . mm·

! are (1.19)

18

Chapter 1

Coding for Reliable Digital Transmission and Storage

Thus, for soft-decision MLD, the asymptotic coding gain of an (n, k) code with minimum distance dmin over uncoded BPSK is given by Yasymp =

10 loglO

(1.20)

Rdmin.

For hard-decision MLD, it can be shown [7] that the asymptotic coding gain is Yasymp

= 10 loglO ( i) Rdmin·

(1.21)

From (1.20) and (1.21), we see that soft-decision MLD has a 10log 10 2 = 3 dB asymptotic coding gain advantage over hard-decision MLD. This 3 dB coding gain is achieved only for very large SNR. For more realistic values of SNR, the real coding gain of soft-decision decoding over hard-decision decoding is less than 3 dB, typically between 2 and 2.5 dB. Expressions similar to (1.20) and (1.21) are given in Chapter 12 for convolutional codes, with the minimum free distance dtree replacing dmin. In designing a coding system for error control, it is desired to minimize the SNR required to achieve a specific error rate. This is equivalent to maximizing the coding gain of the coded system compared to an uncoded system using the same modulation signal set. A theoretical limit on the minimum SNR required for a coded system with code rate R to achieve error-free communication (or an arbitrarily small error probability) can be derived based on Shannon's noisy coding theorem [1 ]. This theoretical limit (often called the Shannon limit) simply says that for a coded system with code rate R, error-free commutation is achievable only if the SNR exceeds this

0.9 0.8 0.7 ~ (1)

0.6

~

~ 0.5

"O 0

U 0.4 0.3 0.2 0.1 0 -5

FIGURE 1.10:

0 5 Shannon limit (E1,IN0 (dB))

Shannon limit as a function of code rate R.

10

Performance Measures

Section 1.7

19!

limit. As long as the SNR exceeds this Shannon's coding theorem guarantees the existence of a (perhaps very complex) coded system of achieving error-free communication. For transmission over a binary-input, continuous-output A WGN channel with BPSK signaling, the Shannon limit (in terms of Eb/ No) as a function of code rate R does not have a closed however, it can be evaluated numerically. Figure 1.10 shows the Shannon limit as a function of the code rate R for BPSK signaling on a continuous-output A WGN channeL From this figure we can determine the Shannon limit for a given code rate. For example, to achieve errorfree communication with a coded system of rate R = over a Shannon A WGN channel using BPSK signaling, the minimum required SNR limit) is 0.188 dB. Table 1.2 gives the values of the Shannon limit for various code rates from Oto 0.999. The Shannon limit can be used as a to measure the maximum achievable coding gain for a coded system with a given rate R over an uncoded system with the same modulation signal set. For example, to achieve a BER of 10- 5 , an uncoded BPSK system requires an SNR of 9.65 dB. For a coded BPSK the Shannon limit is 0.188. Therefore, the maximum system with code rate R = achievable (or potential) coding gain for a coded BPSK system with a code rate R = is 9.462 dB. For example, Figure 1.11 shows the bit-error performance of a rate R = convolutional code with memory order m = 6 decoded with soft-decision MLD (see Chapter 12). To achieve a BER of 10- 5 , this code requires an SNR of 4.15 dB and achieves a 5.35 dB coding gain compared to uncoded BPSK; however, it is 3.962 dB away from the Shannon limit. This gap can be reduced using a longer and more powerful code, block or convolutional, decoded with an efficient softdecision MLD or near-MLD algorithm. As another example, consider Figure 1.12

!

!,

!

i

~

BPSK uncoded

~

-®-BER

10-1

10- 6 ~~~~~~~~~~~~~~~~~~~~--'

0

2

3

4

5

FIGURE 1. 11: Bit-error performance of a rate R = order m = 6 decoded with soft-decision MLD.

6

7

8

9

10

i convolutional code with memory

20

Chapter 1

Coding for Reliable Digital Transmission and Storage

TABLE 1.2: Shannon limit of a continuous-output A WON channel

with BPSK signaling for various code rates R. R

Eb/No

R

(dB)

0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.10 U.11

0.12 0.13 0.14 0.15 0.16 0.17 0.18 0.19 0.20 0.21 0.22 0.23 0.24 0.25 0.26 0.27 0.28 0.29 0.30 0.31 0.32 0.33 0.34

-1.548 -1.531 -1.500 -1.470 -1.440 -1.409 -1.378 -1.347 -1.316 -1.285 -1.254 -1.222 -1.190 -1.158 -1.126 -1.094 -1.061 -1.028 -0.995 -0.963 -0.928 -0.896 -0.861 -0.827 -0.793 -0.757 -0.724 -0.687 -0.651 -0.616 -0.579 -0.544 -0.507 -0.469

Eb/No

R

(dB)

0.35 0.36 0.37 0.38 0.39 0.40 0.41 0.42 0.42 0.44 0.45 0.46 0.47 0.48 0.49 0.50 0.51 0.52 0.53 0.54 0.55 0.56 0.57 0.58 0.59 0.60 0.61 0.62 0.63 0.64 0.65 0.66 0.67 0.68

-0.432 -0.394 -0.355 -0.314 -0.276 -0.236 -0.198 -0.156 -0.118 -0.074 -0.032 0.010 0.055 0.097 0.144 0.188 0.233 0.279 0.326 0.374 0.424 0.474 0.526 0.574 0.628 0.682 0.734 0.791 0.844 0.904 0.960 1.021 1.084 1.143

Eb/No

R

(dB)

0.69 0.70 0.71 0.72 0.73 0.74 0.75 0.76 0.77 0.78 0.79 0.800 0.807 0.817 0.827 0.837 0.846 0.855 0.864 0.872 0.880 0.887 0.894 0.900 0.907 0.913 0.918 0.924 0.929 0.934 0.938 0.943 0.947 0.951

1.208 1.275 1.343 1.412 1.483 1.554 1.628 1.708 1.784 1.867 1.952 2.045 2.108 2.204 2.302 2.402 2.503 2.600 2.712 2.812 2.913 3.009 3.114 3.205 3.312 3.414 3.500 3.612 3.709 3.815 3.906 4.014 4.115 4.218

Eb/No (dB)

0.954 0.958 0.961 0.964 0.967 0.970 0.972 0.974 0.976 0.978 0.980 0.982 0.983 0.984 0.985 0.986 0.987 0.988 0.989 0.990 0.991 0.992 0.993 0.994 0.995 0.996 0.997 0.998 0.999

4.304 4.425 4.521 4.618 4.725 4.841 4.922 5.004 5.104 5.196 5.307 5.418 5.484 5.549 5.615 5.681 5.756 5.842 5.927 6.023 6.119 6.234 6.360 6.495 6.651 6.837 7.072 7.378 7.864

which depicts the bit-error performance of a (65520,61425) low-density parity-check code (see Chapter 17) [12] decoded with a near-MLD soft-decision algorithm. The rate of this code is R = 15 /16 = 0.9375. The Shannon limit for this rate is 3.91 dB. At the BER of 10-5 , we find that the error performance of this code is less than 0.5 dB away from the Shannon limit. This example demonstrates that the Shannon limit can be approached with a properly designed code of sufficient length decoded with an

Section 1.8

~

Coded Modulation

21

- BPSK uncoded

-BER

Shannon Limit

E1,IN0 (dB)

FIGURE 1.12: Bit-error performance of a (65520,61425) low-density parity check code decoded with a soft-decision near-MLD algorithm.

efficient soft-decision MLD or near-MLD algorithm. Shannon limit-approaching codes are discussed in Chapters 16 and 17.

Combining coding with binary modulation to achieve coding gain (or to improve error performance) results in channel bandwidth expansion; that the coded system requires a larger channel bandwidth for signal transmission than the uncoded system. This is because redundant symbols must be added to the transmitted information sequence to combat the channel noise (i.e., for error control). As shown in Section 1.3, to maintain the same information transmission rate using a code with rate R = k / n < 1 for error control requires bandwidth expansion by a factor of 1/ R. Therefore, coding gain is achieved at the expense of bandwidth expansion. For example, the Golay-coded system with BPSK modulation described in the previous sectionrequires bandwidth expansion by a factor ofl/ R = 23/12 = 1.917. Therefore, coding gain is achieved at a cost of almost doubling the required bandwidth of the transmitted signal, plus the additional cost of decoder implementation. This type of coding provides an effective trade-off between channel bandwidth and transmitter power and is suitable for digital communication systems that are designed to operate on power-limited channels where R/ VV < 1 [7], such as deep-space, satellite, and other wideband communication systems. For many communication channels, however, such as voiceband telephone, terrestrial microwave, and some satellite channels, bandwidth is limited, and hence bandwidth expansion is not desirable or even possible. For this reason, coding by adding extra redundant symbols for error control is rarely used on

22

Chapter 1

Coding for Reliable Digital Transmission and Storage

bandlimited channels. To overcome this problem, coding must be used in conjunction with bandwidth-efficient multilevel modulation (M-ary modulation with M > 2). If coding and modulation are combined properly and designed as a single entity, coding gain can be achieved without bandwidth expansion [9]. Such a combination of coding and modulation is called coded modulation. The first coded modulation technique, known as trellis coded modulation, was devised by Ungerboeck [10]. The basic concept of coded modulation is to encode information symbols onto an expanded modulation signal set (relative to that needed for uncoded modulation). This signal set expansion provides the needed redundancy for error control without increasing the signaling rate and hence without increasing the bandwidth requirement. Coding is used to form structured signal sequences with large minimum Euclidean distance between any two transmitted signal sequences. For example, consider the simple coded modulation system shown in Figure 1.13, in which a rate R = feedback convolutional encoder with memory order m = 2 and an 8-PSK modulation signal set are used. During each unit of time, the two output code bits of the convolutional encoder together with an uncoded information bit are used to select one 8-PSK signal (properly known as mapping by set partitioning [10]) for transmission. As a result, each 8-PSK signal carries two information bits, and hence the information transmission rate (called the spectral efficiency) for this coded modulation system is 2 bits per transmitted signal. Now, consider an uncoded QPSK (or 4-PSK) system. With this system, information is also transmitted at a rate of 2 bits per transmitted signal. Therefore, the coded 8-PSK and uncoded QPSK systems both have the same spectral efficiency, 2 bits per transmitted signal, and thus they require the same bandwidth for signal transmission. Ungerboeck showed that this coded 8-PSK system with soft-decision MLD achieves a 3 dB asymptotic coding gain over an uncoded QPSK system without bandwidth expansion [10, 11; also see Chapter 18]. This simple coded 8-PSK system can be soft-decision decoded in a straightforward manner (see Chapter 18). Its bit-error performance is shown in Figure 1.14. At the BER of 10-5 , it achieves a 2.4 dB real coding gain over the uncoded QPSK system. Coded modulation can be classified into two categories based on the code structure:

!

1. Trellis coded modulation (TCM), in which convolutional (or trellis) codes are combined with multilevel modulation signal sets, and

8-PSK signal mapping

Rate R =

Output 8-PSK signal

f convolut10nal encoder

FIGURE 1.13:

A trellis coded 8-PSK modulation system.

Bibliography

:B

o Coded 8-PSK * Uncoded QPSK

10- 7 ~~~~~~~~~~~~~~~~~~~~~

2

3

4

5

6 7 Eb/N0 (dB)

8

9

10

FIGURE 1.14: Bit error performance of the coded 8-PSK system shown in Figure 1.13. lo Block coded modulation (BCM) [13], in which block codes are combined with

multilevel modulation signal sets. TCM is the subject of Chapter 18, and BCM is discussed in Chapter 19. Basically, coding for error control is achieved by adding properly designed redundancy to the transmitted information sequence. The redundancy is obtained either by adding extra symbols to the transmitted information sequence or by channel signal-set expansion in which a larger signal set is used for mapping the transmitted information sequence into a signal sequence. Coding by adding redundant symbols results in bandwidth expansion and is suitable for error control in power-limited communication systems. Coding by channel signal set expansion aHows coding gain without bandwidth expansion and is suitable for error control in bandwidth-limited communication systems.

JL C. E. Shannon, "A Mathematical Theory of Communication," Bell Syst. Tech. I., pp. 379-423 (Part 1); pp. 623-56 (Part 2), July 1948. lo R. G. Gallager, Information Theory and Reliable Communication, John Wiley, New York, 1968.

3o T. Berger, Rate Distortion Theory, Prentice Hall, Englewood Cliffs, N.J., 1971 . .:IL R. Gray, Source Coding Theory, Kluwer Academic, Boston, Mass., 1990.

So T. Cover and J. Thomas, Elements of Information Theory, Wiley Inter-Science, New York, 1991.

24

Chapter 1

Coding for Reliable Digital Transmission and Storage

6. J. M. Wozencraft and I. M. Jacob, Principles of Communication Engineering, John Wiley, New York, 1965. 7. J. G. Proakis, Digital Communications, 4th ed., McGraw-Hill, New York, 2000. 8. M. J.E. Golay, "Notes on Digital Coding," Proc. IRE, p. 657, June 1949. 9. J. L. Massey, "Coding and Modulation in Digital Communications," in Proc. 1974 Int. Zurich Seminar on Digital Communications, Zurich, Switzerland, March 1974. lit G. Ungerboeck, "Channel Coding with Multilevel/Phase Signals," IEEE Trans. Inform. Theory, IT-28 (no. 1): 55-67, January 1982.

11.

, "Trellis-Coded Modulation with Redundant Signal Sets: Parts I and II," IEEE Commun. Mag., 25: 5-21, February 1987.

12. R. G. Gallager, "Low Density Parity Check Codes," IRE Trans. Inform. Theory, IT-8: 21-28, January 1962. 13. H. Imai and S. Hirakawa, "A New Multilevel Coding Method Using ErrorCorrecting Codes," IEEE Trans. Inform. Theory, 23 (no. 3): 371-76, May 1977.

The purpose of this chapter is to provide the reader with an ~·-"""'"''~' of algebra that win aid in the understanding of the material in the The treatment is basically descriptive, and no is made to be rigorous. There are many good textbooks on algebra. The reader who is interested in more advance algebraic coding theory is referred to the textbooks listed at the end of the chapter.

Let G be a set of elements. A binary operation * on G is a nde that to each pair of elements a and b a uniquely defined third element c = a * b in G. When such a binary operation* is defined on G, we say that G is closed under*· For example, let G be the set of all integers and let the binary operation on G be real addition+. We all know that, for any two integers i and j in G, i + j is a uniquely defined integer in G. Hence, the set of integers is dosed under real addition. A binary operation * on G is said to be associative if, for any a, b, and c in G, a* (b * c) =(a* b)

* c.

Now, we introduce a useful algebraic system called a group. ]D)JEJFilNnmN 2Jl A set G on which a binary operation * is defined is called a group if the following conditions are satisfied:

ii,

The binary operation* is associative.

ilii, G contains an element e such that, for any a in G,

a*e=e*a=a.

This element e is called an identity element of G. iinii, For any element a in G, there exists another element a' in G such that a* a'= a'* a= e.

The element a' is called an inverse of a (a is also an inverse of a'). A group G is said to be commutative if its binary operation * also satisfies the following condition: For any a and b in G, a* b = b l'l!lIJEl(J)JJUM

ZJL

* a.

The identity element in a group G is unique.

26

Chapter 2

Introduction to Algebra

Proof: Suppose that there exist two identity elements e and e' in G. Then e' = e' * e = e. This implies that e and e' are identical. Therefore, there is one and only one identity element. Q.E.D. TlH!JEORJEM

2.2 The inverse of a group element is unique.

Proof: Suppose that there exist two inverses a' and a" for a group element a. Then a'= a'* e =a'* (a* a") =(a'* a)* a"= e *a"= a".

This implies that a' and a" are identical and there is only one inverse for a. Q.E.D. The set of all integers is a commutative group under real addition. In this case, the integer O is the identity element, and the integer -i is the inverse of integer i. The set of all rational numbers excluding zero is a commutative group under real multiplication. The integer 1 is the identity element with respect to real multiplication, and the rational number b/a is the multiplicative inverse of a/b. The groups just noted contain infinite numbers of elements. Groups with finite numbers of elements do exist, as we shall see in the next example. IEXAMIP'LIE 2.1

Consider the set of two integers G by E9, on G as follows:

= {O, 1}. Let us define a binary operation, denoted

0 E91 = 1,

1E90

= 1,

1E91

= 0.

This binary operation is called modulo-2 addition. The set G = {O, 1} is a group under modulo-2 addition. It follows from the definition of modulo-2 addition E9 that G is closed under E9, and E9 is commutative. We can easily check that E9 is associative. The element O is the identity element. The inverse of Ois itself, and the inverse of 1 is also itself. Thus, G together with E9 is a commutative group. The number of elements in a group is called the order of the group. A group of finite order is called a finite group. For any positive integer m, it is possible to construct a group of order m under a binary operation that is very similar to real addition, as is shown in the next example.

IEXAMPllE 2.2 Let m be a positive integer. Consider the set of integers G = {O, 1, 2, · · · , m - l}. Let + denote real addition. Define a binary operation [±J on G as follows: For any integers i and j in G, i [±J j = r, where r is the remainder resulting from dividing i + j by m. The remainder r is an integer between O and m - 1 (Euclid's division algorithm) and is therefore in G. Hence, G is closed under the binary operation [±J , which is called modulo-m

Section 2.1

Groups

27!

addition. The set G = {O, 1, · · · , m - 1} is a group under modulo-m addition. First, we see that O is the element. For O < i < m, i and m - i are both in G. Since

+ (m

i

- i) = (m - i)

+i

= m,

it follows from the definition of modulo-m addition that i El (m - i)

=

(m - i)

Eli = 0.

Therefore, i and m - i are inverses of each other with respect to El. His also clear that the inverse of O is itself. Because real addition is commutative, it follows from the definition of modulo-m addition that, for any i and j in i El j = j Eli. Therefore, modulo-m addition is commutative. Next, we show that modu.lo-m addition is also associative. Let i, j, and k be three integers in G. Since real addition is associative, we have i + j + k = (i + j) + k = i + (j + k). Dividing i

+ j + k by m, we obtain i

+ j + k = qm + r,

where q and r are the quotient and the remainder, respectively, and O < Now, dividing i + j by m, we have

r < m.

(2.1) with O::: r1 < m. Therefore, i El j = r1. Dividing r1

+ k by m, we obtain (2.2)

with O :S r2 < m. Hence, r1 Elk= r2, and

Combining (2.1) and (2.2), we have

This implies that r2 is also the remainder when i + j + k is divided by m. Because the remainder resulting from dividing an integer by another integer is unique, we must have r2 = r. As a result, we have (i Elj) Elk= r.

Similarly, we can show that i El (j Elk)

= r.

Therefore, (i El j) Elk = i El (j Elk), and modulo-m addition is associative. This concludes our proof that the set G = {O, 1, 2, · · · , m -1} is a group under modulo-m addition. We shall call this group an additive group. Form = 2, we obtain the binary group given in Example 2.1.

28

Chapter 2

Introduction to Algebra TABLE 2.1: Modulo-5 addition. [±]

0

0 1 2 3 4

4

1

2

3

4

0

1

1 2 3

2 3 4 0

2 3 4 0

3 4 0

4 0

1 2

1

1 2 3

The additive group under modulo-5 addition is given in Table 2.1. Finite groups with a binary operation similar to real multiplication also can be constructed. EXAMPLE 2.3

Let p be a prime ( e.g., p = 2, 3, 5, 7, 11, · · · ). Consider the set of integers, G = {l, 2, 3, · · · , p -- l}. Let· denote real multiplication. Define a binary operation Don G as follows: For i and j in G, i D j

= r,

where r is the remainder resulting from dividing i . j by p. First, we note that i . j is not divisible by p. Hence, 0 < r < p, and r is an element in G. Therefore, the set G is closed under the binary operation D, which is referred to as modulo-p multiplication. The set G = {l, 2, · .. , p - 1} is a group under modulo-p multiplication. We can easily check that modulo- p multiplication is commutative and associative. The identity element is 1. The only thing left to be proved is that every element in G has an inverse. Let i be an element in G. Because pis a prime, and i < p, i and p must be relatively prime (i.e., i and p do not have any common factor greater than 1). It is well known that there exist two integers a and b such that a-i+b·p=l

(2.3)

and a and p are relatively prime (Euclid's theorem). Rearranging (2.3), we have a ·i

=

-b · p

+ 1.

(2.4)

This says that when a· i is divided by p, the remainder is 1. If O < a < p, a is in G, and it follows from (2.4) and the definition of modulo-p multiplication that aDi=iDa=l. Therefore, a is the inverse of i. However, if a is not in G, we divide a by p, a= q · p

+ r.

(2.5)

Because a and p are relatively prime, the remainder r cannot be 0, and r must be between 1 and p - l. Therefore, r is in G. Now, combining (2.4) and (2.5), we obtain r·i=-(b+qi)p+l.

Section 2. 1 TABLE 2.2:

Groups

2~

Modulo-5 multiplication.

D

]_

z

3

jJ

1 2 3 4

1 2 3 4

2 4 1 3

3 1 4 2

4 3 2

1

Therefore, r D i = i D r = 1 and r is the inverse of i. Hence, any element i in G has an inverse with respect to modulo-p multiplication. The group G = {1, 2, ... , p -1} under modulo-p multiplication is called a multiplicative group. For p = 2, we obtain a group G = {1} with only one element under modulo-2 multiplication. Up is not a prime, the set G = {1, 2, ... , p -1} is not a group under modulo-p multiplication (see Problem 2.3). Table 22 illustrates the group G = {1, 2, 3, 4} under modulo-5 multiplication. Let H be a nonempty subset of G. The subset H is said to be a subgroup of G if H is closed under the group operation of G and satisfies all the conditions of a group. For example, the set of an rational numbers is a group under real addition. The set of all integers is a subgroup of the group of rational numbers under real addition. A subgroup of G that is not identical to G is called a proper subgroup of G.

Zo3 Let G be a group under the binary operation *· Let H be a nonempty subset of G. Then H is a subgroup of G if the following conditions hold:

'fll-lllEl[])JR?.JEMI

no H is dosed under the binary operation *· the inverse of a is also in H.

nt For any element a in

!P1rl[}l[}t Condition (ii) says that every element of H has an inverse in H. Conditions (i) and (ii) ensure that the identity element of G is also in H. Because the elements in H are elements in G, the associative condition on * holds automatically. Hence, H satisfies all the conditions of a group and is a QJEJl])o subgroup of G.

ZoZ Let H be a subgroup of a group G with binary operation *· Let a be an element of G. Then the set of elements a* H ~ {a* h : h E H} is called a left coset of H; the set of elements H * a ~ {h * a : h E H} is called a right cos et of H.

]J])JEJFJINTirmN

H is dear that if the group G is commutative, is identical to every right coset H * a; that is, a * H this text, we are primarily interested in commutative further distinction between left and right cosets. We cosets.

then every left coset a * H = H * a for any a E G. In groups, so, we will make no will simply refer to them as

30

Chapter 2

Introduction to Algebra

EXAMPLE 2.4

Consider the additive group G = {O, 1, 2, · · · , 15} under modulo-16 addition. We can readily check that H = {O, 4, 8, 12} forms a subgroup of G. The coset 3 [±J His 3 [±J H = {3 [±J 0, 3 [±J 4, 3 [±J 8, 3 [±J 12}

=

{3, 7, 11, 15}.

The coset 7 [±J H is 7[±JH = {7l±JO, 7G:J4, 7G:J8, 7G:J12}

=

{7, 11, 15, 3}.

We find that 3 [±J H = 7 [±J H. There are only four distinct cosets of H. Besides 3 [±J H, the other three distinct cosets are 0 [±J H 1 [±J H

2 [±J H

= {O, 4, 8, 12}, = {1, 5, 9, 13}, = {2, 6, 10, 14}.

The four distinct cosets of H are disjoint, and their union forms the entire group G. In the following theorems, we prove some important properties of cosets of a subgroup of a group. 2.4 Let H be a subgroup of a group G with binary operation *· No two elements in a coset of H are identical.

THEOREM

Proof. The proof is based on the fact that all the elements in the subgroup H are distinct. Consider the coset a* H = {a* h : h E H} with a E G. Suppose two elements, say a * h and a * h', in a * H are identical, where h and h' are two distinct elements in H. Let a- 1 denote the inverse of a with respect to the binary operation *· Then, a-l

* (a* h) = a-l * (a* h'),

(a- 1 *a)* h

e *h h

=

(a- 1 *a)* h 1 ,

= e * h', = h'.

This result is a contradiction to the fact that all the elements of H are distinct. Therefore, no two elements in a coset are identical. Q.E.D. THEOREM 2.5 No two elements in two different cosets of a subgroup Hof a group G are identical.

Section 2.2

Fields

31

with a and bin G. lP'1Ymf, Let a * H and b * H be two distinct cosets of respectively. Suppose Let a * h and b * h' be two elements in a * H and b * a* h = b * h'. Let h- 1 be the inverse of h. Then

* h- 1 = (b * h') * 1i-1 , 1 1 a* (h * h- ) = b * (h' * h- ), a* e = b * h", (a* h)

where h" = h'

* h- 1 is an element in H. The equality a = b * h" a

that

* H = (b * h") * H, =

{(b

* h") * h: h EH},

* (h" * h) : h E H}, = {b * h h E H},

= {b

111

111

:

=h*H.

This result says that a * H and b * H are identical, which is a contradiction to the given condition that a* H and b * H are two distinct cosets of H. Therefore, (Ql,lEJO>o no two elements in two distinct cosets of H are identical. From Theorems 2.4 and 2.5, we obtain the following properties of cosets of a subgroup H of a group G: ii, Every element in G appears in one and only one coset of H; iiii, All the distinct cosets of H are disjoint; and

iiniio The union of all the distinct cosets of H forms the group G.

Based on the preceding structural properties of cosets, we say that all the distinct cosets of a subgroup H of a group G form a partition of denoted by G / H. ZAii (LA
TlH!IEOllUEMI

lP'mifJf H follows from Theorem 2.4 that every coset of H consists of m elements of G. Let i be the number of distinct cosets of H. Then, it follows from the preceding structural properties of cosets that n = i · m. Therefore, m divides n, and i = n/m. (Ql,lEJill,

Now, we use group concepts to introduce another algebraic system, called a field. Roughly speaking, a field is a set of elements in which we can perform addition, subtraction, multiplication, and division without leaving the set. Addition and multiplication must satisfy the commutative, associative, and distributive laws. A formal definition of a field is given next

32

Chapter 2

Introduction to Algebra

2.3 Let F be a set of elements on which two binary operations, called addition "+" and multiplication "· ", are defined. The set F together with the two binary operations + and · is a field if the following conditions are satisfied:

DlEFKI'fffWN

i. F is a commutative group under addition +. The identity element with respect to addition is called the zero element or the additive identity of F and is denoted by 0. ii. The set of nonzero elements in F is a commutative group under multiplication ·. The identity element with respect to multiplication is called the unit element or the multiplicative identity of F and is denoted by 1. m. Multiplication is distributive over addition; that is, for any three elements a,b,andcinF, a · (b + c) = a · b + a · c.

It follows from the definition that a field consists of at least two elements, the additive identity and the multiplicative identity. Later, we will show that a field of two elements does exist. The number of elements in a field is called the order of the field. A field with a finite number of elements is called a finite field. In a field, the additive inverse of an element a is denoted by -a, and the multiplicative inverse of a is denoted by a-1, provided that a =I= 0. Subtracting a field element b from another field element a is defined as adding the additive inverse, -b, of b to a [i.e., a - b ~ a + (-b)]. If b is a nonzero element, dividing a by b is defined as multiplying a by the multiplicative inverse, b- 1 , of b [i.e., a-;- b ~ a . b- 1 ]. A number of basic properties of fields can be derived from the definition of a field.

Pmperty I

For every element a in a field, a· 0

= 0 ·a= 0.

Proof First, we note that a

= a · 1 = a · (1 + 0) = a + a · 0.

Adding -a to both sides of the preceding equality, we have -a

+a =

-a

+a +a ·0

0=0+a·O O=a -0.

Similarly, we can show that O · a

= 0. Therefore, we obtain

Property II

a ·0

= 0·a

=

Q.E.D.

0.

For any two nonzero elements a and bin a field, a · b =I= 0.

Pmof This is a direct consequence of the fact that the nonzero elements of a field are closed under multiplication. Q.E.D. Property III

a · b = 0 and a =I= 0 imply that b = 0.

Fields

Section 2.2

3l3l

!Prt@@fi This is a direct consequence of Property H.

!Pmpert1ty JfV

For any two elements a and bin a field, -(a·b) = (-a)·b=a·(-b).

!Pm@fi O = 0 · b =(a+ (-a))· b =a· b +(-a)· b. Therefore, (-a)· b must be the additive inverse of a· b, and -(a· b) =(-a)· b. we can prove that -(a· b) =a· (-b). IQ)o!Eol]J)o

!Pmperrty V

For a =f- 0, a · b

= a · c implies that b = c.

!Pm@f, Because a is a nonzero element in the field, it has a multiplicative inverse, a- 1 . Multiplying both sides of a · b = a · c by a- 1 , we obtain a-l ·(a· b)

= a- 1 ·(a· c)

(a- 1 · a) · b = (a- 1 · a) · c

1 · b = 1 · c.

Thus, b

= c.

We can readily verify that the set of real numbers is a field under real-number addition and multiplication. This field has an infinite number of elements. Fields with finite number of elements can be constructed and are illustrated in the next two examples and in Section 2.4.

IEKAMl?'llE 2.Si Consider the set {O, 1} together with modulo-2 addition and multiplication, defined in Tables 2.3 and 2.4. fo Example 2.1 we showed that {O, 1} is a commutative group under modulo-2 addition; and in Example 2.3, we showed that {1} is a group under modulo-2 multiplication. We can easily check that modulo-2 multiplication is distributive over modulo-2 addition by simply computing a . (b + c) and a . b + a · c for eight possible combinations of a, b and c(a = 0 or 1, b = 0 or 1, and c = 0 or 1). Therefore, the set {O, 1} is a field of two elements under modulo-2 addition and modulo-2 multiplication. The field given in Example 2.5 is usually called a binary field and is denoted by GF(2). The binary field GF(2) plays an important role in coding theory and is widely used in digital computers and digital data transmission ( or storage) systems. TABLE

2.3: Modulo-2 addition.

+ 0 1

TABLE

2.4: Modulo-2 multiplication.

]_

0 1

1 0

]_

0

1

0 0

0 1

34

Chapter 2

Introduction to Algebra

!EXAMPLE 2.6

Let p be a prime. We showed in Example 2.2 thatthesetofintegers {O, 1, 2, · · · , p-1} is a commutative group under modulo-p addition. We also showed in Example 2.3 that the nonzero elements {1, 2, · ·. , p - l} form a commutative group under modulo-p multiplication. Following the definitions of modulo-p addition and multiplication and the fact that real-number multiplication is distributive over real-number addition, we can show that modulo-p multiplication is distributive over modulo-p addition. Therefore, the set {O, 1, 2, · ·. , p - 1} is a field of order p under modulo- p addition and multiplication. Because this field is constructed from a prime, p, it is called a prime field and is denoted by GF(p). For p = 2, we obtain the binary field GF(2). Let p = 7. Modulo-7 addition and multiplication are given by Tables 2.5 and 2.6, respectively. The set of integers {O, 1, 2, 3, 4, 5, 6} is a field of seven elements, denoted by GF(7), under modulo-7 addition and multiplication. The addition table is also used for subtraction. For example, if we want to subtract 6 from 3, we first use the addition table to find the additive inverse of 6, which is 1. Then we add 1 to 3 to obtain the result [i.e., 3 - 6 = 3 + (-6) = 3 + 1 = 4]. For division, we use the multiplication table. Suppose that we divide 3 by 2. We first find the multiplicative inverse of 2, which is 4, and then we multiply 3 by 4 to obtain the result [i.e., 3 + 2 = 3 . (2- 1 ) = 3 . 4 = 5]. Here we have demonstrated that in a finite field, addition, subtraction, multiplication, and division can be carried out much like ordinary arithmetic, with which we are quite familiar. In Example 2.6, we showed that, for any prime p, there exists a finite field of p elements. In fact, for any positive integer m, it is possible to extend the prime field G F(p) to a field of p 111 elements, which is called an extension field of G F(p) and is denoted by GF(p111 ). Furthermore, it has been proved that the order of any finite field is a power of a prime. Finite fields are also called Galois fields, in honor of their discoverer. A large portion of algebraic coding theory, code construction, and decoding is built around finite fields. In the rest of this section and in the next two sections we examine some basic structures of finite fields, their arithmetic, and the construction of extension fields from prime fields. Our presentation is mainly descriptive, and no attempt is made to be mathematically rigorous. Because finitefield arithmetic is very similar to ordinary arithmetic, most of the rules of ordinary TABLE

TABLE

2.5: Modulo-7 addition.

+

0

1

2

3

41

5

6

1 2 3 4 5 6

1 2 3 4 5 6

2 3 4 5 6 0

3 4 5 6 0 1

4 5 6 0 1 2

5 6 0 1 2 3

6 0 1 2 3 4

0 1 2 3 4 5

0 1 2 3 4 5 6

2.6: Modulo-7 multiplication.

0

1

2

3

41

5

6

0 0 0 0 0 0 0

0 1 2 3 4 5 6

0 2 4 6 1 3 5

0 3 6 2 5 1 4

0 4 1 5 2 6 3

0 5 3 1 6 4 2

0 6 5 4 3 2 1

Section 2.2

Fields

35

arithmetic apply to finite-field arithmetic. Therefore, it is possible to utilize most of the techniques of algebra in the computations over finite fields. Consider a finite field of q elements, GF(q ). Let us form the following sequence of sums of the unit element 1 in GF(q ): 3

2

1

2) =

1,

i=l

2) = 1 + 1 + 1, ... '

2) = 1+1,

i=l

i=l

k

L 1 = 1 + 1 + · · · + l(k times), ... i=l

Because the field is closed under addition, these sums must be elements in the field; and because the field has finite number of elements, these sums cannot be all distinct. Therefore, at some point in the sequence of sums, there must be a repetition; that is, there must exist two positive integers m and n such that m < n and m

n

:z=1

i=l

= :z=1. i=l

This equality implies that I:;'~{" 1 = 0. Therefore, there must exist a smallest positive integer 'A such that I:~= 11 = 0. This integer 'A is called the characteristic of the field GF(q ). The characteristic of the binary field GF(2) is 2, since 1 + 1 = 0. The characteristic of the prime field GF(p) is p, since I:;'= 1 1 = k i= 0 for 1 ::: k < p and I:t=l 1 = 0. 'll'IBIIEOIRIEM

2:1

The characteristic A of a finite field is prime.

]hoof. Suppose that Ais not a prime and is equal to the product of two smaller integers k and m (i.e., A = km). Because the field is dosed under multiplication,

trl . trl k

(

(/11 )

)

is also a field element. It follows from the distributive law that

Because I:}:1 1 = 0, then either I:;'= 1 1 = 0 or I:;'~ 1 1 = 0; however, this contradicts the definition that A is the smallest positive integer such that :[~=l 1 = 0. Therefore, we conclude that A is prime. Q)JE.[l), It follows from the definition of the characteristic of a finite field that for any two distinct positive integers k and m less than A, k

111

:z=1 i= :z=1. i=l

i=l

36

Chapter 2

Introduction to Algebra

Suppose that

L7=1 1 = L;: 1 1. Then, we have m-k

I:1=0 i=l

(assuming that m > k); however, this is impossible, since m - k
ic-l

3

2

1 = I:t I:1, I:t

A

...

' I:t I:1=0 i=l i=l i=l i=l are},, distinct elements in GF(q ). In fact, this set of sums itself is a field of},, elements, GF(A), under the addition and multiplication of GF(q) (see Problem 2.7). Because GF(},,) is a subset of GF(q), GF(},,) is called a subfield of GF(q). Therefore, any finite field GF(q) of characteristic },, contains a subfield of },, elements. It can be proved that if q I= },,, then q is a power of A. Now, let a be a nonzero element in GF(q). Since the set of nonzero elements of GF(q) is closed under multiplication, the following powers of a, i=l

a

1

= a,

a

2

= a · a,

a3

= a · a · a, · · ·

must also be nonzero elements in GF(q). Because GF(q) has only a finite number of elements, the powers of a given cannot all be distinct. Therefore, at some point in the sequence of powers of a there must be a repetition; that is, there must exist two positive integers k and m such that m rel="nofollow"> k and ak =a"'. Let a- 1 be the multiplicative inverse of a. Then (a- 1 i = a-k is the multiplicative inverse of ak. Multiplying both sides of ak = a"' by a-k, we obtain 1=

am-k_

This equality implies that there must exist a smallest positive integer n such that an = 1. This integer n is called the order of the field element a. Therefore, the sequence a 1 , a 2 , a 3 , · · · repeats itself after an = 1. Also, the powers a 1 , a 2 , · · · , an-l, a" = 1 are all distinct. In fact, they form a group under the multiplication of GF(q ). First, we see that they contain the unit element 1. Consider a; . ai. If i + j :s n,

If i

+j

> n, we have i

+j

= n

+ r, where O < r :s n. Hence,

Therefore, the powers a 1 , a 2 , · · ·, a 11 -1, a 11 = 1 are closed under the multiplication of GF(q ). For 1 :S i < n, an-i is the multiplicative inverse of a;. Because the powers of a are nonzero elements in G F( q ), they satisfy the associative and commutative laws. Therefore, we conclude that a 11 = 1, a 1 , a 2 , · ·. , a 11 - 1 form a commutative group under the multiplication of GF(q ). A group is said to be cyclic if there exists an element in the group whose powers constitute the whole group. THEOREM

aq-1

= 1.

2.8

Let a be a nonzero element of a finite field GF(q). Then

I'

Section 2.3

Binary Field Arithmetic

37

/P'm({])J,, Let b1, b2, · · ·, bq-1 be the q - l nonzero elements of GF(q). Clearly, the q - 1 elements a· b1, a· b2, · · · , a· bq-l are nonzero and distinct. (a · b1) · (a · b2) · · · · · (a · bq-l)

= b1

· b2 · · · · · bq-1

aq-l · (b1 · b2 · · · · · bq-l)

= b1

· b2 · · · · · bq-1·

Since a t= 0 and (b1 · b2 · · · bq-1) f= 0, we must have aq-l = 1. Let a be a nonzero element in a finite field GF(q). Let n be the order of a. Then n divides q - 1.

TIBiiEOJR:iEMI Zo(!J)

!P'rm({])J,, Suppose that n does not divide q - 1. Dividing q - 1 by n, we obtain

q - l =kn+ r, where O < r < n. Then

Because aq-l = 1, and a 11 = 1, we must have a,. = 1. This is impossible, since 0 < r < n, and n is the smallest integer such that a" = 1. Therefore, n must divide q 1. IQJJEJil>o

fo a finite field GF(q ), a nonzero element a is said to be primitive if the order of a is q - 1. Therefore, the powers of a primitive element generate all the nonzero elements of GF(q ). Every finite field has a primitive element (see Problem 2.8). Consider the prime field GF(7) illustrated by Tables 2.5 and 2.6. The characteristic of this field is 7. If we take the powers of the integer 3 in GF(7) using the multiplication table, we obtain 31

= 3,

32

= 3 · 3 = 2,

33

= 3 · 32 = 6, 34 = 3 · 33 = 4,

35

= 3 · 34 = 5,

36

= 3 · 35 = 1.

Therefore, the order of the integer 3 is 6, and the integer 3 is a primitive element of GF(7). The powers of the integer 4 in GF(7) are 41

= 4,

42

= 4 · 4 = 2,

43

= 4 · 42 = 1.

Clearly, the order of the integer 4 is 3, which is a factor of 6.

fo general, we can construct codes with symbols from any Galois field GF(q ), where q is either a prime p or a power of p; however, codes with symbols from the binary

field GF(2) or its extension GF(2111 ) are most widely used in digital data transmission and storage systems because information in these systems is universally coded in binary form for practical reasons. In this text we are concerned only with binary codes and codes with symbols from the field GF(2 111 ). Most of the results presented in this text can be generalized to codes with symbols from any finite field GF( q)

38

Chapter 2

Introduction to Algebra

with q -f. 2 or 2111 • In this section we discuss arithmetic over the binary field GF(2), which will be used in the remainder text of this book. In binary arithmetic we use modulo-2 addition and multiplication, which are defined by Tables 2.3 and 2.4, respectively. This arithmetic is actually equivalent to ordinary arithmetic, except that we consider 2 to be equal to O (i.e., 1 + 1 = 2 = 0). Note that since 1 + 1 = 0, 1 = -1. Hence, in binary arithmetic, subtraction is the same as addition. To illustrate how the ideas of ordinary algebra can be used with binary arithmetic, we consider the following set of equations:

+ Y = 1, X + Z = 0, X

X+Y+Z=l. These can be solved by adding the first equation to the third, g1vmg Z = 0. Then, from the second equation, since Z = 0 and X + Z = 0, we obtain X = 0. From the first equation, since X = 0 and X + Y = 1, we have Y = 1. We can substitute these solutions into the original set of equations and verify that they are correct. Because we were able to solve the preceding equations, they must be linearly independent, and the determinant of the coefficients on the left side must be nonzero. If the determinant is nonzero, it must be 1. This result can be verified as follows.

1 1 0

i ~ i ·1~ il- ·li il+ ·li ~I =

1

1

0

=1-1-1-0+0·1=1. We could have solved the equations by Cramer's rule:

X=

1 0 1 1 1 1

1 0 1 1 0 1

0 1 1 0 1 1

=

0

1=

0, y =

1 1 0 1 0 1 1 1 1 1 1 0 1 0 1 1 1 1

=

1

1=

1, z =

1 1 1 1 0 0 1 1 1 1 1 0 1 0 1 1 1 1

0

-- -1 -0 - .

Next, we consider computations with polynomials whose coefficients are from the binary field GF(2). A polynomial f (X) with one variable X and with coefficients from GF(2) is of the following form:

f(X) = fo + fiX + hX 2 + · · · + f, 1 X11 , where f; = 0 or 1 for O :::: i :::: n. The degree of a polynomial is the largest power of X with a nonzero coefficient. For the preceding polynomial, if J, 1 = 1. f (X) is a polynomial of degree n; if f, 1 = 0, f(X) is a polynomial of degree less than 11. The degree of f (X) = fo is zero. In the following discussion we use the phrase "a polynomial over GF(2)" to mean "a polynomial with coefficients from GF(2)."

Section 2.3

Field Arithmetic

39J

over GF(2) with 1: X and l + X. There are four with 2: , 1 + x2. X + . and 1 + X + . In with degree n. and divided

There are two

over in the usual way. Let " - , , , , ..w

g(X) =go+ g1X

+

+···+gmX"'

be another over To add f (X) and we coefficients of the same power of X in f (X) and as follows II)

:::=

11): f(X)

+ g(X)

+go)+ U1 + g1)X + · · ·

Uo

=

.c x111+[ + ··· + .nA, f' + Jm+l Vii

where f; l +X +

+ g; + a(X)

is carried out in moclulo-2 addition. For adding and b(X) = 1 + + + + X 7 , we obtain the

+ b(X) = =

(1

+ 1) +

X

+ X2 +

2

X +X +

+

+ 1)X3 + +x1. (1

+

c1

+

V/hen we multiply f (X) and g(X). we obtain the

f

(X) · g(X)

=co+ c1X +

+···

+c11+111

where

co= for::.o. c1 = foR1

c;

Cn+m

+ figo.

=.fog;+ fig;-1

=

+ h.g;-2 + · · · + f;go.

f;,f!.111 ·

(Multiplication and addition of coefficients are modulo-2.) His clear from (2.6) that if g(X) = 0, then f(X) · 0 = 0. (2.7) Vl/e can readily verify that the polynomials over GF(2) satisfy the following conditions:

iL Commutative: a(X)

+ b(X) = b(X) + o(X).

a(X) · b(){)

= b(X)

· a(X).

40

Chapter 2

Introduction to Algebra

ii. Associative: a(X) + [b(X) + c(X)] = [a(X) + b(X)] + c(X),

a(X) · [b(X) · c(X)]

m..

= [a(X) · b(X)] · c(X).

Distributive: a(X) · [b(X) + c(X)]

= [a(X) · b(X)] + [a(X) · c(X)].

(2.8)

Suppose that the degree of g(X) is not zero. When f (X) is divided by g(X), we obtain a unique pair of polynomials over GF(2)-q(X), called the quotient, and r(X), called the remainder-such that f(X)

= q(X)g(X)

+ r(X),

and the degree of r(X) is less than that of g(X). This expression is known as Euclid's division algorithm. As an example, we divide f (X) = 1 + X + x 4 + X 5 + X 6 by g(X) = 1 + X + X3 . Using the long-division technique, we have x 3 +x 2 x3+

x + 11

x6+

x6

(quotient) x +x4 +x4+x3 5

+X+l

xs xs

+X 3 +X+1 +x3 + x2 x 2 +x+1

(remainder).

We can easily verify that

x6 +

x 5 + X4 + X + 1

= (X 3 + X2)(X 3 +

X + 1) + X 2 + X + 1.

When f (X) is divided by g(X), if the remainder r(X) is identical to zero [r(X) = O]. we say that f(X) is divisible by g(X), and g(X) is a factor of f(X). For real numbers. if a is a root of a polynomial f (X) [i.e., f (a) = OJ, f (X) is divisible by X - a. (This fact follows from Euclid's division algorithm.) This statement is still true for f (X) over GF(2). For example, let f (X) = 1 + X2 + X 3+ X 4. Substituting X = 1. we obtain f(l)

= 1 + 12 + 13 + 14 = 1 + 1 + 1 + 1 = 0.

Thus, f (X) has 1 as a root. and it should be divisible by X

x

X3 + X + 1 + 11 x 4 + x 3 + x 2 x4+x3

+ 1

x2 + 1 x 2 +x X+l X+l 0.

+ 1, as shown:

Section 2.3

For a terms, it is divisible by X to be irreducible over less than m but of 2. + 1, and however.

over GF(2), if the

Field Arithmetic "~"rn~,

+ 1. A

4\1

has an even number of m is said

X

+ X + 1 does not have either O or 1 as a root not divisible X or X + 1. Because the of degree L h cannot be divisible by a of X 3 + X + 1 is irreducible over We may + X + 1 is an irreducible of degree 4. It has been proved that for any 1n > 1 theorem next without a + X + 1 is

irreducible

'lfJHilE([)))kiEM ;),;Jj ill)

over

of

m divides

+L of Theorem 2.10, we can check that X 3

+X 2 +

X

+X +1

divides

+ 1 +1

+X+ll

+1 +1

)(3

+X+l +X+l 0.

An irreducible polynomial p(X) of degree m is said to be primitive if the smallest integer n for which p(X) divides X" + 1 is n = 2111 --- 1. We may check that + X + 1 divides x 15 + 1 but does not divide any 11 X + 1 for 1 :s n < 15. Hence, X 4 + X + 1 is a polynomial. The X4 + + + X + 1 is irreducible but n 1s not since + 1. It is not easy to recognize a primitive polynomial; however, there are tables of irreducible polynomials in which primitive polynomials are indicated [6, For a given m., there may be more than one primitive polynomial of degree m. A list of primitive polynomials is given in Table 2.7. For each degree m, we list a primitive polynomial with the smallest number of terms.

42

Chapter 2

Introduction to Algebra TABLE 2.7: List of primitive polynomials.

m 3 4 5 6 7 8 9 10 11 12 13

m 1 + X + X3 1 + X + X4 1 + x 2 + x5 1 + X + X6 1+ x3+ x7 1 + x2 + x 3 + x 4 + x 8 1 + x 4 + x9 1 + x3 + xio 1+x 2 +x 11 1 + x + x4 + x6 + x i2 1 + x + x 3 + x 4 + x 13

14 15 16 17 18 19 20 21 22 23 24

1+ 1+ 1+ 1+ 1+ 1+ 1+ 1+ 1+ 1+ 1+

+ x 6+ + x 15 + xJ + x3 + x11 x1 + xis x + x2 + x3 + x20 x2 + x21 X + X 22 xs + x23 x + x2+

x x x

x 10 + x 14 x12 + x16

xs + x19

x 7 + x 24

Before leaving this section, we derive another useful property of polynomials over GF(2). Consider J 2(X) =

Uo

+ fiX + · · · + f,,X")2

= Lfo + U1X + h.X 2 + · · · + J;,X1')] 2 =/(;+Jo· (.f1X + fzX 2 + · · · + J, X

11

1

+Jo· (.f1X + =

J5

2

)

2

fzX + · · · + f,,X") + (f1X + h.X + · · · + f, 1 X 2 + (fiX + JzX + · · · + f,,X") 2.

11

)2

Expanding the preceding equation repeatedly, we eventually obtain

Since f,

= 0 or 1,

J? = f,. Hence, we have J2(X)

=Jo+ fiX2 = J (X2).

+ fz(X2)2 + ... + f;, (X2)n (2.9)

It follows from (2.9) that, for any i ::: 0,

[J(X)f 2.4

=

J(X 2\

(2.10)

CONSTRUCTION Of GALOIS FIELD GF(2111 )

In this section we present a method for constructing the Galois field of 2111 elements (m > 1) from the binary field GF(2). We begin with the two elements O and 1 from

Section 2.4

Construction of Galois Field GF(2 111 )

43!

GF(2) and a new symbol a. Then, we define a multiplication "." to introduce a sequence of powers of a as follows: 0-0 =0,

= 1- 0 = 0,

0 -1

1-1=1, 0 ·a= a· 0 = 0,

= a -1

1-a

=a,

(2.11)

a2 =a. a, a3 =a.a. a,

aj =a· a · · · · · a (j

times),

H follows from the preceding definition of multiplication that 0 · 0/,j

= 0/,j

·

0

= 0,

1 · 0/j

= 0/j

·

1 = 0/,j,

(2.12)

Now, we have the following set of elements on which a multiplication operation"." is defined: 2 . F = (0, 1, a, a , · · · , a.I,···}. The element 1 is sometimes denoted by a 0 . Next, we put a condition on the element a so that the set F contains only 2111 elements and is closed under the multiplication"." defined by (2.11). Let p(X) be a primitive polynomial of degree mover GF(2). We assume that p(a) = 0 (i.e., a is a root of p(X)). Since p(X) divides X 21"-l + 1 (Theorem 2.10) we have X

2111 1 -

+ 1 = q(X)p(X).

(2.13)

H we replace X with a in (2.13), we obtain 111 a2 - 1

Because p(ot)

+ 1 = q(a)p(a).

= 0, we have 111 ot 2 - l +l=q(ot)·O.

H we regard q(ot) as a polynomial of ot over GF(2), it follows from (2.7) that = 0. As a result, we obtain the following equality:

q(a) · 0

0{2"'-1

+1=

0.

44

Introduction to Algebra

Chapter 2

Adding 1 to both sides of a 2"'-l + 1 = 0 (using modulo-2 addition), we obtain the following equality: a2"'-1 = 1. (2.14) Therefore, under the condition that p(a) the following elements: F*

=

= 0, the set F becomes finite and contains

{O, 1, a, a 2 ,

··· ,

a 2"'- 2 }.

The nonzero elements of F* are closed under the multiplication operation "·" defined by (2.11). To see this, let i and j be two integers such that O ::: i, j < 2m -1. If i + j < 2111 - 1, then a; · ai = ai+i, which is obviously a nonzero element in F*. If i + j > 2111 - 1, we can express i + j as follows: i + j = (2111 - 1) + r, where 0 s r < 2111 - 1. Then,

which is also a nonzero element in F*. Hence, we conclude that the nonzero elements of F* are closed under the multiplication"." defined by (2.11). In fact, these nonzero elements form a commutative group under ". ". First, we see that the element 1 is the unit element. From (2.11) and (2.12) we see readily that the multiplication operation "." is commutative and associative. For O < i < 2m - 1, a 2111 -i -l is the multiplicative inverse of a;, since 111 a2 -i-l.

a;=

a2"'-1

= 1.

(Note that a 0 = a 2"'-l = 1.) It will be clear in the discussion that follows that 2111 1, a, a 2 , ... , a - 2 represent 2111 - 1 distinct elements. Therefore, the nonzero elements of F* form a commutative group of order 2111 - 1 under the multiplication operation"." defined by (2.11). Our next step is to define an addition operation"+" on F* so that F* forms a commutative group under"+". For Os i < 2111 -1, we divide the polynomial X; by p(X) and obtain the following: X;

= q;(X)p(X) + a;(X),

(2.15)

where q; (X) and a; (X) are the quotient and the remainder, respectively. The remainder a; (X) is a polynomial of degree m - 1 or less over GF(2) and is of the following form: a;(X)

= a;o +

a;iX

+

a;2X

2

+ · · · + a;_ 111 -1X 111 - 1.

Because X and p(X) are relatively prime (i.e., they do not have any common factor except 1), X; is not divisible by p(X). Therefore, for any i ::: 0, a; (X)

For Os i, j < 2111

-

=/- 0.

(2.16)

1, and i =I- j, we can also show that a;(X) =I- aj(X).

(2.17)

Section 2.4

that a;(X) X;

= ai

Construction of Galois Field

it follows from

= =

+

that

[q; (X) + lJi (X) ]p(X) + a; (X) + ai (X)

[q;(X)

+ lJJ(X)]p(X).

(1 + that j > i). prime, p(X) must divide + 1; however. since j - i < 2 111 - 1, and p(X) is a polynomial of m that does not divide X" + 1 for n < 2111 - 1. Therefore. our that a; (X) = ui (X) is invalid. As a result. for O :s i. j < 2111 - 1. and i cf. j, we must have cf. ai (X). for i = 0. 1. 2. · ... 2111 -- 2. we obtain 2111 - 1 distinct nonzero a; (X) of degree m -- 1 or less. Now. X with a in (2.15) and the equality that q; (a) . 0 = 0 [see (2.7) ]. we obtain the for a;:

This

(2.18) From (2.16). and (2.18). we see that the 2111 -1 nonzero ao. a 1 •... . 2 2 111 a "' - in F*, are 2 - 1 distinct non zero polynomials of a over with degree m - l or less. The zero element O in F* may be zero polynornial. As a result. the 2111 elements in F* are represented by 2111 distinct polynomials of a over with degree 111 - 1 or less and are regarded as 2"' distinct elements. Now, we define an addition"+" on F* as follows: (2.19a) for O :s i. j < 2111

-

1. (2.19b)

j ( a i +a= a;o+o;1+

=

( Cl;()+ Clj()).

+

·+o;. 111 -JOI m - l )+(ajo+aj1a+··· ( O;J

+

Clj[ ) Oi

+ ··· +

( lli.m-1

+

+ oj.m-1a . m-1)

Clj.m-1 ) 0/ m-1 .

(2.1

where au+ aj.k is carried out in modulo-2 addition for O :s k < see that, for i = j,

111.

From (2.19c) we (2.20)

and for i

f j, (a;o

+ ajo) +(ail+

1)a

+ · · · + (a;_m-1 + oj.111-1 )a"'- 1

is nonzero and must be the polynomial expression for some ak in F*. Hence., the set F* is closed under the addition"+" defined (2.19). 'Ne can immediately verify that F* is a commutative group under"+". First, we see that O is the additive identity. Because modulo-2 addition is commutative and associative, the addition defined on F* is also commutative and associative. From (2.19a) and (2.20) we see that the additive inverse of any element in F* is itself. Up to this point we have shown that the set F* = {0, 1. a. a 2 . . . . . a 2'"- 2 ) is a commutative group under an addition operation "+",and the nonzero elements of

46

Chapter 2

Introduction to Algebra

F* form a commutative group under a multiplication operation".". Using the polynomial representation for the elements in F* and (2.8) (polynomial multiplication satisfies distributive law), we readily see that the multiplication on F* is distributive over the addition on F*. Therefore, the set F* = {O, 1, a, a 2, · · . , a 2"' - 2 } is a Galois field of 2111 elements, G F(2111 ). We notice that the addition and multiplication defined on F* = GF(2 111 ) imply modulo-2 addition and multiplication. Hence, the subset {O, l} forms a subfield of GF(2 111 ) [i.e., GF(2) is a subfield of GF(2 111 ) ]. The binary field GF(2) is usually called the ground field of GF(2 111 ). The characteristic of GF(2 111 ) is 2. In our process of constructing GF(2 111 ) from GF(2), we have developed two representations for the nonzero elements of GF(2 111 ): the power representation and the polynomial representation. The power representation is convenient for multiplication, and the polynomial representation is convenient for addition.

IEXAMIP'LE 2.1

Let m = 4. The polynomial p(X) = 1 + X + X 4 is a primitive polynomial over GF(2). Set p(a) = 1 +a+ a 4 = 0. Then, a 4 = 1 +a. Using this relation, we can construct GF(2 4 ). The elements of GF(2 4 ) are given in Table 2.8. The identity a 4 = 1 + a is used repeatedly to form the polynomial representations for the elements of GF(24 ). For example, a 5 = a · a 4 = a (l +a) a6

=a

. as

=

= a(ot + a2)

c/, a2 + a3,

a +

=

a7 =a. a6 = a(a2 + a3) = a3 + a4

= a3 + 1 +a= l

+a+ a3.

To multiply two elements a; and a.i, we simply add their exponents and use the fact that a 15 = 1. For example, a 5 · a 7 = a 12 , and a 12 · a 7 = a 19 = a 4. Dividing al by ai, we simply multiply a.i by the multiplicative inverse a 15 -i of ai. For example, a4 /a12 = a4. a3 = a 7, and a12 /as = a12. alO = ot22 = ot 7. To add a; and al, we use their polynomial representations given in Table 2.8. Thus, as + a 7 = (a + a2) + (l + ot + ot3) = 1 + ot2 + a3 1 + ot 5 + a 10 = 1 + (a + a 2) + (1 + a + ot 2) = 0.

= a13,

There is another useful representation for the field elements in GF(2m). Let ao + a1a + a2a 2 + · · · + a 111 _iam-l be the polynomial representation of a field element {3. Then, we can represent f3 by an ordered sequence of m components called an m-tuple, as follows:

where the m components are simply the m coefficients of the polynomial representation of {3. Clearly, we see that there is one-to-one correspondence between this 111-tuple and the polynomial representation of {3. The zero element O of GF(2111 ) is represented by the zero m-tuple (0, 0, · · · , 0). Let (bo, b1, · · · , b111 _1) be them-tuple

Basic

Section 2.5

of a Galois Field GF(2111 )

~1

for the elements of

0 1

t!l 1fil.i!ffDile

IPoUywl[Drffilln&nil

IPov-ver

0

0 1 a

O!

a3

+

0/

ot+ + l + l 0/9 0{10

all 0/12

_J_

0/

I

+

+

0/

+ ot 2 + 0{3 o,+ l+a+ + 1 T T -1- 0{3 1 1+

O!

I

I

(0 0 0 (100 0) (0 100) 01 (0 0 0 1) (1 1 0 0) (0 11 0) (0 0 11) (l l O 1) (l O 1 0 l) (1 11 0) (0111) (1 11 1) (1 0 11) (100 1)

·~--~----------·------

----

as follows:

where a; + b; is carried out in modulo-2 addition. resultant are the coefficients of the three for the elements of are Galois fields of 2111 elen1ents with m ""' 3 to 10 are

of the for f3 + y. AH in Table 2.8. ~~CcH~~H~H'Cc,

fo we often see that a polynomial with real coefficients has roots not from the field of real numbers but from the field of numbers that contains the field of real numbers as a subfield. For the +6X + 25 does not have roots from the field of reaI numbers but has two roots, -3 + 4i and -3 - 4i. where i = J=I. This situation is also true for polynomials fo this case, a with coefficients from GF(2) with coefficients from may not have roots from but has roots from an extension field of GF(2). For + + 1 is irreducible over and therefore it does not have roots U we substitute the from however, it has four roots from the field elements of GF(2 4 ) given Table 2.8 into X 4 + x 3 + 1, we find that ot 7 . ot 11 , a 13 • and a 14 are the roots of + + l. We may this result as follows:

48

Chapter 2

Introduction to Algebra

Indeed, a 7 is a root for X 4 + X 3 + 1. Similarly, we may verify that a 11 , a 13, and a 14 are the other three roots. Since a 7, a 11 , a 13, and a 14 are all roots of X 4 + X 3 + 1. then (X +a 7)(X +a 11 )(X +a 13 )(X +a 14 ) must be equal to X 4 + X 3 + 1. To see this, we multiply out the preceding product using Table 2.8: (X + a 7)(X + a 11 )(X + a 13 )(X + a 14 )

= [x2 + = (X2 +

(a 7 + all )X + a1s][ x2 + (a13 + a14)X + a27] as X + a3) (X2 + a2 X + a12)

= x4 +(as+ a2)X3 + (a12 + alO + a3)X2 + (a20 + as)X + a1s

= x4 + x3 +

1.

Let f (X) be a polynomial with coefficients from GF(2). If {3, an element in GF(2 111 ), is a root of f(X), the polynomial f (X) may have other roots from GF(2 111 ). Then, what are these roots? This question is answered by the following theorem. Zoll Let f (X) be a polynomial with coefficients from GF(2). Let f3 be an element in an extension field of GF(2). H f3 is a root off (X), then for

THEOREM

any l::: 0, {32' is also a root of f(X). Proof: From (2.10), we have

Substituting f3 into the preceding equation, we obtain

Since f({J)

= 0,

21

f({3 )

= 0. Therefore, {3 21

is also a root of f(X).

QoKDo

The element {32' is called a conjugate of {3. Theorem 2.11 says that if {J, an element in GF(2 111 ), is a root of a polynomial f (X) over GF(2), then all the distinct conjugates of {3, also elements in GF(2 111 ), are roots of f (X). For example, the polynomial f (X) = 1 + x 3 + X4 + x 5 + x 6 has a4, an element in GF(2 4 ) given by Table 2.8, as a root. To verify this, we use Table 2.8 and the fact that a 15 = 1: f(a4) = 1 + a12 + a16 + a20 + a24 = l + a12 +a+ <XS+ a9

= 1+

(1 + a + a 2 + a 3 ) + a

+ (a +

3 2 a ) + (a + a )

= 0.

The conjugates of a 4 are (a4)2

= as,

(a4)2

2

= al6 = a,

(a4)2"

= a32 = a2.

[Note that (a 4) 24 = a 64 = a 4.] It follows from Theorem 2.11 that as, a, and a 2 must also be roots of f(X) = 1 + X 3 + X 4 + X 5 + X 6. We can check that as and its conjugate, a 10 , are roots of f (X) = 1 + X 3 + X 4 + X 5 + X 6. Therefore, f (X) = 1 + X 3 + X 4 + x 5 + x 6 has six distinct roots in GF(2 4).

Basic Properties of a Galois Field GF(2 111 )

Section 2.5

4'!?J

Let f3 be a nonzero element in the field GF(2111 ). H follows from Theorem 2.8 that /32"'-l = 1. Adding 1 to both sides of {3 2"'-l

= 1, we obtain /32"'-l

+ 1 = 0.

This says that f3 is a root of the polynomial x 2'" - l + 1. Hence, every nonzero element of G F(2111 ) is a root of x 2"' - l + 1. Because the degree of x 2"' - l + 1 is 2111 - L the 2111 - 1 nonzero elements of GF(2111 ) form all the roots of -l + 1. Summarizing the preceding we obtain Theorem 2.12.

ZJlZ The 2111 x2'"-1 + 1.

'lrJHIJEl[J)JRJEM

-

1 nonzero elements of GF(2 111 ) form all the roots of

Since the zero element O of GF(2111 ) is the root of X. Theorem 2.12 has the following corollary: C([))JRl[J)JLILAJRY

ZJLZJl

The elements of GF(2 111 ) form an the roots of x2"'

+ X.

Because any element f3 in GF(2 ) is a root of the polynomial x + X. f3 may be a root of a polynomial over GF(2) with a degree less than 2111 • Let ¢,(X) be the polynomial of smallest degree over GF(2) such that c/>(/3) = 0. [We can easily prove that ¢,(X) is unique.] This polynomial ¢(X) is called the minimal polynomial of {3. For example, the minimal polynomial of the zero element Oof GF(2111 ) is X, and the minimal polynomial of the unit element 1 is X + 1. Next, we derive a number of properties of minimal polynomials. 2"'

111

'lfJHIJEORIEM

'.tB

The minimal polynomial ¢(X) of a field element f3 is

irreducible.

= ¢1 (X)¢2 (X), where both ¢1 (X) and ¢2 (X) have degrees greater than O and less than the degree of¢ (X). Since ¢ (/3) = c/>1 (/3)¢2 (/3) = 0, either ¢1 (/3) = 0 or ¢2 (/3) = 0. This result contradicts the hypothesis that cf>(X) is a polynomial of smallest degree such that c/>(/3) = 0. Therefore, cp(X) must be irreducible. (QlJEJil>o

IPnmf. Suppose that ¢ (X) is not irreducible and that ¢ (X)

::tll41 Let f (X) be a polynomial over GF(2). Let ¢(X) be the minimal polynomial of a field element {3. H f3 is a root off (X), then f (X) is divisible by ¢(X). 'lrJHIJE([J)IR!EM

IP/J"![}J![}JJ. Dividing

f (X) by cf> (X), we obtain f(X) = a(X)c/>(X)

+ r(X),

where the degree of the remainder r(X) is less than the degree of ¢(X). Substituting f3 into the preceding equation and using the fact that f (/3) = ¢(/3) = 0, we have r(/3) = 0. H r(X) f= 0, r(X) would be a polynomial of lower degree than ¢(X), which has f3 as a root. This is a contradiction to the fact that ¢(X) is the minimal polynomial of {3. Hence, r(X) must be identical to O and ¢(X) divides f(X). IQ)JEJil>o

50

Chapter 2

Introduction to Algebra

The following result follows from Corollary 2.12.1 and Theorem 2.14. THEOREM

divides

x

2.15

2"'

+ X.

The minimal polynomial ¢(X) of an element f3 in GF(2 111 )

Theorem 2.15 says that all the roots of ¢(X) are from GF(2111 ). Then, what are the roots of ¢(X)? This question is answered by the next two theorems. 2.16 Let f (X) be an irreducible polynomial over GF(2). Let f3 be an element in GF(2111 ). Let ¢(X) be the minimal polynomial of /3. If f (/3) = 0, then ¢(X) = f (X).

THEOREM

Proof. It follows from Theorem 2.14 that ¢(X) divides f (X). Since ¢(X) =I= 1 and f (X) is irreducible, we must have ¢(X) = f (X). Q,E.D. Theorem 2.16 says that if an irreducible polynomial has f3 as a root, it is the minimal polynomial ¢(X) of /3. It follows from Theorem 2.11 that f3 and its 22 conjugates {3 2 , {3 , · · · , 13l1, · · · are roots of ¢ (X). Let e be the smallest integer 1 22 such that {3 2' = f3. Then, {3 2, {3 , · · ·, 13r are all the distinct conjugates of f3 (see 2111 Problem 2.15). Since /3 = /3, e ::: m (in fact e divides 111 ). 2.17 Let f3 be an element in GF(2111 ), and let e be the smallest nonnegative integer such that 13 2' = /3. Then,

THEOREM

e-1

f(X) =

TI (X + /32') i=O

is an irreducible polynomial over GF(2).

Proof. Consider

[/(X)]

Since (X + 132' )2

2

= x2 +

~

[

8

(X

+ fi'')

r5 ~

(/32' + 132' )X + 13i+1 e-1

[f(X)]2=

(X

= x2

+ fi 2') 2

+ 132'+1,

e

f1cx 2 +13 2'+ 1)= f1cx 2 +13 2')

Since {3 2' = f3, then e-1

2

[f(X)] =

f1cx 2 + /3 21 ) =

i=O

2 J(X ).

(2.21)

Basic Properties of a Galois Field GF(2 111 )

Section 2.5

Let

=

to+ 2

[f00] =

51

, where fe = 1.

)2

Uo + f1X + · · · + e

e

= '•)'

+ (1 + 1)

~==1

f;fj

i=O

i=O j=O

i=O

i f-j

From

) and

we obtain e

'Vf ,L.,

!

·2 v2i

. i

I

./ii..

•

i=O

for Os i ::: e. YNe must have f; =

This result holds

i.

when f; = 0 or L

has coefficients from

Now. suppose that f(X) is not irreducible over and f(X) = = O. either o(f3) = 0 or b(/3) = 0. If a(/3) = O. (/(X) has 1 {3. {3 2.... . !3 2"- as roots. so o(X) has degree e. and o = f()(). Similarly. if b(,6) = 0. b(X) = /(X). Therefore. f (X) must be irreducible.

a(X)b(X). Since f(/3)

A direct consequence of Theorems 2.16 and 2.17 is Theorem 2.18. l'JHIJEOJP?JEM

2Ji§

Let ¢(X) be the rninimal polynomial of an element f3 m /3 2" = {3. Then

GF(2m ). Let e be the smallest integer such that

n

e-1

¢

)=

+ {32' ).

i=O

Consider the Galois held /3 are

f3-?

given by Table 2.8. Let (

t3-72

=a'.

The minimal polynomial of f3 = <jJ(X)

=

(X

0!

3

I?

=Of-.

/3 2-'

_

~Oi

. Tbe

of

2.J ---

is then

+ a 3 )(X + a 6 )(X + 0! 12 )

out the right-hand side of the preceding we obtain ¢(X)

/3 =

+ cx 9 ). with the aid of

+ (0!3 + cx6)X + 0!9)[X2 + (cvt2 + cx9)X + a21] = (X2 + + cv9)(X2 + cv8 X + a6) = X.J + (cv2 + 018)X3 + (0!6 + cxlO + a9)X2 + (a17 + a8)X + al5 =

[X2

=

x 4 + x 3 + x 2 + x + 1.

52

Chapter 2

Introduction to Algebra

There is another way of finding the minimal polynomial of a field element, which is illustrated by the following example. IEXAMPLIE 2.9

Suppose that we want to determine the minimal polynomial cp (X) of y GF(2 4 ). The distinct conjugates of y are

=

a 7 in

3

y 2 =a 56 =a 11 .

Hence, cp(X) has degree 4 and must be of the following form:

Substituting y into cp(X), we have

Using the polynomial representations for y. y 2 . y 3 , and y 4 in the preceding equation, we obtain the following: oo

+ O[ (1 +a+ a 3 ) + 02(1 + a 3 ) + 03(a 2 + a 3 ) + (1 + a 2 + a 3 ) = 0 2 3 (oo + 01 + 02 + 1) + cqa + (03 + l)a + (01 + 02 + 03 + l)a = 0.

For the preceding equality to be true, the coefficients must equal zero: oo

+ 01 + 02

+1

= 0, = 0,

01

+1= a1 + a2 + a3 + 1 = a3

0,

0.

Solving the preceding linear equations, we obtain ao = 1, a1 = a2 = 0, and a3 = 1. Therefore, the minimal polynomial of y = a 7 is cp (X) = 1 + X3 + X4 . All the minimal polynomials of the elements in GF(2 4 ) are given by Table 2.9.

TABLE 2.9:

Minimal polynomials of the elements in GF(2 4 ) generated by p(X) = X 4 -t- X + 1. 0

X

1

X+l

a, a2, ex 4,

as

(X3, (X6, (X9, (X12

as. alO a 7, a 11, a l3, (Xl4

x 4 +x + 1 x + x3 + x2 + x + 1 X2 + X + 1 x4 + x3 + 1 4

Basic

Section 2.5

a Galois Field

A direct consequence of Theorem 2.18 is Theorem 2.19.

2,]_9J Let rp be the minimal of an element f3 in ). Let e be the of rp (X). Then e is the smallest such that (3 2" = (3. Moreove,., e :::: m.

TJBTJEu,JR;JEMi

In

of the minimal of any element in is omitted here. Table 2.9 shows that the of each element in GF(24 ) divides of the elements in ) for 111 = 2 to 10 are lin the construction of the Galois field GF(2111 ) we use a p(X) of and that the element 01 be a root of aH tLe nonzero elements of of 01 are elements of let n

divides m.

<-~LU
it follows from Theorem 2.9 that

divides 2111

11

2111 -1

1:

-

= k. 11. =--::

and 2m -- l are

n1ust be a divisible 11

=

q . (2/ll -- 1).

we conclude that 11 = 2m ). h1 we have the

and

o/ is also a

). an (3 2

. ···

1.

Table 2.8. The powers of f3

(31 =--::

= O{s. (39

its

are also

Consider the field {Jo=

= 0/63 =

{313 = 0/91 =:

= 0/14, (33 = 0121 = 0/6, {34 = 012s = = = a12. fl= a49 = 0/4. ,es= as6

18 2

(36 {310

0/,

= a 7 are

= a70 = 0/10,

(311

= a77

=--=

f3l4 = 0/98 = 0/8, {315 = 0/105 =

a2. (312

=

= all.

=

0

{32

= Ol14,

{32

check that they are all

2

= 0/13,

a9,

1.

the powers of f3 == a 7 generate all the nonzero elements of nn~,,-,., element of GF(27 ). The conjugates of fJ = are

We may

Lni

n must be

3

(32 = 0/11.

elements of

so

54

Chapter 2

Introduction to Algebra

A more general form of Theorem 2.20 is Theorem 2.21. If f3 is an element of order n in GF(2 111 ), all its conjugates have the same order n. (The proof is left as an exercise.)

THEOREM

2o2l

EXAMPLE 2.11

Consider the element as in GF(2 4 ) given by Table 2.8. Since (as)22 = a 20 = as, the only conjugate of as is a 10 . Both as and a 10 have order n = 3. The minimal polynomial of as and a 10 is X 2 + X + 1, whose degree is a factor of m = 4. The conjugates of a 3 are a 6 , a 9, and a 12 . They all have order n = 5.

2.6

COMPUTATIONS USING GALOIS FIIEU) GF(2111 ) ARITHMETIC

Here we perform some example computations using arithmetic over GF(2111 ). Consider the following linear equations over GF(2 4 ) (see Table 2.8): X

+ a 7Y = a 2, (2.26)

Multiplying the second equation by a 3 gives X+a 7 Y=a 2. X +ally= a7.

By adding the two preceding equations, we get (a

7

+ a 11 )Y = a 2 + a 7 , a8Y

= a12,

y =a4.

Substituting Y = a 4 into the first equation of (2.26), we obtain X = a 9 . Thus, the solution for the equations of (2.26) is X = a 9 and Y = a 4 . Alternatively, the equations of (2.26) could be solved by using Cramer's rule:

X

=

:: I u u I ::

-I

-1-1--a-7

a12

1

~10

"'

+ ~11 ._,

1 +a 3

a 14

a8

al2

a2 I a4

1 a12

a7 cx8

Y=

I

9

= _a_8_+_a_l_9 = _a_+_a_2 = -a-5 = a .

a4 + al4 a+ a3 a9 4 - - - - - - - - - -a a8+al9 - a+a2 - as -

As one more example, suppose that we want to solve the equation f(X)=X 2 +a 7 X+a=0

Section 2.7

Vector Spaces

55

over GF(24). The quadratic formula will not work because it requires dividing by 2, and in this field, 2 = 0. If f (X) = 0 has any solutions in GF(24 ), the solutions can be found simply by substituting all the elements of Table 2.8 for X. By doing so, we would find that f (a 6 ) = 0 and f (a 10 ) = 0, since

+ 0/7. (¥6 + ot = 0/12 + (¥13 +a= 0, (a10)2 + ot 7 . 0/10 + a = as + ot2 + a = O.

f(ot6) = (ot6)2

f

(otlO)

=

a 6 and a 10 are the roots of f(X), and j(X) = (X + a 6 )(X + a 10 ). The preceding computations are typical of those required for decoding codes such as BCH and Reed-Solomon codes, and they can be programmed quite easily on a general-purpose computer. H is also a simple matter to build a computer that can do this kind of arithmetic.

Let V be a set of elements on which a binary operation called addition, +, is defined. Let F be a field. A multiplication denoted ·, between the elements in F and elements in V is also defined. The set V is called a vector space over the field F if it satisfies the following conditions: lio V is a commutative group under addition. lino For any element a in F and any element win V, a· w is an element in V. ilnlio (Distributive Laws) For any elements llil and win V and any elements a and b in F, a · (un

+ 'Y) = a · un + a · w,

(a+ b) · w =a· w + b · w. IlWo

(Associative

For any win V and any a and bin F, (a· b) · w =a· (b · w).

Wo

Let 1 be the unit element of F. Then, for any win V, 1 · w =

o/.

The elements of V are called vectors, and the elements of the field Fare called scalars. The addition on V is called a vector addition, and the multiplication that combines a scalar in F and a vector in V into a vector in V is referred to as scalar multiplication ( or product). The additive identity of V is denoted by I[]). Some basic properties of a vector space V over a field F can be derived from the preceding definition. IP'Tl'opeT!'lty ff

Let O be the zero element of the field F. For any vector w in V,

0-w=i[J). IP'T!'ooj, Because 1 + 0 = 1 in F, we have 1 · w = (1

+ 0) · w = 1 · w + 0 · w. Using condition (v) of the preceding definition of a vector space, we obtain

56

Introduction to Algebra

Chapter 2

v = v + 0 · v. Let -v be the additive inverse of v. Adding -v to both sides of v = v + 0 · v, we have

Property II

For any scalar c in F, c · 0 = 0. (The proof is left as an exercise.)

Property III

For any scalar c in F and any vector v in V, (-c) · v = c · (-v) = -(c · v)

That is, (-c) · v or c. ( -v) is the additive inverse of the vector c · v. (The proof is left as an exercise.) Next, we present a very useful vector space over GF(2) that plays a central role in coding theory. Consider an ordered sequence of n components, (ao, a1, · · · a11-1), where each component a; is an element from the binary field GF(2) (i.e., a; = 0 or 1). This sequence is generally called an n-tuple over GF(2). Because there are two choices for each a;, we can construct 211 distinct n-tuples. Let V,, denote this set of 211 distinct n-tuples over GF(2). Now, we define an addition, +, on V11 as the following: For any u = (uo, u1, · · · , u11 _1) and v = (vo, v1, · · · , v11 _1) in V, 1 , u + v = (uo + vo, u1 + v1, · · · , Un-1 + V11-1),

(2.27)

where u; + v; is carried out in modulo-2 addition. Clearly, u +vis also an n-tuple over GF(2). Hence, V,1 is closed under the addition defined by (2.27). We can readily verify that V,, is a commutative group under the addition defined by (2.27). First, we note that the all-zero n-tuple O = (0, 0, · · · , 0) is the additive identity. For any v in V11 , V

+

V

=(VO+

=

VO,

V1 + Vl, · · · , Vn-1 + Vn-1)

(0, 0, · · · , 0)

= 0.

Hence, the additive inverse of each n-tuple in V, 1 is itself. Because modulo-2 addition is commutative and associative, we can easily check that the addition defined by (2.27) is also commutative and associative. Therefore, V11 is a commutative group under the addition defined by (2.27). Next, we define scalar multiplication of an n-tuple v in V, 1 by an element a from GF(2) as follows: a· (vo, v1, · · · , v 11 _i) =(a· vo, a· VJ,··· , a· v11 -1),

(2.28)

where a· v; is carried out in modulo-2 multiplication. Clearly, a· (vo, v1, · · · , v11 _1) is also an n-tuple in V,1 • If a = 1, 1 · (vo, v1, · · · , v11 _1) = (1 · vo, 1 · v1, · · · .1 · v11 -1)

= (VO, Vl, · · · , Vn-1),

Section 2.7

Vector Spaces

57

VVe can

the distributive and associative laws. forms a vector space over

Let n = 5. The vector space V5 of all. 32 vectors: (00000). (00001). (00100). (00101), (01000). (01001), (0110 0). (01101). (10000). (10001). (10100). (10101). (11000). (11001), (1110 (11101).

consists of the

over (00010). (00110). (010 l (01110\ (10010). (101 l. 0). (1101 (11110).

(00011). (0011 (01011). (01111). (10011), (10 111). (11011) (11111).

The vector sum of (1 0111) and (1 1 0 0 1) is (1 0111)

+ (11001) =

(1

+ 1. 0 + 1. + 0. 1 + 0. 1 + l) =

the rule of scalar multiplication defined 0 (11010)

=

(0 111

(228), we obtain

(0 · 1. 0 -1, 0. 0. 0. LO· 0) = (00000).

1 · (110 l 0) = (1 · L 1 . l. 1 . 0, l . 1. 1 · 0) = (11010).

The vector space of aH n-tuples over any field F can be constructed in a similar manner; however, in this text we are concerned with the vector space of all over or over an extension field of [e . g.. Because Vis a vector space over a field F. it may happen that a subset S of V is also a vector space over F. Such a subset is called a subspace of V. 'fJHiiEOi1UE1Vil

Zo22

Let S be a nonempty subset of a vector space V over a field F.

S is a subspace of V if the following conditions are satisfied:

no For any two vectors

ill

and "I in S. w1 + w is also a vector in S. llil in S. c1 · ill is also in S.

nu. For any element a in F and any vector

JPupof. Conditions (i) and (ii) simply say that Sis dosed under vector addition and scalar multiplication of V. Condition

ensures that for any vector w in + (-1) . ·y = I!]) is also in S. Therefore. S is a subgroup of V. Because the vectors of S are also vectors of V. the associative and distributive laws must hold for S. Hence. S is a vector space over F and is a subspace of V. QJEJ[]). S its additive inverse (-1) . vis also in S. Then, w

58

Chapter 2

Introduction to Algebra

IEXAMPLIE 2.13

Consider the vector space Vs of all 5-tuples over GF(2) given in Example 2.12. The set {(00000), (00111). (11010), (11101)} satisfies both conditions of Theorem 2.22, so it is a subspace of Vs. Let v1. v2, · · · , v1c be k vectors a1. a2, · · · , ak be k scalars from F. The

in a vector space V over a field F. Let sum

is called a linear combination of v1, v2, · · · , v1c. Clearly, the sum of two linear combinations of v1, v2, · · · , v1c, (a1v1

+ a2v2 + · · · + akvk) + (b1v1 + b2v2 + · · · + bkvk) = (a1 + b1)v1 + (a1 + b2)v2 + · · · + (ak + b1c)vk,

is also a linear combination of v1, v2, · · · , vk, and the product of a scalar c in F and a linear combination of v1, v2, · · · , vk,

is also a linear combination of v1, v2, · · · , v1c. It follows from Theorem 2.22 that we have the following result. 2.23 Let w1, w2, · · · , vk be k vectors in a vector space V over a field F. The set of all linear combinations of v1, v2, · · · , Wk forms a subspace of V.

TlH!EOREM

IEX:AMPllE 2.14

Consider the vector space Vs of all 5-tuples over GF(2) given by Example 2.12. The linear combinations of (00111) and (11101) are

+ 0 · (11101) = (0 0 0 0 0), 0 · (00111) + 1 · (11101) = (11101). 1 · (00111) + 0 · (11101) = (00111), 1 · (00111) + 1 · (11101) = (110 l 0).

0 · (00111)

These four vectors form the same subspace given by Example 2.13. A set of vectors v1, v2. · · · , 'Vk in a vector space V over a field F is said to be linearly dependent if and only if there exist k scalars a 1 , a2, · · · , ak from F, not all zero, such that

Section

That

unless

a1

if v1.

·ef2, · · · , »'k

Vector

are

= 02 = · · · = ak = 0.

Thevectors(10110).(0 00 1 · (1 0 11 0)

since

and(ll111)are

+ 1 · (0 1 0 0 1) + 1 · (11111) = (0 0 0 0

however, (10 10). (010 0 and (11011) are here: combinations of these three vectors are

All eight linear

+ 0 · (01001) + 0 · (11011) = (0 0 0 0 0). 0 (1011 0) + 0 · (01001) + 1 · (11011) = (11 0 l l)s 0 · (1011 0) + 1 · (01001) + 0 · (11011) = (0100 l). 0 · (1011 0) + 1 · (01001) + 1 · (11011) = (l OO1 1·(10110) + 0·(01001) + 0·(11011) = (10110). 1 . (10110) + 0 · (0 001) + 1 · (11011) = (01101). 1 . (10110) + 1 · (01001) + 0 · (11011) = (11111). 1 · (1011 0) + 1 · (01001) + 1 · (11011) = (0010 0). 0 · (10110)

A set of vectors is said to span a vector space V if every vector in V is a linear combination of the vectors in the set. In any vector space or there exists at least one set B of vectors that span the space. This set is called a basis of the vector space. The number of vectors in a basis of a vector space is called the dimension of the vector space. (Note that the number of vectors in any two bases are the Consider the vector space V, 1 of all over Let us form the following n 11-tuples:

o.

o.

(1. 0. 0,

e1

= =

(0. 1. 0. 0.

0. 0),

en-l

=

(0, 0. 0. 0,

0, 1),

l\';Q

r?;; has one nonzero every (ao. a1. a2, · · · , a 11 _1) in V11 can be of <J;;o. e1, · · · . e11 _1 as follows:

0),

at the ith position. as a linear combination

60

Chapter 2

Introduction to Algebra

Therefore, eo, e1, · · · , e11 _1 span the vector space V,1 of all n-tuples over GF(2). From the preceding equation we also see that eo, e1, · · · , e11 _ 1 are linearly independent. Hence, they form a basis for V11 , and the dimension of V,, is 11. If k < 11 and v1, v2, · · · . vk are k linearly independent vectors in V11 , then all the linear combinations of v1, v2, · · · , vk of the form U

= CIVl + QV2 + · · · + Ck'lk

form a k-dimensional subspace S of V11 • Because each c; has two possible values, 0 or 1, there are 2k possible distinct linear combinations of v1, v2, · · · . Vk. Thus, S consists of 2k vectors and is a k-dimensional subspace of V11 • Let u = (uo, u1, · · · , u 11 _1) and v = (v1, v1, · · · , v11 _1) be two n-tuples in V,,. We define the inner product ( or dot product) of u and v as u ·v

= uo · vo + u1 · v1 + · · · + u

11 _1

· Vn-l,

(2.29)

where u; · v; and u; · v; + u;+1 · v;+1 are carried out in modulo-2 multiplication and addition. Hence, the inner product u ·vis a scalar in GF(2). If u. v = 0, u and v are said to be orthogonal to each other. The inner product has the following properties: R, U · V = V · U. ii. u · (v + w) = u · v + u · w. m. (au) · v = a(u · v).

(The concept of inner product can be generalized to any Galois field.) Let S be a k-dimensional subspace of V, 1 and let Sd be the set of vectors in V11 such that for any u in Sand v in Sd, u · v = 0. The set Sd contains at least the all-zero 11-tuple O = (0, 0, · · · , 0), since for any u in S, 0. u = 0. Thus, S11 is nonempty. For any element a in GF(2) and any v in Sd, if a= 0, if a = 1. Therefore, a· vis also in Sd. Let v and w be any two vectors in Sd. For any vector u in S, u, (v + w) = u · v + u · w = 0 + 0 = 0. This says that if v and ware orthogonal to u, the vector sum v + w is also orthogonal to u. Consequently, v + w is a vector in S11. It follows from Theorem 2.22 that Sd is also a subspace of V,,. This subspace Sd is called the null (or dual) space of S. Conversely, Sis also the null space of Sd. The dimension of Sd is given by Theorem 2.24, whose proof is omitted here [l ]. 2.24 Let S be a k-dimensional subspace of the vector space V11 of all n-tuples over GF(2). The dimension of its null space SJ is n - k. In other words, dim(S) + dim(Sd) = n.

THEOREM

EXAMPLE 2.16

Consider the vector space Vs of all 5-tuples over GF(2) given by Example 2.12. The following eight vectors form a three-dimensional subspace S of Vs: (00000),

(11100),

(01010).

(10001).

(10110).

(01101),

(11011),

(00111).

Section 2.8

Matrices

rs 1

The null space Sc1 of S consists of the following four vectors: (00000),

(10101).

(01110),

(11011).

,-.,u,u,~·~ by (10101) and (0111 0), which are linearly dimension of Sc1 is 2.

the

AH the results in this section can be ma manner to the vector space of all n-tuples over GF(q), where q is a power of (see Section

:2.&9i

MArm«:IE5 A k x n matrix over GF(2) rows and n

G~l

over any other

goo

g01

g02

g10

g11

g12

gk-1.0

gk-1.1

is a rectangular array with k

go,n-1

]

g1.11-l

gk-l.11-l

gk-1.2

(2.30) •

where each entry g;J with O ::c i < k and O ::c j < n is an element from the field GF(2). Observe that the first index, i, indicates the row g;J, and the second index, j, tells which column g;i is in. VVe shall sometimes abbreviate the matrix of (2.30) by the notation [g;Jl. We also observe that each row of eG is an n-tuple over GF(2), and each column is a k-tuple over GF(2). The matrix CG can also be represented by its k rows go. 1!;1, · · · , !l:k-1 as follows:

eG=[ :~ ]· il;k-1

H the k(k ::c n) rows of CG are linearly independent, then the 2" linear combinations of these rows form a k-dimensional subspace of the vector space V11 of all then-tuples over GF(2). This subspace is called the row space of CG. We may interchange any two rows of CG or add one row to another. These are called elementary row operations. Performing elementary row operations on CG, we obtain another matrix (G1 over GF(2); however, both eG and CG' give the same row space.

IE}{P,MIPU: 2. H Consider a 3 >< 6 matrix

eG over GF(2),

c~u

1 0 0 1 1 0

1 1 0

1 0 1 0 1 1

l

62

Introduction to Algebra

Chapter 2

Adding the third row to the first row and interchanging the second and third rows, we obtain the following matrix:

1 0 0 1 G'=

01 11 ] . 1 1 1 0

0100

[ 0 0

Both G and G' give the following row space: (000000),

(100101),

(010011),

(001110),

(110110),

(101011),

(011101),

(111000).

This is a three-dimensional subspace of the vector space V6 of all the 6-tuples over GF(2). Let S be the row space of a k x n matrix G over GF(2) whose k rows are linearly independent. Let Sc1 be the null space of S. Then, the dimension of Sc1 is n -k. Let ho, lht1, · · · , lht11 _k-l be n -k linearly independent vectors in Sc1. Clearly, these vectors span Sc1. We may form an (n - k) x n matrix Bl using lhto, lht1, · · · , lht11 _k-l as rows: go, g1, .. · , 1/:k-l

h01 h11

ho,11-1 h1,11-l

]

hn-k~l,n-1

.

The row space of Bl is Sc1. Because each row g; of G is a vector in S, and each row lhtj of His a vector of Sc1, the inner product of g; and hj must be zero (i.e., g; · !hi j = 0). Because the row space S of G is the null space of the row space Sc1 of H, we call S the null ( or dual) space of H. Summarizing the preceding results, we have Theorem 2.25. 2,25 For any k x n matrix Gover GF(2) with k linearly independent rows, there exists an (n - k) x n matrix Bl over GF(2) with n - k linearly independent rows such that for any row g; in G and any llnj in H, g; · lhi.j = 0. The row space of G is the null space of H, and vice versa. THEOREM

IEX.AMPlE 2.18

Consider the following 3 x 6 matrix over GF(2): G=

1 0

[ 0

1 0 0 1

1 1

1 1

1 0 0

1

The row space of this matrix is the null space 1

H=

0

1

1

0 1 1 0 0 1 0 0 ] . [ 1 1 0 0 0 1

We can easily check that each row of G is orthogonal to each row of Bl.

Problems

!Ei:!

have the same number of rows and the Two matrices can be added if same number of columns. To add two k x 11 matrices A = [a; 1 ] and IE ~= [b; 1]. we add their entries o;1 and as follows:

+ [b;1] :=

[aij

+

].

the resultant 1natrix is also a k >< 11 matrix. Tvvo matrices can be that the number of columns in the first rnatrix is to the number of an n >< l matrix a k x 11 rnatrix P..\ == rovvs in the second matrix. JIB = [b; 1], we obtain the iC=P\XlIB:c::

fo the resultant k x 1 matrix the 21; in A and the column 1JJ1 in

Cij

].

is

that 11-l

C;j

Let

= 21;

· ]b,j

be a k >< 11 matrix over The of (G, denoted by q:;T, is an n x k matrix whose rows are columns of (G and whose columns are rows of G. and A k x k matrix is called an O's elsewhere. This matrix is Il1c. A submotrix of a matrix (G is a matrix that is obtained rows or columns of G. H is the and results in this (G

IP!RlOraltlEM§ Zoll. Construct the group under modulo-6 addition. ZoZ Construct the group under modulo-3 multiplication. 23 Let 111 be a positive ff mis not a prove that the set {L 2, · · · , m -1} is not a group under modulo-m multiplication. 2A\ Construct the prime field 1) with modulo-11 addition and multiplication. Find all the primitive elements, and determine the orders of other elements. 2o5 Let 111 be a positive integer. H mis not prime, prove that the set {O, L 2. · · · . m -1) is not a field under modulo-111 addition and multiplication. 2oilii Consider the integer group G =-~ (0, 1. 2. · · · , 31} under modulo-32 addition. Show that H = {O, 4, 8, 16, 20. 24, 28) forms a subgroup of G. Decompose G into cosets with respect to H ( or modulo 2o7 Let Jc be the characteristic of a Galois field GF(q). Let 1 be the unit element of GF(q). Show that the sums 1,

1, i=l

i=l

>.

>.-1

3

2

1,

::[1, ::[1 =0 i=l

i=l

form a subfield of GF(q). 2J'l Prove that every finite field has a primitive element.

64

Chapter 2

Introduction to Algebra

2.9 Solve the following simultaneous equations of X, Y, Z, and W with modulo-2 arithmetic: X+Y +W=l, X + z+ W=O, X + Y + Z + W = 1, Y + Z + W =0. 2.10 Show that X 5 + X 3 + 1 is irreducible over GF(2). 2.11 Let f (X) be a polynomial of degree n over GF(2). The reciprocal of defined as f*(X)

=

X"f

f (X) is

(f).

a. Prove that f*(X) is irreducible over GF(2) if and only if f(X) is irreducible 2.12 2.13

2.14

2Jl5

2.16 2.17 2.18

2.19

over GF(2). b. Prove that f*(X) is primitive if and only if f(X) is primitive. Find all the irreducible polynomials of degree 5 over GF(2). Construct a table for GF(2 3 ) based on the primitive polynomial p(X) = l +X +X 3 . Display the power, polynomial, and vector representations of each element. Determine the order of each element. Construct a table for GF(2 5 ) based on the primitive polynomial p(X) = 1 + X 2 + X 5 . Let <X be a primitive element of G F(25 J. Find the minimal polynomials of <X 3 and <X 7 . Let f3 be an element in GF(2 111 ). Let e be the smallest nonnegative integer 1 22 such that /3 2' = f3. Prove that /3 2 , {3 . · · · , !3 2"- , are all the distinct conjugates of /-J. Prove Theorem 2.21. Let <X be a primitive element in GF(2 4 ). Use Table 2.8 to find the roots of f(XJ = x3 + (X6x2 + (X9x + a9. Let <X be a primitive element in GF(2 4 ). Divide the polynomial f (X) = <X 3 x 7 + aX6 + a 7 x 4 + <X 2 x 2 + <X 11 X + 1 over GF(2 4 ) by the polynomial g(XJ = x 4 + <X 3 X 2 + <X 5 X + 1 over GF(2 4 ). Find the quotient and the remainder (use Table 2.8). Let <X be a primitive element in GF(2 4 ). Use Table 2.8 to solve the following simultaneous equations for X. Y, and Z: X X 2 a X

+<X 5 Y+

Z =<X 7 ,

+ <XY +a 7 Z=a 9 , + Y + a 6 Z = a.

2.20 Let V be a vector space over a field F. For any element c in F, prove that c · 0 = 0. 2.21 Let V be a vector space over a field F. Prove that, for any c in F and any v in V, (-c) · v = c · (-v) -(c · v). 2.22 Let S be a subset of the vector space V11 of all 11-tuples over GF(2). Prove that S is a subspace of V11 if for any Ill and v in S. u +vis in S. 2.23 Prove that the set of polynomials over GF(2) with degree 11 - 1 or less forms a vector space GF(2) with dimension n. 2.24 Prove that GF(2"') is a vector space over GF(2). 2.25 Construct the vector space Vs of all 5-tuples over GF(2). Find a three-dimensional subspace and determine its null space.

Bibliography

155

2,J)(fi Given the matrices

G= i

1 1

L0

1 0 1 1 1 1

1 0 1

1 0 0 1 0 0

nH~u

0

0

1

0

0 0

1 0

0 0

0 l

l 1 0 1

l

0

1

1 1 1

1 0

l

show that the row space of (G is the null space of IHI, and vice versa. 2,2,7 Let S1 and S2 be two subspaces of a vector V. Show that the intersection of S 1 and S2 is also a subspace in V. 2Jm Construct the vector space of all 3-tuples over Form a two-dimensional subspace and its dual space.

~mUOGIR\A~i~nf JL G. Birkhoff and S. 1953.

::t R. D.

lHUH'v,

A

Introduction to the

of Modern Algebra, Macmillan, New York, of Groups of Finite

Ginn&

Co., Boston, 1937.

Elements of Abstract Algebra, Dover, New

1984.

~-, R. A. Dean, Classical Abstract Algebra, Harper & Row, New §,

1990.

T. W. Hungerford, Abstract Algebra: An Introduction, 2d ed., Saunders College Publishing, New 1997. Table of Irreducible Polynomials over GF(2) through Degree NSA. Washington, D.C., 1957.

(/i R. W. Marsh,

7, J. E. Maxfield and M. W. Maxfield, Abstract Algebra and Solution by Radicals, Dover, New York, 1992.

it W.W. Peterson, Error-Correcting Codes, Mff Press, Cambridge, 1961. (\I),

B. L. Van der Waerden, Modern Algebra, Vols. 1 and 2, Ungar, New York, 1949.

CHAPTER

3

Linear Block Codes In this chapter we introduce basic concepts of block codes. For ease of code synthesis and implementation, we restrict our attention to a subclass of the class of all block codes, the linear block codes. Because information in most current digital data communication and storage systems is coded in the binary digits O and 1, we discuss only the linear block codes with symbols from the binary field GF(2). It is straightforward to generalize the theory developed for the binary codes to codes with symbols from a nonbinary field. First, linear block codes are defined and described in terms of generator and parity-check matrices. The parity-check equations for a systematic code are derived. Encoding of linear block codes is discussed. In Section 3.2 the concept of syndrome is introduced. The use of syndrome for error detection and correction is discussed. In Sections 3.3 and 3.4 we define the minimum distance of a block code and show that the random-error-detecting and random-error-correcting capabilities of a code are determined by its minimum distance. Probabilities of a decoding error are discussed. In Section 3.5 the standard array and its application to the decoding of linear block codes are presented. A general decoder based on the syndrome decoding scheme is given. References [1] through [6] contain excellent treatments of linear block codes. 3.1

INTRODUCTION TO UN!EAR BLOCK COIDIES

We assume that the output of an information source is a sequence of the binary digits 0 and 1. In block coding, this binary information sequence is segmented into message blocks of fixed length; each message block, denoted by u, consists of k information digits. There are a total of 2k distinct messages. The encoder, according to certain rules, transforms each input message u into a binary n-tuple v with n > k. This binary n-tuple vis referred to as the codeword (or code vector) of the message ID!. Therefore, corresponding to the 2k possible messages, there are 2k codewords. This set of 21c codewords is called a block code. For a block code to be useful, the 2k codewords must be distinct. Therefore, there should be a one-to-one correspondence between a message u and its codeword v. For a block code with 21c codewords and length n, unless it has a certain special structure, the encoding apparatus will be prohibitively complex for large k and n, since it has to store the 2k codewords of length n in a dictionary. Therefore, we restrict our attention to block codes that can be mechanized in a practical manner. A desirable structure for a block code to possess is linearity, which greatly reduces the encoding complexity, as we shall see. 3o1 A block code of length n and 2" codewords is called a linear if and only if its 2" codewords form a k-dimensional subspace of the vector space of all then-tuples over the field GF(2).

lDJEFKNITmN

(n, k) code

66

Section 3:l

Introduction to Linear Block Codes

157

In a block code is linear if and if the modulo-2 sum of two codewords is also a codeword. The block code given in Table 3.1 is a (7, 4) linear code. One can easily check that the sum of any two codewords in this code is also a codeword. Because an (n, k) linear code C is a k-dimensional of all the 11-tuples, it is to find k i;;o. 1/;1, · · · , 1/;k-l, in C such that every codeword Yin C is a linear combination of these k that v

= uoi;;o + u1g1 + · · · + uk-lgk-1,

(3.1)

where u; = 0 or 1 for O :Si < k. We arrange these k as the rows of a k >< 11 matrix as follows:

I

G~l

go g1 g1'-l

]

l

codewords

goo g10

go1 g11

g02 g12

go.11-1

gk-1.0

gk-1.l

gk-1,2

gk-1.n-l

gl.n-l

J ,

where I!§; = C!?iO, g;1, · · · , g;_ 11 -1) for O :S i < k. H llil = (uo, iq. · · · , uk-1) is the message to be encoded, the codeword can be as follows:

v=

llil ·

G

~:~ 1 I + ·· · +

= (uo, u1, · · · , uk-1) ·

gk-1

= uogo

+ u1g1

uk-lgk-1·

Clearly, the rows of G generate span) the (11, k) linear code C. For this reason, the matrix G is called a generator matrix for C. Note that any k linearly independent codewords of an (n, k) linear code can be used to form a matrix for the code. H follows from (3.3) that an (n, k) linear code is completely specified by the k rows of a generator matrix G. Therefore, the encoder has to store the k rows of G and to form a linear combination of these k rows based on the input message llil = (uo, u1. · · · . uk-1).

The (7, 4) linear code given in Table 3.1 has the matrix: 1 1 0 1 0 1 1 0 G= 1 1 1 0 1 0 1 0

l[l l

following matrix as a generator 0 1 0 0

0 0 1 0

n

68

Chapter 3

Linear Block Codes TABLE 3.1: Linear block code

with k

= 4 and n = 7. Codewords

(0 0 0 0) (10 0 0) (010 0) (110 0) (0010) (1010) (0110) (1110) (0 0 0 l) (10 0 l) (0 1 0 l)

(110 l) (0 0 11)

(1011) (011 l) (1111)

(0 0 0 0 0 0 0) (11010 0 0) (011010 0) (101110 0) (111001 0) (0011010) (1000110) (0101110) (1010001) (0111001) (1100101) (0001101) (0 1000 11) (100 1011) (0010111) (1111111)

If ill = (1 1 0 1) is the message to be encoded, its corresponding codeword, according

to (3.3), will be

V = 1 · go + 1 · g1 =

+ 0 · g2 + 1 · g3 (11 0 1 0 0 0) + (0 11 0 1 0 0) + (1 0 1 0 0 0 1)

= (0 0 0 11 0 1). A desirable property for a linear block code to possess is the systematic structure of the codewords, as shown in Figure 3.1, in which a codeword is divided into two parts, the message part and the redundant checking part. The message part consists of k unaltered information (or message) digits, and the redundant checking part consists of n - k parity-check digits, which are linear sums of the information digits. A linear block code with this structure is referred to as a linear systematic block code. The (7, 4) code given in Table 3.1 is a linear systematic block

Redundant checking part

r---n

~

Message part

k digits ______,,_'4----- k digits

--->-

FIGURE 3.1: Systematic format of a codeword.

Section 3.1

Introduction to Linear Block Codes

6~

the ak x form:

the

k >< k n1att1x

p matrix ~~-----"-----

iG=

- --~--~ --·-- - --- --

r==~~--~---.~-------------===.,,

!'lo

POO

PO!

PO.n-1:-1

1§1

PlO

Pn

Pl.11--k-l

P20

JJ21

P2.11-k-l

Pk-1.l

Pk--l.11-k-- l

!°l2 1§k-!

-- Pk-1.0

I

1

0

0

0 ·1

0

0

0

0

1 0

1

0

0

0

0

1

I

where PiJ = 0 or 1. Let Ilk denote the k >< k matrix. Let w1 = (110. iq .... , 11k-t) be the message to be encoded. codeword is

:=c

matrix
11

l

(IIQ.11],··· ,Uk-1) .(G.

H follows from (3.4) and

(3.6a)

for O °Si < k

Vn--k-1-i = l/;

and Vj

= UOJJOj + ll[Plj + .-+- llk--lPk--1.j

(3.6b)

0

for O s j < n - k. shows that the k of a codeword w are identical to the infonnation uo. 111, · · · • uk-1 to be and (3.6b) shows that the leftmost 11 - k redundant are linear sums of the information The 11 -- k by are called of the code.

The matrix G 3.1 is in the message to be vH'v~•,v•>, and let w = ( vo, codeword. Then,

Vl ·

1 1 0 W

=

(110. 1q. ll2, 11]) ·

l 0 1 1 1 0 1 0 1 0

0

J_

By matrix multiplication, we obtain the

V3

form. Let 1rn = (110, V5. be the

u1. u2, 1.13)

v2 0 V3 0 V4.

== LIO

v2=u1+u2-t-u3

0 1 0 0

0 0 J_

0

n

of the codeword w:

be

70

Chapter 3

Linear Block Codes

vi =uo+u1 +u2

The codeword corresponding to the message (1 0 1 1) is (1 0 0 1 0 1 1). There is another useful matrix associated with every linear block code. As stated in Chapter 2, for any k x n matrix G with k linearly independent rows, there exists an (11 - k) x n matrix H with n - k linearly independent rows such that any vector in the row space of G is orthogonal to the rows of H, and any vector that is orthogonal to the rows of H is in the row space of G. Hence, we can describe the (n, k) linear code C generated by Gin an alternative way as follows: Ann-tuple vis a codeword in the code C generated by G 1f and only if v ·HT= 0. The code is said to be the null space of H. This matrix H is called a parity-check matrix of the code. The 211 -k linear combinations of the rows of matrix H form an (n, n - k) linear code Cd. This code is the null space of the (11. k) linear code C generated by matrix G (i.e., for any v E C and any w E C1. v · w = 0). Cd is called the dual code of C. Therefore, a parity-check matrix for a linear code C is a generator matrix for its dual code Cc1. If the generator matrix of an (n, k) linear code is in the systematic form of (3.4). the parity-check matrix may take the following form:

II~ [i,,_,P'] - [

!

0 0 1 0 0 1 0

0 0 0

0

/Jk-1 0

POO

/710

Poi

PII

P02

/Jl2

Pk-I I /Jl,-1.2

PO.n-k-1

Pl.11-k-l

/Jl,-l n-A-1

l .

(3.7) where pT is the transpose of the matrix P. Let h.i be the jth row of H. We can readily check that the inner product of the ith row of G given by (3.4) and the jth row of H given by (3.7) is g; · hj = Pi.i -1- Pi.i = 0 for O :S i < k and O :S j < n - k. This implies that G. Hr =Cl.Also, the 11 - k rows of H are linearly independent. Therefore, the H matrix of (3.7) is a parity-check matrix of the (n. k) linear code generated by the matrix G of (3.4). The parity-check equations given by (3.6b) also can be obtained from the parity-check matrix H of (3.7). Let u = (uo. u 1. · · · • uk-1) he the message to be encoded. In systematic form the corresponding codeword will be '¥

=

(vo.

VJ, .. · , V11-k-l, llO, Uj •. "·, lik-[).

Using the fact that v · HT = 0, we obtain Vj + ll()/J()j +

u I Plj +···+Uk-I Pk-1.j

=0

(3.8)

for O s j < n - k. Rearranging the equations of (3.8), we obtain the same paritycheck equations of (3.6b ). Therefore, an (n, k) linear code is completely specified by its parity-check matrix.

Section 3.1

Introduction to Linear Block Codes

Consider the generator matrix of the (7, 4) linear code given in ,~,,~,."~ parity-check matrix is

n

3.1. The

0 1 0 1 1 ] . 1 0 1 1 1 0 0 1 0 1 1 1 0

At this point we summarize the foregoing results: For any (n, k) linear block code C there exists a k x n matrix iG whose row space gives C. Furthermore, there exists an (11 ~ k) x n matrix IHI such that an n-tuple w is a codeword in C if and if v · = ljj)_ H iG is of the form given by (3.4), then IHI may take the form given by (3.7), and vice versa. Based on the equations of (3.6a) and (3.6b ), the encoding circuit for an (n. k) linear systematic code can easily be implemented. The encoding circuit is shown in Figure where ~ denotes a shift-register stage (e.g., a EB denotes a modulo-2 and ----<>-
0 0 0

---[;;}l

FIGURE 3.2: Encoding circuit for a linear systematic

(11, k) code.

Tl

Chapter 3 Input

Linear Block Codes

1!11

0

~

To channel

)o~

o/

Parity register

FIGURE 3.3:

3.2

Encoding circuit for the (7, 4) systematic code given in Table 3.1.

SYNDROME AND !ERROR DETECTION

Consider an (11, k) linear code with generator matrix G and parity-check matrix H. Let v = (vo, v1, · · · . v11 _1) be a codeword that was transmitted over a noisy channel. Let r = (ro, r1, · · · , r11 _1) be the received vector at the output of the channel. Because of the channel noise, r may be different from v. The vector sum (3.9)

is an 11-tuple, where e; = 1 for r, -1- v,, and e, = 0 for r, = v,. This n-tuple is called the error vector ( or error pattern), which simply displays the positions where the received vector r differ from the transmitted codeword v. The 1's in e are the transmission errors caused by the channel noise. It follows from (3.9) that the received vector r is the vector sum of the transmitted codeword and the error vector; that is, r=v+e. Of course, the receiver does not know either v ore. On receiving r, the decoder must first determine whether r contains transmission errors. If the presence of errors is detected, the decoder will either take actions to locate the errors and correct them (FEC) or request a retransmission of v (ARQ). When r is received, the decoder computes the following (n - k)-tuple: §=lr·HT

(3.10)

=

(so, s1, · · · . s11-k-1).

which is called the syndrome of r. Then,§ = 0 if and only if r is a codeword, and§ -1- 0 if and only if r is not a codeword. Therefore, when § -1- 0, we know that r is not a codeword and the presence of errors has been detected. When§ = 0. r is a codeword, and the receiver accepts r as the transmitted codeword. It is possible that the errors in certain error vectors are not detectable (i.e., r contains errors buts = r · HT = 0).

Section 3.2

Syndrome and Error Detection

13

This happens when the error e is identical to a nonzero codeword. In this event, ff is the sum of two which is a codeword, and ff . IHIT = @. Error of this kind are called undetectable error Because there are i' - 1 nonzero codewords, there are 2" - 1 undetectable error patterns. When an undetectable error occurs, the decoder 1nakes a error. In a laier section of this we derive the of an undetected error for a BSC and show that this error probability can be made very small. Based on (3.7) and (3.10). the digits are as follows:

+ rn-kPOO + rn-k+lPlO + · · · + rn-lPk-1.0 = r1 + r11-kPOl + rn-k+lPll + · · · + rn-lPk-1.1

so= ro s1

(3.11) S11-k-l = i"n-k-l

+ rn-kPO.n-k-1 + rn-k+lPl.n-k-1 + · · · + r11-lPk-l.11-k-l·

H we examine the preceding equations we find that the the vector sum of the received parity digits (ro, r1, · · · , rn-k-1) and the digits from the received information digits (r 11 -k, rn-k+l· Therefore, the syndrome can be formed a circuit similar to the A general syndrome circuit is shown in Figure 3.4.

r11 _1). circuit.

··· .

Consider the (7, 4) linear code whose matrix is given in Example 3.3. Let ir = (ro, r1, r2, r3, r4. rs, r6) be the received vector. Then, the 1s by §=(so, s1, s2)

= (ro. r1, r2, r3. r4, rs, r6)

1 0 0 1 0 0 1 1 0 1 1 1

1 0

0 0 1 0 1 1 1

The syndrome digits are so

s1

= ro + r3 + rs + r6 = r1 + r3 + r 4 + r5

s2 = r2

+ q + r5 + r6.

The syndrome circuit for this code is shown in Figure 3.5. The syndrome § computed from the received vector ir depends only on the error pattern e and not on the transmitted codeword w. Because re is the vector sum of wand e, it follows from (3.10) that §=ii· l!--TIT = (w

+ e)IHIT

= w · IHIT

+ e. IHIT;

74

Chapter 3

Linear Block Codes

••• rn-k+I

t

r11~1

Pk-1.11-k-1

f

FIGURE 3.4: Syndrome circuit for a linear systematic (n, k) code.

FIGURE 3.5: Syndrome circuit for the (7, 4) code given in Table 3.1.

however, w. HT = 0. Consequently, we obtain the following relation between the syndrome and the error pattern: §=e-l-ll:T.

(3.12)

If the parity-check matrix H is expressed in the systematic form as given by (3.7), multiplying out e . HT yields the following linear relationship between the syndrome digits and the error digits: so

= eo + €11-kPOO + en-k+lPlO + · · · + €11-lPk-l,O

s1

= e1 + €11-kPOl + en-k+lPll + · · · + en-lPk-1,1 (3.13)

Sn-k-1 = €11-k-l

+ €11-kPO,n-k-l + en-k+lPl,n-k-1 + · · · + €11-lPk-l,11-k-l·

Section

and En-or Detection

75

are linear combinations of the error information about the error and therefore can be used for error one would think that any error correction scheme would be a the n -- k linear of for the error Once the + e -would be taken as the actual transmitted the true error vector e is not a matter. This is because the n - k linear do not have a solution but have wiH be are error patterns that result in the same and the true error e 1s one of them. the decoder has to determine the true error vector from a set of candidates. To minimize the most error that satisfies the true error vector. H the channel is a is the one that has the smallest number of nonzero The notion of for error correction may be clarified an

m 3.3. we consider the (7. 4) code whose matrix is Let v = (1 0 0 1 0 11) be the transmitted codeword and r (1 0 0 1 0 0 1) be the rt . the receiver the received vector. On §

= li" . JHlT

:.=

(111).

the receiver ""=''"' to determine the true error vector H follows from es, e6), which error are related to the the 0

e

=

(eo, e1, e2, e3, e4,

or

that the

+ e3 + es + e6 1 = e1 + e3 + e4 + es 1 = Q + q +es+ e6. 1 == eo

There are 24 = 16 error patterns that satisfy the preceding (0000010), (1101010). (0110110), (1011110), (1110000). (0011000), (1000100), (0101100).

namely,

(1010011), (0111011), (1100111). (0001111), (0100001). (1001001), (0010101), (1111101).

The error vector e = (000001 0) has the smallest number of nonzero components. H the channel is a BSC, e = (0 0 0 0 0 1 0) is the most probable error vector that satisfies the preceding equations. Taking e = (0 0 0 0 0 1 0) as the true error vector, the

76

Chapter 3

Linear Block Codes

receiver decodes the received vector r = (1 00100 1) into the following codeword: v*

= r +e = (1001001) + (0000010) = (1001011).

We see that v* is the actual transmitted codeword. Hence, the receiver has performed a correct decoding. Later we show that the (7, 4) linear code considered in this example is capable of correcting any single error over a span of seven digits; that is, if a codeword is transmitted and if only one digit is changed by the channel noise, the receiver will be able to determine the true error vector and to perform a correct decoding. More discussion on error correction based on syndrome is given in Section 3.5. Various methods of determining the true error pattern from then -k linear equations of (3.13) are presented in later chapters. 3.3

THE MINIMUM DISTANCE Of A 18:lOCK CODE

In this section we introduce an important parameter of a block code called the minimum distance. This parameter determines the random-error-detecting and random-error-correcting capabilities of a code. Let v = (vo, v1, · · · . v11 _ 1) be a binary n-tuple. The Hamming weight (or simply weight) of v. denoted by w(v). is defined as the number of nonzero components of v. For example, the Hamming weight of v = (10010 11) is 4. Let v and w be two n-tuples. The Hamming distance ( or simply distance) between v and w, denoted d(v, w). is defined as the number of places where they differ. For example, the Hamming distance between v = (1001011) and w = (0100011) is 3; they differ in the zeroth, first, and third places. The Hamming distance is a metric function that satisfies the triangle inequality. Let v, w, and x be three 11-tuples. Then, d(v. w)

+ d(w. x)

::=: d(v. x).

(3.14)

(The proof of this inequality is left as a problem.) It follows from the definition of Hamming distance and the definition of modulo-2 addition that the Hamming distance between two 11-tuples v and w is equal to the Hamming weight of the sum of v and w; that is, (3.15) d(v. w) = w(v + w). For example, the Hamming distance between v = (1001011) and w = (1 11001 0) is 4. and the weight of v + w = (0111001) is also 4. Given a block code C, one can compute the Hamming distance between any two distinct codewords. The minimum distance of C, denoted by dmin, is defined as !::,

dmin =

•

mm{d(v. w): v. w EC, v f w}.

(3.16)

If C is a linear block code, the sum of two codewords is also a codeword. It follows from (3.15) that the Hamming distance between two codewords in C is equal to the

Section 3.3

The Minimum Distance of a Block Code

7J7!

I'-famming weight of a third codeword in C. Then, it follows from (3.16) that (v

+ w)

: w. w E C, w f- w}

(x) : KE C, I,

f-

(3.17)

I[]))

W111in·

The linear code C. Tm!IEl(J)JRJE!\l!l

minimum

f- @) is called the minimum weight of the

,I

Wmin

result, we have the following theorem. 3\.]_ The minimum distance of a linear block code is equal to the of its nonzero codewords and vice versa.

Therefore, for a linear block code, determining the minimum distance of the code is to determining its minimum weighL The (7, 4) code given in Table 3.1 has minimum weight 3; thus, its minimum distance is 3. Next, we prove a nmnber of theorems that relate the weight structure of a linear block code to its parity-check matrix. 3\.Z Let C be an (n, k) linear code with parity-check matrix TI1l For each codeword of Hamming weight !, there exist I columns of JI-JI such that the vector sum of these l columns is equal to the zero vector. Conversely, if there exist l columns of JI-JI whose vector sum is the zero vector, there exists a codeword of Hamming weight I in C.

TmWiR.JEM

!PIJ'@([/)J: We express the parity-check matrix in the following form: Lf-[ = [llno. lln1, · · · , llnn-1],

where Ihli represents the ith column of JI-JI. Let w = ( vo, v1, · · · , v11 _ 1) be a codeword of weight l. Then, w has I nonzero components. Let vi 1 , vi 2 , · · · , Vi 1 be the I nonzero components of w, where O :S i 1 < i2 < · · · < i1 :S n - 1. Then, vi 1 = vi 2 = · · · = vi 1 = 1. Because w is a codeword, we must have (jJ)

=

'if

·JI-ITT

= Vi 1Ilni 1 +

Vi 2 illli 2

+ ···+

Vi 1Jlni 1

=llni1 +llni2+···+llni1·

This proves the first part of the theorem. Now. suppose that Ilni 1. lhli 2, · · · . llni 1 are I columns of JI-JI such that (3.18) VVe form a binary n-tuple x = (xo, x1, · · · , x11 _1) whose nonzero components are Xi 1 , xi 2 • · · · , Xi 1• The Hamming weight of% is/. Consider the product %-

JI-ITT

= xollno +

x1lhl1

+ · · · + X11-1lln11-l

78

Chapter 3

Linear Block Codes

It follows from (3.18) that x · HT = 0. Thus, xis a codeword of weight l in C. This proves the second part of the theorem. QoEJDlo The following two corollaries follow from Theorem 3.2. CoROJLJLARY 3o'.tl Let C be a linear block code with parity-check matrix H. If no d - 1 or fewer columns of H add to 0, the code has minimum weight at least d. COROLLARY 3o2J: Let C be a linear code with parity-check matrix H. The minimum weight (or the minimum distance) of C is equal to the smallest number of columns of H that sum to 0.

Consider the (7, 4) linear code given in Table 3.1. The parity-check matrix of this code is 1 0 0 1 0 1 H= 0 1 0 1 1 1 [ 0 0 1 0 1 1 We see that all columns of H are nonzero and that no two of them are alike. Therefore, no two or fewer columns sum to 0. Hence, the minimum weight of this code is at least 3; however, the zeroth, second, and sixth columns sum to 0. Thus, the minimum weight of the code is 3. From Table 3.1 we see that the minimum weight of the code is indeed 3. It follows from Theorem 3.1 that the minimum distance is 3. Corollaries 3.2.1 and 3.2.2 are generally used to determine the minimum distance or to establish a lower bound on the minimum distance of a linear block code. 3.4

IERROliH)IETIECliNG AND IERROR-CORRECliNG CA?ABiUHES OF A BLOCK CODE

When a codeword v is transmitted over a noisy channel, an error pattern of l errors will result in a received vector r that differs from the transmitted codeword v in l places [i.e., d(v, r) = !]. If the minimum distance of a block code C is dmin, any two distinct codewords of C differ in at least dmin places. For this code C, no error pattern of dmin - 1 or fewer errors can change one codeword into another. Therefore, any error pattern of dmin - 1 or fewer errors will result in a received vector r that is not a codeword in C. When the receiver detects that the received vector is not a codeword of C, we say that errors are detected. Hence, a block code with minimum distance dmin is capable of detecting all the error patterns of dmin - 1 or fewer errors. However, it cannot detect all the error patterns of dmin errors because there exists at least one pair of codewords that differ in dmin places, and there is an error pattern of dmin errors that will carry one into the other. The same argument applies to error patterns of more than dmin errors. For this reason we say that the random-error-detecting capability of a block code with minimum distance dmin is dmin l. Even though a block code with minimum distance dmin guarantees detection of all the error patterns of dmin - 1 or fewer errors, it is also capable of detecting a large fraction of error patterns with dmin or more errors. In fact, an (n, k) linear code is capable of detecting 211 - 21c error patterns of length n. This can be shown as follows. Among the 211 1 possible nonzero error patterns, there are 2k - l error

Section 3.4

Error-Detecting and Error-Correcting Capabilities o-f a Block Code

19

that are identical to the 2k - 1 nonzero codewords. H any of these 2k - 1 occurs, it alters the transmitted codeword w into another codeword w. Thus, w is and its syndrome is zero. fo this case, the decoder accepts ,w as the transmitted codeword and thus performs an incorrect decoding. there are 2k - 1 undetectable error patterns. U an error is not identical to a nonzero codeword, the received vector rr will not be a codeword and the will not be zero. In this case, an error will be detected. There are 211 - 2k error that are not identical to the codewords of an (11. k) linear code. These 211 error are detectable error patterns. For large n. 2k - 1 m much smaller than 211 • Therefore, only a small fraction of error pass the decoder without detected. Let C be an (11, k) linear code. Let A; be the number of codewords of i in C. The numbers Ao, A 1 , · · · . are called the weight distribution of C. U C is used only for error detection on a BSC, the that the decoder will fail to detect the presence of errors can be computed from the weight distribution of C. Let (E) denote the of an undetected error. Because an undetected error occurs when the error pattern is identical to a nonzero codeword of C, 11

(£) = LA;pi(l - p)n-i,

(3.19)

i=l

where p is the transition probability of the BSC. If the minimum distance of C is then A1 to Ac1111 ;,,-l are zero. Consider the (7, 4) code given i.n Table 3.1. The weight distribution of this code is Ao= 1, A1 = A2 = 0, A3 = 7, A4 = 7, As= A6 = 0, and A7 = 1. The of an undetected error is dmin,

Pu(E) = 7p3(1- p)

4

+ 7p 4 (1- p) 3 + p7.

H p = 10-2 , this probability is approximately 7 x 10- 6 . fo other if 1 million codewords are transmitted over a BSC with p = 10- 2 , on average seven erroneous codewords pass through the decoder without detected. If a block code C with minimum distance d111 ;11 is used for random-error one would like to know how many errors the code is able to correct. The minimum distance dmin is either odd or even. Let t be a integer such that 2t

+ 1 :<:'. d

111 ; 11

:<:'. 2t

+ 2.

(3.20)

Next, we show that the code C is capable of correcting all the error patterns of t or fewer errors. Let v and rr be the transmitted codeword and the received vector, respectively. Let w be any other codeword in C. The Hamming distances among w, ir,

and w satisfy the triangle inequality: d(v. rr)

+ d(w. rr·)

> d(v, w).

Suppose that an error pattern oft' errors occurs during the transmission of w. the received vector Ir differs from v int' places, and therefore d(w, rr) = t'. Because v and w are codewords in C, we have d(w. w) ::>: dmin ::>: 2t

+ 1.

(3.22)

80

Chapter 3

Linear Block Codes

Combining (3.21) and (3.22) and using the fact that d(v, r) following inequality: d(w,ir)::'.:2t+l-t'.

= t', we obtain the

If t' ::=: t'

d(w, ir) > t. The preceding inequality says that if an error pattern of t or fewer errors occurs, the received vector iris closer (in Hamming distance) to the transmitted codeword v than to any other codeword win C. For a BSC, this means that the conditional probability P(irlv) is greater than the conditional probability P(rlw) for w -:ft v. Based on the maximum likelihood decoding scheme, iris decoded into v, which is the actual transmitted codeword. The result is a correct decoding, and thus errors are corrected. In contrast, the code is not capable of correcting all the error patterns of l errors with l > t, for there is at least one case in which an error pattern of l errors results in a received vector that is closer to an incorrect codeword than to the transmitted codeword. To show this, let v and w be two codewords in C such that d(v, w) = dminLet e1 and e2 be two error patterns that satisfy the following conditions: io e1 + e2 = v + w. ii.o e 1 and e2 do not have nonzero components in common places.

Obviously, we have (3.23) Now, suppose that vis transmitted and is corrupted by the error pattern e1. Then, the received vector is Ir= V

+ e1.

The Hamming distance between v and ir is d(v, r)

= w(v + ir) = w(e1).

(3.24)

The Hamming distance between w and r is (3.25) Now, suppose that the error pattern e1 contains more than terrors (i.e., w(e1) > t). Because 2t + 1 ::=: dmin ::=: 2t + 2, it follows from (3.23) that

Combining (3.24) and (3.25) and using the fact that w(e1) > t and w(e2) ::=: t + 1, we obtain the following inequality: d(v, ir) ::'.: d(w, r).

and

Section 3A-

of a Block Code

~1

This says that there exists an error of l(l > t) errors that resuhs in a received vector that is closer to an incorrect codeword than to the transmitted codeword. Based on the maximum likelihood an In summary, a block code vvith minim1..m1 distance dmin correction of all the error patterns oft = l(dmin - 1)/2J or fewer errors, where L(d111 in - 1)/2J denotes the - 1) /2. The t = L(dmin - 1) /2J is called the of the code. The code is referred to as a minimum distance 3 and thus t any error of error over a block of seven A block code with of of t or fewer errors (this win be seen in the next

Ha code is used for error correction on a BSC with transition p., the that the decoder co:mrnits an erroneous is upper bounded by P(E) :'S, n

(

11 )

/(1- p)n-i.

l=t+l

Jc or fewer errors and detecting 1(l > A) or fewer errors. That when A or fewer errors occur, the code is capable of correcting when more than Jc but fewer than 1 + 1 errors occur, the code is capable of their presence without a error. For this purpose. the minim.um distance d111 i 11 of the code is at least A + 1 + 1 (left as a problem). Thus. a block code with d 111 i 11 = 10 is of three or fewer errors and simultaneously six or fevver errors. the case in which the receiver makes a hard· So far. we have considered however, a receiver may be to declare decision for each received a erased when it is received (or unreliably). In this case. the received sequence consists of zeros, ones, or erasures. A code can be used to correct combinations of errors and erasures. A code with minimum distance dmin is capable of correcting any of v errors and e erasures the following condition d111i11 ~ 2v

+e +1

is satisfied. To see this, delete from all the codewords the e components where the receiver has declared erasures. This deletion results in a shortened code of length 11 - e. The minimum distance of this shortened code is at least dmin - e ~ 2v + 1. Hence, v errors can be corrected in the unerased positions. As a result, the shortened codeword withe components erased can be recovered. Final.ly, because d 111 i11 ~ e + 1, there is one and only one codeword in the original code that agrees with the unerased components. Consequently, the entire codeword can be recovered. This correction of combinations of errors and erasures is often used in practice. From the preceding discussion we see that the random-error-detecting and random-error-correcting capabilities of a block code are determined the code's minimum distance. Clearly, for a given n and k, one would like to construct a block

82

Chapter 3

Linear Block Codes

code with as large a minimum distance as possible, in addition to the implementation considerations. Often an (n, k) linear block code with minimum distance dmin is denoted by (n, k, d111 i 11 ). The code given by Table 3.1 is a (7, 4, 3) code. 3.5

STANDARD ARRAY AND SYNDROME DECODING

In this section we present a scheme for decoding linear block codes. Let C be an (n, k) linear code. Let v1, v2, · · · , v 21. be the codewords of C No matter which codeword is transmitted over a noisy channel, the received vector r may be any of the 211 n-tuples over GF(2). Any decoding scheme used at the receiver is a rule to partition the 2" possible received vectors into 2k disjoint subsets D 1 , D2, · · · . D 21. such that the codeword v; is contained in the subset D; for 1 :s i :s 2k. Thus, each subset D 1 is one-to-one correspondence to a codeword v;. If the received vector r is found in the subset D;, r is decoded into v;. Decoding is correct if and only if the received vector r is in the subset D; that corresponds to the codeword transmitted. A method to partition the 211 possible received vectors into 2k disjoint subsets such that each subset contains one and only one codeword is described here. The partition is based on the linear structure of the code. First, we place the 2k codewords of C in a row with the all-zero codeword v1 = (0, 0, · · · . 0) as the first (leftmost) element. From the remaining 211 - 2" n-tuples, we choose an n-tuple e2 and place it under the zero vector v1. Now, we form a second row by adding e2 to each codeword v1 in the first row and placing the sum e 2 + v; under v;. Having completed the second row, we choose an unused n-tuple e3 from the remaining n-tuples and place it under v1. Then, we form a third row by adding e3 to each codeword v; in the first row and placing e3 + v; under vi. We continue this process until we have used all the 11-tuples. Then, we have an array of rows and columns, as shown in Figure 3.6. This array is called a standard array of the given linear code C. It follows from the construction rule of a standard array that the sum of any two vectors in the same row is a codeword in C. Next, we prove some important properties of a standard array. THEOREM 3.3 No two n-tuples in the same row of a standard array are identical. Every n-tuple appears in one and only one row.

Proof The first part of the theorem follows from the fact that all the codewords of C are distinct. Suppose that two 11-tuples in the 1th row are identical, say

=0

v2

Vi

">'2k

e2

e2 + v2

te2 + Vi

e2 + V21,

e3

e3 + v2

e3 + Vi

te3 + V21,

e1

Cf+ V2

e1 +vi

te/ + V21

<e211-1, + v2

te211-A + Vi

V[

<e211-/,

FIGURE

3.6: Standard array for an (n,

<e211

k)

-k

+ V21.

linear code.

Section 3.5

Standard Array and Syndrome Decoding

~3

er + 'V; = er + 'VJ with i i- j. This means that 'V; = 'VJ, which is impossible. Therefore, no two n-tuples in the same row are identical.

H follows from the construction rule of the standard array that every n-tuple appears at least once. Now, suppose that an n-tuple appears in both the lth row and the mth row with l < m. Then this must be equal to er + 'V; for some i and equal to <em + 'VJ for some j. As a result, <er+ w; = <e 111 + v.1. From this equality we obtain <em = er+ ('V; + Because v; and VJ are codewords in C, v; + is also a codeword in say Vs. Then <em = <e1 +'Vs-This implies that the <em is in the lth row of the array, which contradicts the construction rule of the array that <em, the first element of the mth row, should be unused in any previous row. Therefore, no 11-tuple can appear in more than one row of the array. This concludes the proof of the second part of the theorem. QJE:,!DJ, From Theorem 3.3 we see that there are 211 /2k = 2 11 -k disjoint rows in the standard array, and that each row consists of 21c distinct elements. The 211 -k rows are called the cosets of the code C, and the first n-tuple e.i of each coset is called a cos et leader ( or coset representative). The coset concept for a subgroup was presented in Section 2.1. element in a coset can be used as its coset leader. This does not change the elements of the coset; it simply permutes them.

Consider the (6, 3) linear code generated G=

the following matrix:

011100] 101 0 1 0 . [ 110001

The standard array of this code is shown in Figure 3.7. A standard array of an (n. k) linear code C consists of 21c disjoint columns. Each column consists of 2n-k n-tuples, with the topmost one as a codeword in C. Let denote the jth column of the standard array. Then,

= {'VJ, re2 + "IJ, e3 +

VJ. · · · , re 211-1.

+ "IJ},

(3.27)

Cos et leader 000000

011100

101010

110001

110110

101101

011011

000111

100000 010000 001000 000100 000010 000001 100100

111100 001100 010100 011000 011110 011101 111000

001010 111010 100010 101110 101000 101011 001110

010001 100001 111001 110101 110011 110000 010101

010110 100110 111110 110010 110100 110111 010010

001101 111101 100101 101001 101111 101100 001001

111011 001011 010011 011111 011001 011010 111111

100111 010111 001111 000011 000101 000110 100011

FIGURE 3.7: Standard array for a (6, 3) code.

84

Chapter 3

Linear Block Codes

where Vj is a codeword of C, and e2, e3, · · · , e 2,,-1c are the coset leaders. The 2k disjoint columns D1, D2, · · · , D21c can be used for decoding the code C as described earlier in this section. Suppose that the codeword v j is transmitted over a noisy channel. From (3.27) we see that the received vector ir is in D j if the error pattern caused by the channel is a coset leader. In this event, the received vector ir will be decoded correctly into the transmitted codeword Vj; however, if the error pattern caused by the channel is not a coset leader, an erroneous decoding will result. This can be seen as follows. The error pattern x caused by the channel must be in some coset and under some nonzero codeword, say in the lth coset and under the codeword v; f- 0. Then, x = <ez + v;, and the received vector is

Jr= 'lj

+ x

= <ez +

(v; +

Vj)

= e1

+

Vs,

The received vector ll' is thus in Ds and is decoded into Vs, which is not the transmitted codeword. This results in an erroneous decoding. Therefore, the decoding is correct if and only if the error pattern caused by the channel is a coset leader. For this reason, the 2n-k coset leaders (including the zero vector 0) are called the correctable error patterns. Summarizing the preceding results, we have the following theorem: Tl!-!!EOJREM 3,4! Every (n, k) linear block code is capable of correcting 2n-k error patterns.

To minimize the probability of a decoding error, the error patterns that are most likely to occur for a given channel should be chosen as the coset leaders. For a BSC, an error pattern of smaller weight is more probable than an error pattern of larger weight. Therefore, when a standard array is formed, each coset leader should be chosen to be a vector of least weight from the remaining available vectors. If coset leaders are chosen in this manner, each coset leader has minimum weight in its coset. As a result, the decoding based on the standard array is the minimum distance decoding (i.e., the maximum likelihood decoding). To see this, let ir be the received vector. Suppose that ll' is found in the ith column D; and /th coset of the standard array. Then, ir is decoded into the codeword v;. Because ll' = e1 + v;, the distance between ll' and v; is d(ll', v;)

= w(ll' +

v;)

= w(<e1

+ v; + v;)

= w(<ez).

(3.28)

Now, consider the distance between ll' and any other codeword, say v j, d(ll', v 1)

= w(r+v1 ) = w(e1

+v; +vi).

Because v; and vi are two different codewords, their vector sum, v; + v1, is a nonzero codeword, say Vs. Thus, (3.29) Because <ez and <ez + Vs are in the same coset and since w(<e1) ::: w(ez + vs), it follows from (3.28) and (3.29) that d(r, v;) < d(ll', v1 ).

This says that the received vector is decoded into a closest codeword. Hence, if each coset leader is chosen to have minimum weight in its coset, the decoding based on the standard array is the minimum distance decoding, or MLD.

Section 35

Let

Standard

and Syndrome Decoding

f§:5

denote the number of coset leaders of weight i. The numbers are called distribution of the coset leaders. of a decoding error. Because a these numbers, we can the decoding error occurs if and if the error pattern is not a coset the error for a BSC with transition p is 0t;

0to, 0t1. · · · , 0t 11

11

P(E)

= 1- LOl;p'(l -

p)"-'.

(3.30)

i=O

!EJ{AMIP>ILE 3L 71

Consider the (6, 3) code in 3.6. The standard array for this code is shown in Figure 3.7. The weight distribution of the coset leaders is 0to = 1, a 1 = 6, a2 = 1, and a3 = 0t4 = 0ts = Ol6 = 0. P(E) = 1 -- (1 - p) 6 - 6p(l - p) 5 - p 2 (1 - p) 4 .

For p

= 10- 2 , we have P(E)

~ 1.37 x 10- 3 .

An (n, k) linear code is capable of detecting 211 - 21c error patterns; however, it is capable of correcting 211 -k error patterns. For large n, 211 -k is a small fraction 11 of 2 - 2k. Therefore, the of a decoding error is much than the probability of an undetected error. Rlll[JE((J)JR.JEM 3,,§ For an (n. k) linear code C with minimum distance dmin, all the 11-tuples of weight t = L(dmin - 1)/2J or less can be used as coset leaders of a standard array of C. Han then-tuples of weight t or less are used as coset leaders, there is at least one 11-tuple of weight t + 1 that cannot be used as a coset leader.

JPml[})f. Because the minimum distance of C is dmin, the minimum weight of C is also dmin. Let x and y be two of weight t or less. the weight oh+ yis w(i,

+ y)

_::=

w(i;r)

+ w(y)

::'. 2t < dmin·

Suppose that i::<: and y are in the same coset; then, 'X + y must be a nonzero codeword in C. This is impossible, because the weight of 'X + y is less than the minimum weight of C. Therefore, no two 11-tuples of weight tor less can be in the same coset of C, and all then-tuples of ,Neight t or less can be used as coset leaders. Let w be a minimum weight codeword of C [i.e., w(w) = clminl Let 'X and y be two n-tuples that satisfy the following two conditions: ii, 'X

+ y = w.

ilii, Y, and y do not have nonzero components in common places.

It follows from the definition that w('X)

K

and y must be in the same coset and

+ w(y) = w(w) = dmin·

86

Chapter 3

Linear Block Codes

Suppose we choose y such that w(y) = t + 1. Because 2t + 1 .:S dmin .:S 2t + 2, we have w(x) = t or t + 1. U x is used as a coset leader, then y cannot be a Q.E.D. coset leader. Theorem 3.5 reconfirms the fact that an (n, k) linear code with minimum distance dmin is capable of correcting all the error patterns of L(dmin - l)/2J or fewer errors, but it is not capable of correcting all the error patterns of weight t + 1. A standard array has an important property that can be used to simplify the decoding process. Let H be the parity-check matrix of the given (n, k) linear code C. 3.6 All the 2k n-tuples of a coset have the same syndrome. The syndromes for different cosets are different.

TlH!JIWREM

Proof. Consider the coset whose coset leader is e1. A vector in this coset is the sum of e1 and some codeword v; in C. The syndrome of this vector is (e1

+ v;)HT = e1HT + v;HT = e1HT

(since v;HT = 0). The preceding equality says that the syndrome of any vector in a coset is equal to the syndrome of the coset leader. Therefore, all the vectors of a coset have the same syndrome. Let e1 and ez be the coset leaders of the jth and lth cosets, respectively, where j < l. Suppose that the syndromes of these two cosets are equal. Then,

e1HT = e1HT, (e1

+ e1)HT

= 0.

This implies that e1 + e1 is a codeword in C, say v;. Thus, e i + e1 = v;, and ez = e 1 + v;. This implies that ez is in the jth coset, which contradicts the construction rule of a standard array that a coset leader should be previously unused. Therefore, no two cosets have the same syndrome. Q.E.D. We recall that the syndrome of an n-tuple is an (n - k)-tuple, and there are 2 11 -k distinct (n - k)-tuples. It follows from Theorem 3.6 that there is a one-toone correspondence between a coset and an (n - k)-tuple syndrome; or there is a one-to-one correspondence between a coset leader (a correctable error pattern) and a syndrome. Using this one-to-one correspondence relationship, we can form a decoding table, which is much simpler to use than a standard array. The table consists of 2 11 -k coset leaders ( the correctable error patterns) and their corresponding syndromes. This table is either stored or wired in the receiver. The decoding of a received vector consists of three steps:

1. Compute the syndrome of r, r . HT. 2. Locate the coset leader ez whose syndrome is equal to r · HT. Then e1 is assumed to be the error pattern caused by the channel. 3. Decode the received vector r into the codeword v* = r + e1. The described decoding scheme is called the syndrome decoding or tablelookup decoding. In principle, table-lookup decoding can be applied to any (n, k) linear code. It results in minimum decoding delay and minimum error probability;

Section 3.5

Standard Array and Syndrome Decoding

~1

for large n - k. the implementation of this decoding scheme becomes impractical. and either a large storage or a complicated logic circuitry is needed. Several practical decoding schemes that are variations of table-lookup decoding are discussed in subsequent Each of these decoding schemes requires additional properties of a code other than the linear structure.

Consider the (7. 4) linear code given in Table 3.L The parity-check matrix, as in Example 3.3, is 1 0 0 1 0 1 lHI=

0

1 0

1

1

1

[ 0 0 1 0 1 1

n

The code has 23 = 8 cosets. and therefore there are eight correctable error patterns (including the all-zero vector). Because the minimum distance of the code is 3. it is capable of correcting all the error patterns of weight 1 or 0. Hence. all the 7-tuples of weight 1 or O can be used as coset leaders. There are (

b) + ( I ) =

8 such

vectors. We see that for the (7, 4) linear code considered in this example, the number of correctable error patterns guaranteed by the minimum distance is equal to the total number of correctable error patterns. The correctable error patterns and their corresponding syndromes are given in Table 3.2. Suppose that the codeword 'if = (1001011) is transmitted. and 11' = (1001111) is received. For decoding 11'. we compute the syndrome of 11':

1 0 0 0 1 0 0 0 1 §

=

(1 0 0 1 1 1 1)

1

1

0

=

(0 1 1).

0 1 1 1 1 1 1 0 1 3.2: Decoding table for the (7, 4) linear code given in Table 3.1.

TABLE

(10 0) (010) (001)

(100 0 0 0 0) (0 100000)

(110)

(0 0 0 1000) (0 0 0 0 100)

(011) (1 11) (1 0 1)

(0010 0 0 0) (0000010) (0 0 0 0 0 0 1)

88

Chapter 3

Linear Block Codes

From Table 3.2 we find that (0 1 1) is the syndrome of the coset leader e = (0 0 0 0 1 0 0). Thus, (0 0 0 0 100) is assumed to be the error pattern caused by the channel, and r is decoded into v*

= r+e = (1001111) + (0 0 0 0 100) = (1001011),

which is the codeword transmitted. The decoding is correct, since the error pattern caused by the channel is a coset leader. Now, suppose that v = (0000000) is transmitted, and r = (1000100) is received. We see that two errors have occurred during the transmission of v. The error pattern is not correctable and will cause a decoding error. When r is received, the receiver computes the syndrome: §

= r · Hf = (111).

From the decoding table we find that the coset leader e = (000001 0) corresponds to the syndrome s = (111). As a result, r is decoded into the codeword v* = r + e

= (1 0 0 0 1 0 0) + (0 0 0 0 0 1 0) = (1 0 0 0 11 0). Because v* is not the codeword transmitted, a decoding error is committed. Using Table 3.2, we see that the code is capable of correcting any single error over a block of seven digits. When two or more errors occur, a decoding error will be committed. The table-lookup decoding of an (n, k) linear code may be implemented as follows. The decoding table is regarded as the truth table of n switching functions: eo

=

fo(so, SJ, · · · , s11-1<-1),

€]

=

fi(so,

=

fn-1

€11-l

SJ, ... ' Sn-k-1),

(so,

Sl, · · · , Sn-k-1),

where so. s1, · · · , sn-k-1 are the syndrome digits, which are regarded as switching variables, and eo, q, · · · , e11 _1 are the estimated error digits. When these n switching functions are derived and simplified, a combinational logic circuit with the n - k syndrome digits as inputs and the estimated error digits as outputs can be realized. The implementation of the syndrome circuit was discussed in Section 3.2. The general decoder for an (n. k) linear code based on the table-lookup scheme is shown

Section 3.5

and

Standard Received vector buffer register r

Error-pattern-detecting circuit (a combinational logic circuit)

eu

I

e1

\I

~~-->(~

j Vo

I

i

~~
e,, 0

0

j

0

1

I

t

r,,--=--~1{~ I

I + vn -

Vi

1

Corrected output

FIGURE :t8: General decoder for a linear block code.

3.8. The cost of this decoder depends in combinational cucmt

of the

we consider the (7, code in Table 3.1. The syndrome circuit for this code is shown in 3.5. The table is given Table 3.2. From this table we form the truth table (Table 3.3). The switching for the seven error digits are

eo

= soAsi As~.

e2

= s0As~ = s 0i\s1As2. = so As{ i\s2.

e4

e6

where i\ denotes the logic-AND operation ands' denotes the logic-COMPLEMENT of s. These seven switching expressions can be realized seven 3-input AND gates. The complete circuit of the decoder is shown in Figure 3.9.

90

Chapter 3

Linear Block Codes TABLE 3.3: Truth table for the error digits of the

correctable error patterns of the (7, 4) linear code given in Table 3.1. Sylllldromes so

s1

0 1 0

0 0 1 0 1 1 1 0

0

1 0 1 1

Correctable error patterns (coset leaders)

s2

eo

e1

e2

e3

e4

es

e6

0

0 1 0 0 0 0 0 0

0 0 1 0 0 0 0 0

0 0 0 1 0 0

0 0 0 0 1 0 0 0

0 0 0 0 0 1 0 0

0 0 0 0 0 0 1 0

0 0 0 0 0 0 0 1

0

0 1 0

1 1 1

0

0

Corrected output

FIGURE 3.9: Decoding circuit for the (7, 4) code given in Table 3.1. 3.6

PROBABILITY OF AN UNDETECTED ERROR FOR LINEAR CODES OVER A BSC

If an (n, k) linear code is used only for error detection over a BSC, the probability of an undetected error Pu (E) can be computed from (3.19) if the weight distribution of the code is known. There exists an interesting relationship between the weight distribution of a linear code and the weight distribution of its dual code. This relationship often makes the computation of Pu(E) much easier. Let {Ao, A1, · · ·, A 11 )

Probability of an Undetected Error for linear Codes over a BSC

Section 3.6

!9Ji

be the distribution of an (n, k) linear code C, and let {Bo, B1, · · · , B11 } be the weight distribution of its dual code Cc1. Now, we represent these two weight distributions in form as follows: A(z) = Ao+ A1z

+ · · · + A11z 11 ,

B(z) =Bo+

+ .. ·+

Then, A(z) and B(z) are related A(z)

(3.31)

the following

= r( 11 -"\1 + 2) 11 B

(

1-

2

l+z

).

(3.32)

[13]. The polynomials A(z) and This identity is known as the MacWilliams B (z) are called the weight enumerators for the (11, k) linear code C and its dual Cc1. From the Mac Williams identity, we see that if the weight distribution of the dual of a linear code is the distribution of the code itself can be determined. As a result, this gives us more flexibility in the weight distribution of a linear code. Using the MacWilliams we can compute the probability of an undetected error for an (n, k) linear code from the weight distribution of its dual. First, we put the expression of (3.19) into the form: 11

Pu(E)

=

LA;/(l- p)1'-i i=l

=

(3.33)

11 ' A; ( _P_ ); (1- p) 11 ' \

l-p

L,,

i=l

z

=

p/(l

p) in A(z) of (3.31) and using the fact that Ao= 1, we obtain

the following A

(-p ) -1 = f (_E__); 1 p

l-p

b,,d

i=l

(3.34)

Combining (3.33) and (3.34), we have the following expression for the probability of an undetected error: Pu(E)

= (1- p) 11

[

A

(i ~ JJ)-1].

(3.35)

From (3.35) and the MacWiHiams identity of (3.32), we finally obtain the foHowing expression for Pu(E): Pu(E)

= r(n-k) B(l -

where

2p) - (1 - p) 11 ,

(3.36)

11

B(l - 2p)

=

L B;(l -

2p)i.

i=O

Hence, there are two ways for computing the probability of an undetected error for a linear code; often, one is easier than the other. If n - k is smaller thank, it is much easier to compute (£) from (3.36); otherwise, it is easier to use (3.35).

92

Chapter 3

Linear Block Codes

EXAMPLE 3.10

Consider the (7, 4) linear code given in Table 3.1. The dual of this code is generated by its parity-check matrix, 0 1 0

0 0 1

1 0 1 1 0 1

1 1 ] 1 0 1 1

(see Example 3.3). Taking the linear combinations of the rows of H, we obtain the following eight vectors in the dual code: (0 0 0 0 0 0 0),

(11 0 0 1 0 1),

(1 0 0 1 0 11),

(1 0 11 1 0 0),

(0 1 0 1 11 0),

(0 11 1 0 0 1),

(0 0 1 0 111),

(111001 0).

Thus, the weight enumerator for the dual code is B(z) = 1 + 7z 4 . Using (3.36), we obtain the probability of an undetected error for the (7, 4) linear code given in Table 3.1: This probability was also computed in Section 3.4 using the weight distribution of the code itself. Theoretically, we can compute the weight distribution of an (n, k) linear code by examining its 2k codewords or by examining the 2n-k codewords of its dual and then applying the MacWilliams identity; however, for large n, k, and n - k, the computation becomes practically impossible. Except for some short linear codes and a few small classes of linear codes, the weight distributions for many known linear codes are still unknown. Consequently, it is very difficult, if not impossible, to compute their probability of an undetected error. Although it is difficult to compute the probability of an undetected error for a specific (n, k) linear code for large n and k, it is quite easy to derive an upper bound on the average probability of an undetected error for the ensemble of all (n, k) linear systematic codes. As we showed earlier, an (n, k) linear systematic code is completely specified by a matrix G of the form given by (3.4). The submatrix P consists of k(n k) entries. Because each entry Pij can be either a O or a 1, there are 2k(n-k) distinct matrices G's of the form given by (3.4). Let r denote the ensemble of codes generated by these 2k(n-k) matrices. Suppose that we choose a code randomly from r and use it for error detection. Let C j be the chosen code. Then, the probability that Ci will be chosen is P(Cj)

= rk
(3.37)

Section .3.6

Probability of an Undetected Error for Linear Codes over a BSC

93

Let A Ji denote the number of codewords in C 1 with weight i. It follows from (3.19) that the of an undetected error for C1 is given by 11

Pu(EIC 1)

=

I>i;/(1- p)1'-i.

(3.38)

i=l

The average

of an undetected error for a linear code in

r is defined as

lfl

IP'l/(lE) =

L P(Cj)P11(EICJ),

(3.39)

j=l

where If I denotes the number of codes in r. Substituting (3.37) and (3.38) into (3.39), we obtain 1r1

11

lt\(JE)

= rk(n-k) L/(l -

p)n-i LAJi·

(3.40)

j=l

i=l

A nonzero n-tuple is contained in either exactly 2(k-lJ(n-kJ codes in the codes (left as a problem). Because there are (

1 ;

)

r or in none of

n-tuples of weight i, we have

(3.41) Substituting (3.41) into (3.40), we obtain the following upper bound on the average probability of an undetected error for an (n, k) linear systematic code: lP'u(lE) :': r(n-k)

t(

1; ) /

(1 - p)n-i

1=1

= r(n-k)[l

(3.42)

- (1 - p)/Jl

Because [1 - (1 - p )"] ::= 1, it is dear that IP'i, (lE) ::= 2-(n-k). The preceding result says that there exist (n, k) linear codes with the probability of an undetected error, Pu (E), upper bounded by 2-ln-kJ. In other words, there exist (n, k) linear codes with Pu (E) decreasing exponentially with the number of parity-check digits, n - le Even for moderate n - k, these codes have a very small probability of an undetected error. For example, let n - k = 30. There exist (n, k) linear codes for which Pu (E) is upper bounded by 2- 3o ~ 10- 9 . Many classes of linear codes have been constructed for the past five decades; however, only a few small classes of linear codes have been proved to have a Pu (E) that satisfies the upper bound 2-/n-k). H is still not known whether the other known linear codes satisfy this upper bound. Good codes for error detection and their applications in error control will be presented in later chapters. An excellent treatment of error-detecting codes can be found in [12].

94

3.7

Chapter 3

Linear Block Codes

SINGLE-PARITY-CHECK CODES, RIEPE1/UION CODIES, AND SELF-DUAL CODES

A single-parity-check (SPC) code is a linear block code with a single paritycheck digit. Let u = (uo, iq, ... , uk-1) be the message to be encoded. The single parity-check digit is given by

= UQ + Ul + · · · + Uk-1

p

(3.43)

which is simply the modulo-2 sum of all the message digits. Adding this parity-check digit to each k-digit message results in a (k + 1, k) linear block code. Each codeword is of the form V

=

(p, UO, Ul, ... , Uk-1),

From (3.43), we readily see that p = 1 if the weight of message u is odd, and p = 0 if the weight of message u is even. Therefore, all the codewords of a SPC code have even weights, and the minimum weight (or minimum distance) of the code is 2. The generator of the code in systematic form is given by ' 1,1000 ... 0

1,'

'

G=

'

1:0100 ... 0

1:

1:0010 ... 0

1

l'OOOQ ... J

l'

'

:n"

(3.44)

'

From (3.44) we find that the parity-check matrix of the code is (3.45)

lH! = [1 1 · .. 1].

Because all the codewords have even weights, a SPC code is also called an evenparity-check code. SPC codes are often used for simple error detection. Any error pattern with an odd number of errors will change a codeword into a received vector of odd weight that is not a codeword. Hence, the syndrome of the received vector is not equal to zero. Consequently, all the error patterns of odd weight are detectable. A repetition code of length n is an (n, 1) linear block code that consists of only two codewords, the all-zero codeword (0 0 · · · 0) and the all-one codeword (1 1 · · · 1). This code is obtained by simply repeating a single message bit n times. The generator matrix of the code is G

= [11 .. · l].

(3.46)

From (3.44) through (3.46), we readily see that the (n, 1) repetition code and the (n, n - 1) SPC code are dual codes to each other. SPC and repetition codes are often used as component codes for constructing long, powerful codes as will be seen in later chapters. A linear block code C that is equal to its dual code C,1 is called a self-dual code. For a self-dual code, the code length n must be even, and the dimension k of the code must be equal to n /2. Therefore, its rate R is equal to Let G be a generator matrix of a self-dual code C. Then, G is also a generator matrix of its dual code C,1 and hence is a parity-check matrix of C. Consequently,

!.

G · GT

=0

(3.47)

Problems

(Gisin


= [IP TI11;2]. From

j [ ' . ll

_'£

-

(3.48)

-11/2·

n/2) linear block code C satisfies the condition of self-dual code is left as a

a

•Consider the

see that

TD)T -1·1

m,

11

vve can

!l!5

linem· block code

~~~~iiii 1·

G=

l. The code has a rate R ::-.::

the matrix

00110011 01010101

.

-

!- His easy to check that CG· GT =

@.

it is a

self-dual code.

is the

'fhere are rnany 12)

3J_ Consider a

self-dual codes but the m.ost well known self-dual code 4. which will be discussed in

(8, 4) code whose parity-check equations are

v3=uo+u2+u3.

vvhere uo, u1, u2, and u3, are message digits, and vo, v1, v2, and v3 are paritycheck digits. Find the generator and parity-check matrices for this code. Show analytically that the minimum distance of this code is 4. Construct an encoder for the code given in Problem 3.1. Construct a syndrome circuit for the code given in Problem 3.1. Let JHl be the parity-check matrix of an (11, k) linear code C that has both oddand even-weight codewords. Construct a new linear code C1 with the following parity-check matrix:

'

IHI1

=

-

o: o:

lr ::

IHI

O'

--~-------

1:11 ... 1

(Note that the last row of!HI1 consists of all l's.) Show that C1 is an (11 + 1, k) linear code. is called an extension of C. rOlo Show that every codeword of C1 has even weight

i
96

Chapter 3

Linear Block Codes

c. Show that C1 can be obtained from C by adding an extra parity-check digit, denoted by v00 , to the left of each codeword v as follows: (1) ih has odd weight, then v00 = 1, and (2) if v has even weight, then v00 = 0. The parity-check digit v00 is called an overall parity-check digit. 3.5 Let C be a linear code with both even- and odd-weight codewords. Show that the number of even-weight codewords is equal to the number of odd-weight codewords. 3.6 Consider an (n, k) linear code C whose generator matrix G contains no zero column. Arrange all the codewords of C as rows of a 2k-by-n array. a. Show that no column of the array contains only zeros. b. Show that each column of the array consists of 2k-l zeros and 2k-l ones. c. Show that the set of all codewords with zeros in a particular component position forms a subspace of C. What is the dimension of this subspace? 3.7 Prove that the Hamming distance satisfies the triangle inequality; that is, let x, y, and z be three n-tuples over GF(2), and show that d(x, y)

+ d(y, z)

::: d(x, z).

3.8 Prove that a linear code is capable of correcting >.. or fewer errors and simultaneously detecting l (l > >..) or fewer errors if its minimum distance dmin ::: >.. + l + 1. 3.9 Determine the weight distribution of the (8, 4) linear code given in Problem 3.1. Let the transition probability of a BSC be p = 10- 2 . Compute the probability of an undetected error of this code. 3.10 Because the (8, 4) linear code given in Problem 3.1 has minimum distance 4, it is capable of correcting all the single-error patterns and simultaneously detecting any combination of double errors. Construct a decoder for this code. The decoder must be capable of correcting any single error and detecting any double errors. 3.11 Let r be the ensemble of all the binary systematic (11, k) linear codes. Prove that a nonzero binary 11-tuple vis contained in either exactly 2
Show that this code has a minimum distance of at least d 1 + d2. 3.14 Show that the (8, 4) linear code C given in Problem 3.1 is self-dual. 3.15 For any binary (n, k) linear code with minimum distance (or minimum weight) 2t + 1 or greater, show that the number of parity-check digits satisfies the following inequality:

The preceding inequality gives an upper bound on the random-error-correcting capability t of an (n, k) linear code. This bound is known as the Hamming

Bibliography

97

bound [14]0 (Hint: For an (n, k) linear code with minimum distance 2t + 1 or greater, all the n-tuples of t or less can be used as coset leaders in a standard arrayo) :3U«ii Show that the minimum distance dmin of an (11, k) linear code satisfies the following inequality: 11

dmin :':

0

2k-l

TI"

(Hint: Use the result of Problem 3o6(b )0 This bound is known as the Plotkin bound [1-3]o) 3o17 Show that there exists an (n, k) linear code with a minimum distance of at least d if

(Hint: Use the result of Problem 3011 and the fact that the nonzero 11-tuples of weight d - 1 or less can be at most in

(n' k) systematic linear code so) 3"1~ Show that there exists an (11, k) linear code with a minimum distance of at least dmin that satisfies the following inequality:

(Hint: See Problem 3017" The second provides a lower bound on the minimum distance attainable with an (11, k) linear code" This bound is known as the Varsharmov-Gilbert bound [1-3]0) 2U9 Consider a rate-~ (11, n/2) linear block code C with a generator matrix Go Prove that C is self-dual if Gr OGT = 1)))0 302® Devise an encoder for the (11, 11 - 1) SPC code with only one memory element (or flip-flop) and one X-OR gate (or modulo-2 adder)"

18l!18UOG~AIPHV ]_" E R Berlekamp, Algebraic Coding Theory, McGraw-Hill, New York, 1968; Revo edo, Aegean Park Press, Laguna NOY, 19840

:t

WO WO Peterson and E Jo Weldon, JL, Error-Correcting Codes, 2d edo, MIT Press, Cambridge, 19720

2t F Jo MacWilliams and Jo Jo A Sloane, The Theory of Error-Correcting Codes, North Holland, Amsterdam, 19770

4L R Jo McEliece, The Theory of Information and Coding, Addison-Wesley, Reading, Masso, 19770

98

Chapter 3

Linear Block Codes

5. G. Clark and J. Cain, Error-Correcting Codes for Digital Communications, Plenum Press, New York, 1981. 6. R. E. Blahut, Algebraic Codes for Data Transmission. Cambridge University Press, Cambridge, UK, 2003. 7. A. M. Michelson and A. H. Levesque, Error Control Techniques for Digital Communication, John Wiley & Sons, New York, 1985. 8. T. R. N. Rao and E. Fujiwara, Error-Correcting Codes for Computer Systems, Prentice Hall, Englewood Cliffs, N.J., 1989. 9. S. A. Vanstone and P. C. van Oorschot, An Introduction to Error Correcting Codes with Applications, Kluwer Academic, Boston, Mass., 1989. 10. A. Poli and L. Huguet, Error Correcting Codes, Theory and Applications, Prentice Hall, Hemel Hempstead, UK, 1992. 1L S. B. Wicker, Error Control Systems for Digital Communication and Storage, Prentice Hall, Englewood Cliffs, N.J., 1995. 12. T. Klove and V. I. Korzhik, Error Detecting Codes, Kluwer Academic, Boston, Mass., 1995. 13. F. J. Mac Williams, "A Theorem on the Distribution of Weights in a Systematic Codes," Bell Syst. Tech. I. 42: 79-94, 1963. 14. R. W. Hamming, "Error Detecting and Error Correcting Codes," Bell Syst. Tech. I. 29: 147-160, April 1950.

This chapter presents severai classes of important linear block codes that were discovered in the early days of error-correcting codes. The first class of linear block codes for error correction was discovered Richard W. in 1950 [1 ], two years after Shannon published his landmark paper, which asserted that by proper encoding of information, errors induced a noisy channel or storage medium can be reduced to any desired level without sacrificing the rate of information transmission or storage. Hamming codes have a minimum distance of 3 and therefore are vu,~_.,;,~ of correcting any single error over the span of the code block length. The weight enumerator of Hamming codes is known. Hamming codes are pe1ject codes and can be decoded easily using a table-lookup scheme. Good codes with a minimum distance of 4 for single-error correction and double-error detection can be obtained by properly shortening the Hamming codes. Hamming codes and their shortened versions have been widely used for error control in digital communication and data storage systems over the years owing to their high rates and decoding simplicity. The second class of linear block codes constructed in the early days for error correction and detection was the class of Reed-Muller codes. Reed-Muller codes were first discovered by David E. Muller in 1954 [9] for switching-circuit design and error detection. H was Irwin S. Reed, also in 1954 [10], who reformulated these codes for error correction and detection in communication and data storage systems and devised the first decoding algorithm for these codes. Reed- Muller codes form a large class of codes for multiple random error correction. These codes are simple in construction and rich in structural properties. They can be decoded in many ways using either hard-decision or soft-decision decoding algorithms. The Reed decoding algorithm for Reed- Muller codes is a majority-logic decoding algorithm that can easily be implemented. Several soft-decision decoding algorithms for Reed-Muller codes have been devised that achieve very good error performance with low decoding complexity. Also presented in this chapter is the (24, 12) Golay code with minimum distance 8. This code has many interesting structural properties and has been extensively studied by many coding theorists and mathematicians. H has been used for error control in many communication systems, especially by U.S. space communication programs. Several additional code construction methods are also presented in this chapter. These construction methods allow us to construct long, powerful codes from short component codes.

100

4.1

Chapter 4

Important Linear Block Codes

HAMMING CODES

For any positive integer m :::: 3, there exists a Hamming code with the following parameters: n = 2111 -1, k = 2111 - m - l, n -k = m, t = 1 (dmin = 3).

Code length: Number of information symbols: Number of parity-check symbols: Error-correcting capability:

The parity-check matrix IHI of this code consists of all the nonzero m-tuples as its columns. In systematic form, the columns of lH[ are arranged in the following form:

=[

lHI

Il111

Q ],

where i!111 is an m x m identity matrix, and the submatrix Q consists of 2111 - m - 1 columns that are the m-tuples of weight 2 or more. For example, let m = 3. The parity-check matrix of a Hamming code of length 7 can be put in the form H=

1001011] 0101110 , [ 0010111

which is the parity-check matrix of the (7, 4) linear code given in Table 3.1 (see Example 3.3). Hence, the code given in Table 3.1 is a Hamming code. The columns of Q may be arranged in any order without affecting the distance property and weight distribution of the code. In systematic form, the generator matrix of the code is

where QT is the transpose of Q, and lJ.i111 _111 __ 1 is a (2111 - m - l) x (2 111 - m -1) identity matrix. Because the columns of H. are nonzero and distinct, no two columns add to zero. It follows from Corollary 3.2.1 that the minimum distance of a Hamming code is at least 3. Since lH[ consists of all the nonzero m-tuples as its columns, the vector sum of any two columns, say lhl, and lhi;, must also be a column in H, say lhl1. Thus, h.,

+ Hnj + 1h11

=-cc

O.

It follows from Corollary 3.2.2 that the minimum distance of a Hamming code is

exactly 3. Hence, the code is capable of correcting all the error patterns with a single error or detecting all the error patterns of two or fewer errors. If we form the standard array for the Hamming code of length 2117 - 1, all the (2 111 - 1)-tuples of weight 1 can be used as coset leaders (Theorem 3.5). The number of (2111 - 1)-tuples of weight 1 is 2111 - 1. Because n - k = m, the code has 2 111 cosets. Thus, the zero vector O and the (2111 - 1)-tuples of weight 1 form all the coset leaders of the standard array. Thus, a Hamming code corrects only error patterns of single error and no others. This is a very interesting structure. A t-error-correcting code is called a perfect code if its standard array has all the

Section LL 1

Codes

1«n

error of t or fewer errors and no others as coset leaders. codes form a codes Perfect codes are rare. Besides the codes, the only other nontiivial binary code is code (see Section [3-5]. be decoded with the caiuis,-"Jv1"u,u in Section 3.5. The decoder for a in the same manner as that for the 4-) V\/e may delete any l columns frorn the code. This deletion results an m x (2 111 - l -1) matrix IPJ'. v,,1e obtain a shortened code with the n=2"'--l--l = 2111 -m - l - l n--k=m k

dmin =:,: 3.

If we delete columns from Ifi1 we may obtain a shortened code \Nith a minimum distance of 4. For if vve delete from the submatrix (Q) all the columns of even we obtain an m x 2111 - 1 huJ'

=[

TI111

CO!' ] .

where consists of -- m columns of odd Because all the columns of IHI' have odd no three columns adcl to zero; however, for a column !hi; of 3 in \Qi', there exist three columns lhi.i, lin1. and !hi.,· in IT,,, such that !hi, + Ili1.i + lhi1 + !his = illl. the shortened code with JHI' as a matrix has a minimum distance of 4. all error The distance.Lt shortened of double error and error occurs the transmission of a code vector. the and it contains an odd number of when double errors occur. the is also nonzero, but it contains an even number of l's. Based on these can be as follovvs: JL H the

:t

§

is zero, we assume that no error occurred.

§ is nonzero and it contains an ocld number of 1 we assume that a single error occurred. The errOI pattern of a error that to § is added to the received vector for error correction. J, If s is nonzero and it contains an even number of l's, an uncorrectable error n~H,>no has been detected.

If

The distribution of a Hamming code of length n =--= 2111 - 1 is known [2]. The number of code vectors of weight i, . is simply the coefficient of zi in the of the A(z)

= ~-1 -{(1 + z)" + n(l 11

This

is the

+1

- z)(l - z2)(n-])/ 2 }.

enumerator for the Hamming codes.

(4.1)

102

Chapter 4

Important Linear Block Codes

EXAMPLE 4.1

Let m = 3. Then, n = 23 -1 = 7, and the weight enumerator for the (7, 4) Hamming code is A(z) = }{(1 + z) 7 + 7(1 - z)(l - z2) 3 } = 1 + 7z 3 + 7z 4 + z7 . Hence, the weight distribution for the (7, 4) Hamming code is Ao= 1, A3 and A7 = 1.

= A4 = 7,

The dual code of a (2111 - 1, 2111 - m - 1) Hamming code is a (2111 - 1, m) linear code. This code has a very simple weight distribution; it consists of the all-zero codeword and 2111 - 1 codewords of weight 2"'- 1 . Thus, its weight enumerator is B(z)

= 1 + (2

111

-

l)z

1

2111 -

(4.2)

•

If a Hamming code is used for error detection over a BSC, its probability of an undetected error, Pu (E), can be computed either from (3.35) and (4.1) or from (3.36) and ( 4.2). Computing Pu (E) from (3.36) and (4.2) is easier. Combining (3.36) and (4.2), we obtain Pu(E) =

r

111

{1

+ (2

111

-

l)(l -2p) 2/}/-I}- (1- p/

111 -

1.

(4.3)

The probability Pu (E) for Hamming codes does satisfy the upper bound 2-
i

4.2

A CLASS OF SINGLE-ERROR-CORRECTING AND DOUBLE-ERROR-DETECTING CODES

Single-error-correcting and double-error-detecting (SEC-DED) codes have been widely used for error control in communication and computer memory systems. In this section we present a class of SEC-DED codes that are suitable and commonly used for improving computer memory reliability. This class of codes was constructed by Hsiao [8]. The most important feature of this class of codes is their fast encoding and error detection in the decoding process, which are the most critical on-line processes in memory operations. SEC-DED codes that have the features described can be constructed by shortening Hamming codes presented in the previous section. Construction begins with a Hamming code of length 2111 - 1 and minimum distance 3. Let H be its parity-check matrix. Delete columns from H such that the new parity-check matrix Ho satisfies the following requirements: 1. Every column should have an odd number of l's. 2. The total number of 1's in the Ho matrix should be a minimum. 3. The number of 1 's in each row of Ho should be made equal, or as close as possible, to the average number (i.e., the total number of 1 's in Ho divided by the number of rows).

Section 4.2

A Class of

Codes

TABLE 4.1:

Parameters of a list of Hsiao's codes. 1['(j])~:ffill 111l\lll!l1lll\b)err

n

12 14 15 16 22 26 30 39 43 47 55 72

80 88 96 104 112 120 128 130 137

1k

n - lk

(j])jf ]_'§ Illl1l lH[

8

4 5 5 5 6 6 6 7 7 7 7 8 8 8 8 8 8 8 8 9 9

16 32 35 40 54 66 86 103 117 157 177 216 256 296 336 376 416 456 512 446 481

9

10 11 16 20 24 32 36

40 48 64 72

80 88 96 104 112 120 121 128

111ll3l

Awerr:rn~e 111lllilrnm\bJerr (j])jf

li'§

fil)teff

rnw

4 6.4 7 8 9

11 14.3 14.7 16.7 22.4 25.3 27 32 37 42 47 52 57 64 49.6 53.5

The first requirement guarantees the code generated lliio has a minimum distance of at least 4. Therefore, it can be used for single-error correction and double-error detection. The second and third requirements minimum logic levels in forming parity or syndrome bits, and less hardware in implementation of the code. Hsiao [8] provided an to construct lliio and found some optimal SEC-DED codes. Several parity-check matrices in systematic form for message (or data) lengths of 16, 32, and 64 are given in Figure 4.1. The parameters of a list of Hsiao's codes are given in Table 4.1. fo computer applications, these codes are encoded and decoded in parallel manner. In encoding, the message bits enter the encoding circuit in parallel, and the parity-check bits are formed simultaneously. In decoding, the received bits enter the decoding circuit in parallel, the syndrome bits are formed simultaneously, and the received bits are corrected in parallel. Single-error correction is accomplished by the table-lookup decoding described in Example 3.9. Double-error detection is accomplished by examining the number of l's in the syndrome vector §. H the syndrome§ contains an even number of l's, then either a double-error pattern or a multiple-even-error pattern has occurred. Consider the parity-check matrix of the (72, 64) SEC-DED code given in Figure 4.l(c). Each row contains 27 ones. If three-input X-OR gates are used to form syndrome bits, each syndrome bit is formed by a three-level X-OR tree with

104

Chapter 4

Important linear Block Codes 1000001001100100111100 0100000011111010001010 0010001110111001100000 0001001110000111010001 0000100001001111000111 0000010100010000111111 (a)

100000010001010100000100000111100011011 010000000010000000111110111000101100001 001000000010110111100001001001010100110 Ho= 000100011111111000000011010010001000100 000010001101100111111110000100000001000 000001000100001001001001111111110010000 000000111000001010010000100000011111111 (b)

01000000 01100100 11111111 00000111 00111000 11001000 00001000 00001001 10010010 00100000 10010010 01100100 11111111 00000111 00111000 11001000 00001000 00001001 00010000 00001001 10010010 01100100 11111111 00000111 00111000 11001000 00001000 IH!o= 00001000 00001000 00001001

10010010 01100100 11111111 00000111 00111000 00001000

00000100 11001000 00001000 00001001 10010010 01100100 11111111 00000111 00000010 00111000 11001000 00001000 00001001 00000001

11001000

10010010 01100100 11111111 00000111

00000111 00111000 11001000 00001000 00001001

10010010 01100100 11111111

(c)

10000000 11111111 00001111 00001111 00001100 01101000 10001000 10001000 10000000 01000000 11110000 11111111 00000000 11110011 01100100 01000100 01000100 01000000 00100000 00110000 11110000 11111111 00001111 00000010 00100010 00100010 00100110 Ho=

00010000 11001111 00000000 11110000 11111111 00000001 00010001 00010001 00010110 00001000 01101000 10001000 10001000 10000000 11111111 00001111 00000000 11110011 00000100 01100100 01000100 01000100 01000000 11110000 11111111 00001111 00001100 00000010 00000010 00100010 00100010 00100110 11001111 00000000 11111111 00001111 00000001 00000001 00010001 00010001 00010110 00110000 11110000 11110000 11111111 (d)

FIGURE 4.1: (a) Parity-check matrix of an optimal (22, 16) SEC-DED code; (b) parity-check matrix of an optimal (39, 32) SEC-DED code; (c) parity-check matrix of an optimal (72, 64) SEC-DED code; (d) parity-check matrix of another optimal (72, 64) SEC-DED code.

Section 4.3

Reed-Muller Codes

it!!:ii

13 gates. The eight X-OR trees for generating the eight syndrome bits are identical. These provide uniform and minimum in the error-correction process. ,/j],,3;

~IEIE[l)-MlRl!E~, (O[l)!ES

Reed-Muller (RM) codes form a class of multiple-error-correction codes. These codes were discovered by Muller in 1954 [9], but the first decoding algorithm for these codes was devised by Reed, also in 1954 [10]. They are simple in construction and rich in structural properties. They can be decoded easily in several ways using either hard- or soft-decision algorithms. For any integers m and r with O :S r :s m, there exists a binary rth-order RM code, denoted by RM(r. m), with the following parameters: Code length: n = 2"', Dimension: k(r. m) = 1 +Cr)+ G) Minimum distance: dmin = 2m-r,

+ · · · + (';').

where (m) = (111 ~.7l ') 1.1.• 1 is the binomial coefficient. For example, let m = 5 and r = 2. I Then. n = 32. k(2. 5) = 16. and dmin = 8. There exists a (32, RM code with a minimum distance of 8. For 1 :S i :Sm, let w; be a 2m-tuple over GF(2) of the following form: wl =

(0 · ..

0, 1 .. -1 0 .. · 0 · .... · . '-=-=y==" 1 .. · 1)

~_, ~ ' ~ '

2i-1

which consists of 2m-i+l alternating all-zero and all--one i- 1 -tuples. Form have the following four 16-tuples: '¥4 V3 V2

V1

(4.4)

21-I

= 4. we

= (0 0 0 0 0 0 0 0 11111 111), = (0 0 00111100001111), = (0 0 11 0 0 1100 11001 1), = (0 1010101 0 1010101).

Let !iil = (ao, a1, · · · , a 11 _1) and lb= (ho, b1, · · · • b11 _1) be two binary n-tuples. We define the following logic (Boolean) product of mand lb: 6

2l

·lb= (ao · ho, Cll · b1, · · · , G11-l · bn_i),

where "." denotes the logic product (or AND operation), i.e., ai . bi = 1 if and only if ai = h1 = 1. Form = 4, '¥3 · '¥2

= (0 0 0 0 0 0 1 1000 0 0 0 11).

For simplicity, we use 21lb for 21 · lb. Let vo denote the all-one 2111 -tuple, vo = i1 :S m, the product vector

(1.

1. · · · , 1). For 1 :S i1 < i2 < · · · <

106

Chapter 4

Important Linear Block Codes

is said to have degree I. Because the weights of v1, v2, · · · , v 111 are even and powers of 2, the weight of the product v; 1wi 2 · · · Wi1 is also even and a power of 2, in fact, 2111 - 1. The rth-order RM code, RM(r, m ), of length 2m is generated (or spanned) by the following set of independent vectors: (4.5)

· · · , up to products of degree r}. There are k(r, m)

= 1+

G+ (7;) + ··· + (7) 1

)

vectors in GRM(r, m). Therefore, the dimension of the code is k(r, m). If the vectors in GRM(r, m) are arranged as rows of a matrix, then the matrix is a generator matrix of the RM(r. m) code. Hereafter, we use GRM(r, m) as the rows in GRM(r, m) of weight generator matrix. For O:::: I :::c r, there are exactly 2111 - 1. Because all the vectors in GRM(r, 111) are of even weight, all the codewords in the RM(r, m) code have even weight. From the code construction we readily see that the RM(r - 1, m) code is a proper subcode of the RM(r, m) code. Hence, we have the following inclusion chain:

en

RM(O, m) c RM(l, m) c · · · C RM(r, m).

(4.6)

Furthermore, RM codes have the following structural property: the (m - r - l)thorder RM code, RM(m - r - 1, m), is the dual code of the rth-order RM code, RM(r, m) (see Problem 4.9). The zeroth-order RM code is a repetition code and the (m - l)th-order RM code is a single-parity-check code.

IEXAMPllE 4.2 Let 111 = 4 and r = 2. The second-order RM code of length n = 16 is generated by the following 11 vectors:

wo W4 V3 Vz VJ W3V4 VzW4 V(o/4 VzV3 V1V3 v1v2

1111111111111111 0000000011111111 0000111100001111 0011001100110011 0101010101010101 0000000000001111 0000000000110011 0000000001010101 0000001100000011 0000010100000101 0001000100010001

This is a (16, 11) code with a minimum distance of 4. With the preceding construction, the generator matrix GRM(r, 111) of the RM(r, 111) code is not in systematic form. H can be put in systematic form with

Section 4.3

Reed-Muller Codes

W1

row and column form have many and useful structures that reduce Uvv~<~rnc;,. v~H<1fJ.Cvr,H.f This will be discussed in a later The Reed for RM codes is best by an example. Consider the second-order RM code RM(2, of 1n 4.2. nn,µ;,•s '"

Note that the sum of the first four of each vector is zero the vector v1 w2. The same is true for the other three groups of four consecutive As a we have the four sums that relate the information bit 012 to the code bits:

+ b2 + b3, a12 = be + bs + b6 + b7, a12 = bs + b9 + b10 + b11, a12 = b12 + bn + b14 + bis-

012

=ho+ b1

These four sums four determinations of the information bit 012 from the code bits. H the codevvord (bo, b1, · · · , bis) is transmitted and there is a transmission error in the received vector, the error can affect one determination of a12. As a the other three determinations of 012 will the correct value of 012- This is the basis for RM codes. Let fi' = (ro. r1, · · · be the received vector. In 012, we form the sums: A1

= ro + r1 + r2 + r3,

+ r7, = rg + r9 + r10 + r11. A4 = r12 + r13 + r14 + r15.

A2 =

r4

+rs+ r6

replacing the code bits ',i\/ith the corresponding received bits in the four determinations of 012- These sums are called check-sums, which are the estimates of 012. Then, 012 is decoded based on the following majority-logic decision rule: 012 is taken to be to the value assumed by the majority in {A1, . A4}. If there is a a random choice of the value of a 12 is made. h is dear that if there is only one error in the received vector, a 12 is decoded correctly.

108

Chapter 4

Important Linear Block Codes

Similar independent determinations of information bits a13, a23, a14, a24, and a34 can be made from the code bits. For example, the four independent determinations of a13 are: a13 = bo + b1 + b4 + bs, a13 a13

= b2 + b3 + b6 + b7, = bs + b9 + b12 + b13,

a13 = b10

+ b11 + b14 + b15.

At the decoder, we decode a 13 by forming the following four check-sums from the received bits using the preceding four independent determinations of a13:

= ro + r1 + r4 + rs, A2 = r2 + r3 + r6 + r7, A3 = rs + r9 + r12 + r13, A4 = r10 + r11 + r14 + r15.

A1

From these four check-sums we use the majority-logic decision rule to decode a13H there is a single transmission error in the received sequence, the information bits a12, a13, a23, a14, a24, and a34 will be decoded correctly. After the decoding of a12, a13, a23, a14, a24, and a34, the vector

is subtracted from r. The result is a modified received vector: r(l)

=

(r(l) 0 '

r(l)

1 '

·· ·

r(l))

' 15

In the absence of errors, r
+ a4v4 + a3v3 + a2v2 + afv1 = (b0(1) , b 1(1) , · · ·

(1) , b 15 ).

We note that starting from the first component, the sums of every two consecutive components in vo, v4, v3, and v2 are zero; however, the sum of every two consecutive components of v1 is equal to 1. As a consequence, we can form the following eight independent determinations of the information bit a 1 from the code bits 1) through bf~):

b6

a1 a1 a1 a1 =

+ b(l) 1 ' b(l) + b(l) 2 3 , b(l) + b(l) 4 5 (l) + l (1) b6 ?7 , b(l)

o

- b(l) a1 8 a1

=

a1 = a1 =

(1)

+ b(l) 9 ' (1)

+ bll ' (1) (1) h12 + b13 , (1) (1) bl4 + blS .

blO

Similar independent determinations of a2, a3, and a4 can be formed. In decoding we form the following check-sums from the bits of the modified received vector

a 1,

Section LU

Reed-Muller Codes

11Ql!9J

1r< 1l and the preceding eight independent determinations of a1:

A1(l) _A 2(l) -_ A3(l) --

A4(l) --

+ 1)1) 1 ' 1.(1) + /Jl) 2 3 ' 1.(1) + 1.(1) 4 5 ' /Jl) + /Jl) 6 7 ' ,.Cl)

0

A~l)

= ril) + r~l)'

A6(l)

_ 1.(1) -'- ,Jl) 10 I 11 ' (l) _ 1.(1) ,Jl)

A7 A8(l) --

+ '13 ' ,Jl) + ,Jl) 14 15 · 12

From these check-sums we decode cq using the decision rule. we can decode the information bits a2, a3, and a4. After the decoding of a1, a2, a3, and a4, we remove the effect of cq, a2, a3, and a4 from 1rC 1l and form the following modified received vector: Jr(2)

=

(r(2) r(2)

0

= Il°(l)

' l -

.. .

'

r(2))

' 15

a4V4 - a3v3 - a2v2 - a1v1.

fo the absence of errors, rr.i 2l is the following codeword: aowo

=

(oo, ao. · · · , ao).

This result gives 16 independent determinations of ao. fo decoding ao, we set o0 to the value taken by the majority of the bits in Il°(2l. This step completes the entire decoding. The demonstrated decoding is referred to as majority-logic decoding. Because it consists of three steps ( or levels) of decoding, it is called three-step decoding. It can easily be implemented using majority-logic elements. If there is only one error in the received vector ir, the information bits a12, a13, 014, a23, a24, and a34 will be correctly decoded. Then, the modified received vector irUl will stiH contain a single error at the same location. This single error affects only one of the eight check-sums for a;, with 1 ::::: i ::::: 4. The other seven check-sums give the correct value of o;. Therefore. the information bits cq, 02, a3, and a 4 will be correctly decoded. As a result, the next modified received vector ir( 2 ) will contain only one error (still at the same location). and the information bit ao will be correctly decoded. If there are two transmission errors in rt, these two errors may affect two check-sums for information bit a;J. In this case, there is no majority in the four check-sums; two check-sums take 0, and two other check-sums take 1. A choice between these two values as the decoded value of ct;J may result in an incorrect decoding of a;J. This incorrect decoding affects the subsequent levels of decoding and results in error propagation. Consequently, the decodings of a1, 02, 03, a4, and a 0 are very likely to be incorrect. Because the code guarantees correcting any single error but not two errors, its minimum distance is at least 3 but less than 5. Since all the codewords have even weight. the minimum distance of the code must be 4. We have used an example to introduce the concepts of majority-logic decoding and the multiple-step decoding process of the Reed algorithm. Now we are ready to present the general algorithm for decoding RM codes. The major part of the decoding is to form check-sums at each decoding step.

110

Chapter 4

Important Linear Block Codes

Consider the rth-order RM code, RM(r, m). Let !11

= (ao, a1, ...

'a,n' a1,2, ... , am-l.m' ... , a1.2, ... ,r'

... ,

am-r+l,m-r+2, .. ,111)

be the message to be encoded. The corresponding codeword is b

=

(bo, b1, · · · , b11-1)

=aovo+

L

L

a; 1V; 1 +

l.c:i1:C:111

a; 1a; 2 V; 1v; 2 (4.7)

l:C:i1
+···+ Let r = (ro, r1, · · · , r11 _1) be the received vector. Decoding of RM(r, m) code consists of r + 1 steps. At the first step of decoding, the information bits a; 1; 2 ... ;, corresponding to the product vectors v; 1v; 2 • • • v;, of degree r in ( 4. 7) are decoded based on their check-sums formed from the received bits in r. Based on these decoded information bits, the received vector r = (ro, ri, · · · , r11 _1) is modified. Let rC 1) = (rgl), r?), ... , r,;~ 1) denote the modified received vector. At the second step of decoding, the bits in the modified received vector ll'(l) are used to form the check-sums for decoding the information bits a; 1; 2 ... ;,_ 1 that correspond to the product vectors v; 1v; 2 • • • v;,_ 1 of degree r - l in ( 4. 7). Then, the decoded information bits at the second step of decoding are used to modify r(ll. The modification results in the next modified received vector rC 2l = (rg2), r?), · · · , r,;~ 1) for the third step of decoding. This step-by-step decoding process continues until the last information bit ao that corresponds to the all-one vector vo in (4. 7) is decoded. This decoding process is called (r + 1)-step majority-logic decoding [2, 11]. Now, we need to know how to form check-sums for decoding at each step. For 1 :::: i 1 < i2 < · · · < ir-l :::: m with O :::: l < r, we form the following index set: 6 { S = c; 1 -1 2 ; 1-1

+ c; 2 -1 2 io-1 + · · · + c;,_ 1-1 2;,-1-1

.. c;;-1 E

{O , l} f or 1 :::: J:::: · r - /}. (4.8)

which is a set of 2r-l nonnegative integers less than 2 in binary form. The exponent set {i 1 - l, i2 - 1, · · · , i,._, - 1} is a subset of {O, 1, · · · , m -1}. Let Ebe the set of integers in {O, 1, · · · , m -1} but not in {i1 - 1, i2 - 1, · · · , i,._, - 1}; that is, 111

E ;;_ {O, 1, · · · , m - 1} \ {i1 - 1, i2 - 1, · · · , i,._, - 1}

(4.9)

= U1, 12, · · · , j111-r+1 L where O:::: ji < .h < · · · < 6

sc =

.

{dj12 11

.

Jm-r+l ::::

m - 1. We form the following set of integers: .

+ dh212 + ... + dJ,,,-,+l21m-r+I

: dj, E {O, l} for 1 :::: t <

m - r + l}. (4.10)

Note that there are

2m-r+l

nonnegative integers in sc, and Sn S'

For l

= {O}.

= r, S = {O}, and sc = {O, 1, ... , 2 -1}. 111

Section liL3

For O ::: l ::: r, suppose vve have decoded information bits are (2 .. ·ir-f+I . We forn1 the vector:

Reed--Muller Codes

iH

modified received (4.11)

l:Si I<

··
::m

where ll'(l-l) is the modified received vector for the Ith For each integer q E sc. we form the set of I::,

B=q+-S

=

{q

+- s : s

E

S}.

the check-sums for decoding the information bits a; 1; 2 .. ;, _ 1 are AUl

= tEB

for q for

sc. Because there are 2m-r+l

q in

sc. we can form 2

-r+l check-sums each information bit a; 1;,. ;,_,. At the first of l =---= 0 and 2m-r check-sums can be formed for decoding each information bit o; 1; 2 •.. ;,. H there are 2m-r-l - 1 or fewer errors in the received vector ll', then more than half of the check-sums assume the .;, and hence the of ;2 .;, is correct, the or more errors. there is no guarantee however, if ll' contains of the check-sums will assume the value of o; 1;, ••• ;,. fo this case, For consider decision may result in an incorrect an error with 2111 -r-l errors such that each error appears in a different checksum. In this case, half of the 2111 - 1 check-sums assume the value 0, and the other half assume the value 1. There is no clear A random choice of the two values may resuh in an incorrect of a; 1; 2 ... ;,. consider another error of 2m-r-l + 1 errors such that each error appears in a different check-sum. fo this case, 2m-r-l +- 1 (majority) of the check-sums assume the opposite value of a; 1 and majority-logic decision based on the check-sums results in incorrect ,..,",""''"' of a; 1 ;c··-i,· Note that the number of check-sums is doubled at each subsequent decoding step. H there are 2m-r-l - 1 or fewer errors in the received vector rr. then majority-logic decision based on check-sums results in correct decoding at each This that the minimum distance of the RM(r. m) code is at least 2. (2111 -r-l - 1) + 1 = 2111 -r - 1. Because the codewords in RM(r, m) have even weights, the minimum distance is at least 2111 -r; however, each product vector of degree r in the matrix GRJvr(r, m) has 2111 - 1 and is a codeword. 111 Therefore, the minimum distance of the code is exactly 2 -,-. The Reed decoding algorithm for RM codes is simply a multistage decoding algorithm in which the decoded information at each stage of decoding is passed down for the next of ~~·v~,,u,,-,,

E

111

Consider the second-order RM code of 16 with m = 4 given in Example 4.2. Suppose we want to construct the check-sums for the information bit 012. Because

112

i1

Chapter 4

Important Linear Block Codes

= 1 and i2 = 2, we obtain the following sets: S = {co+ ci21 : co, ci E {0, l}}

E

Sc

= {O, 1, 2, 3}, = {O, 1, 2, 3} \ {O, 1} = {2, 3}, = {d222 + d32 3 : d2, d3 = {O, 4, 8, 12}.

E {0, 1}}

Then, the index sets for forming the check-sums for a 12 are B1

B2

B3 B4

It follows from (4.13) with l

= 0 + S = {O, 1, 2, 3}, = 4 + S = {4, 5, 6, 7), = 8 + S = {8, 9, 10, 11}, = 12 + S = {12, 13, 14, 15}.

= 0 that the four check-sums for a12 are (0)

A1

= ro +

A~O) =

r4 +rs+ r6 + r7,

(0)

= rs +

(0)

= ri2 +

A3

A4

r1 + r2 + r3,

r9

+

r10

+ r11,

r13 + r14 + r15.

Now, consider the check-sums for a 13 . Because i 1 following sets:

= 1 and

S = {co+ c22 2 : co, c2 E {O, 1}}

E

= = =

{O, 1, 4, 5), (0, 1, 2, 3} \ {O, 2} (1, 3},

Sc= {d12 + d32 3 : d1, d3 E (0, 1}}

=

(0, 2, 8, 10}.

The index sets for constructing the check-sums for a 13 are B1

B2

= 0+S= = 2+S =

(0, 1, 4, 5), (2, 3, 6, 7},

i2

= 3, we obtain the

Section 4.3

=8+S B4

=

{10, 11, 14, 15}.

we obtain the following check-sums for AjO)

= ro + r1 + r4 + rs,

AiO)

= r2 + r3 + r6 + r7,

A~O) =rs+ r9 (0)

A4

iB

(8. 9. 12, 13},

= 10 + S =

From these index sets and

Reed-Muller Codes

013:

+ r12 + r13,

= rio + r11 + r14 + r15.

Using the same we can form the check-sums for information bits and 034. To find the check-sums for a 1 , a2, 03, and a4, we first form the modified received vector ii'(l) based on (4.11):

014, 023, 024,

ii'(l)

= Ir-

Suppose we want to form the check-sums for a3. Because i1 following sets: S E

= =

2 (c22 : c2 E (0, 1)}

{O, 1, 2, 3} \ {2}

=

=

{O, 4},

{O, L 3},

sc ={do+ d 12 + d32 3 : do, d1, d3 =

= 3, we obtain the

E {O, 1}}

{O, 1, 2, 3, 8, 9, 10, 11}.

Then, the index sets for forming the check-sums of a3 are B1

=

(0, 4},

B2 = {1, 5}, B3 = {2, 6}, B4 = {3, 7},

It follows from (4.13) with I

= {8. 12}, B6 = {9, 13}, B7 = {10, 14}, Bs

Bs

=

{11, 15).

= 1 that we obtain the following eight check-sums: A (1) 5

A6(l) A7(l) A (l) 8

Similarly, we can form the check-sums for a1,

+ r(l) 12' (1) + (1) = r9 r13 , (1) + (1) =r10 r14' - r(l) + r(l) 11 15 · -

r(l)

-

8

a2,

and 04.

114

Chapter 4

Important Linear Block Codes

10° -

10-1 10-2

~

10-,

:.0

"

..Cl 0 .... 0..

.... ........0

10-0 - - :uncoded BPSK __... :RM(16, 11, 4)

!

~

-

:RM(32, 6, 16) :RM(32, 16, 8) - ....., - :RM(32, 26, 4) - -><- - :RM(64, 22, 16) - -o- - :RM( 64, 42, 8) ~

10-7 10-8 10-9 0

2

4

6

8

10

12

E1/N0 (in dB) FIGURE 4.2:

Bit-error performances of some RM codes with majority-logic decoding.

Error performances of some RM codes of lengths up to 64 using majority-logic decoding are shown in Figure 4.2. 4.4

OlHIER CONSTRUCTIONS FOR RIEED-MUlllER CODES

Besides the construction method presented in Section 4.3, there are other methods for constructing RM codes. We present three such methods in this section. These methods reveal more structures of RM code that are useful in constructing trellis diagrams and soft-decision decodings. Let A = [aij] be an m x m matrix and B = [bij] be an n x n matrix over GF(2). The Kronecker product of A and B, denoted by A@B, is the mn x mn matrix obtained from A by replacing every entry a;J with the matrix a;J B. Note that for aij = 1, a;J B = B, and for a;J = 0, a;J B is an n x n zero matrix. Let (4.14) be a 2 x 2 matrix over GF(2). The twofold Kronecker product of Gc2.2) is defined as

(4.15)

Section 4.4

Other Constructions for Reed-Muller Codes

The three-fold Kronecker

of G(2.2J is defined as 6

.2>)

ns

=

[~ i J

® [

~

iJ

1 1 1 0 0 1

:i lt

-- [ 01

0

1

1 1 0 0 0

0 0 0 0 0 0 0

0 0 0

1 0

1 0 0 0 0 0

1 1 1 1 0 0 0 0

® [

0

1 0 0 0 1 0 0 0

~

iJ

j]

0

(4.16)

1 1 1 1 0 1 0 1 1 0 0 1 1 1 1 1 0 1 0 1 1 0 0 1

Similarly, we can define the ,n-fold Kronecker product of G(2.2J· Let n = 2111 • We use G(n.nJ to denote the 111-fold Kronecker of G(2.2)· G(l,. 11 ) is a 2m x 2111 matrix over The rows of G(n.n) have weights 2°. 2 1 , 22 , ... , 2"', and the number of rows with weight 2111 - 1 is (';')for O :s l :s m. The generator matrix GRM(r. m) of the rth-order RM code RM(r, m) of length n = 2"' consists of those rows of G(l,, 11 J with to or greater than 2m-r. These rows are the same vectors given by (4.5), except they are a different permutation. IEJ{AMrUE 4.1.ii

Let m

= 4. The fourfold Kronecker product of G(2.2J

G<24.2"J

=

1 1 1 1 0 1 0 1 0 0 1 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0

0 0 0

1 1 1 1 1 1 1 0 1 0 1 0 1 1 0 0 0 0 1 0 0 1 1 1 1 0 0 0 1 0 1 0 0 0 0 1 1 0 0 0 0 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

is

1 1 1 1 1 0 1 0

0 0 0 0 1 0

0 0 0 0

0 0 0 0

0 0 0 0

1 1 1 1 0 1

1 1 1 1 0 0 1 0 0 1 1 1 0 1 0

0 0 0

0 0 1 0 0 0 1 0 0 0

1 1 1 1 1 1 0 1 1 0 0 1 1 1 1 1 0 1 0 1 1 0 0 1 1 1 1 1 0 1 0 1 1 0 0 1

116

Chapter 4

Important Linear Block Codes

The generator matrix GRM(2, 4) of the second-order RM code, RM(2, 4), of length 16 consists of the rows in G (16. 16) with weights 22 , 23 , and 24 . Thus, we obtain 1 0 0

0 0 0

GRM(2, 4) =

0 0 0 0

0

1 1 1 0 0 1 0 0

0 0

0

0

0 0 0

0 0 0 0 0

0 0

1 1 1 1 0 1 1 0 0 1 0 0 0 1 1 0 0 1 0 0 0

1 1 1 1 0 1 1 1 0 1 1 1

0 0 0 0

0 0 0 0

0 0 0

0 0

0

0 0

1

1

0

0 0 0 0

0 0 0 0

0 0

0

1 1 1 1 1 0 1 0 0 1 1 0 0 0 1 0 0 0 0 1 0 0 0 0

0

0

0

0

0

1

1 1

1 0

0 0

0

1 1 1 1 0 1 1 0 0 0 1 1

0 0 0

1

0

1 1 1 1 0 1 1 1 0 1 1 1 1 1 0 1 1 1 1 1

1 1

1

0

1 1

which is exactly the same matrix given in Example 4.2, except for the ordering of the rows. Let u = (uo, u1, ... , u11 _1) and v = (vo, vi, ... , v11 _1) be two n-tuples over GF(2). From llll and v we form the following 2n-tuple: I:,

juju+ vi=

(uo, u1, ... , Un-1, uo

+ vo, u1 + v1,

... , U 17 -1

+v

11

_1).

(4.17)

For i = 1, 2, let C; be a binary (n, k;) linear code with generator matrix G; and minimum distance d;, respectively. Assume that d2 > d1. We form the following linear code of length 2n:

+ C2I = {llllllfill + vi : fill E

C = IC1!C1

C is a binary (2n, k1

(4.18) C1 and V E C2}.

+ k2) linear code with generator matrix (4.19)

where O is a k2 x n zero matrix. The minimum distance dminCC) of C is (4.20) To prove this, let x = IM!llll + vi and y = IIDl'lllll' + v'I be two distinct codewords in C. The Hamming distance between x and y can be expressed in terms of Hamming weights as follows: d(x., y)

= w(fill + fill1) + w(fill + 1lll1 + v + v'),

(4.21)

where w(z) denotes the Hamming weight oh. There are two cases to be considered, v = v' and v -jc v'. If v = v', since x -jc y, we must have lUl -jc u'. In this case, d(x, y) = w(llll

+ llll1 ) + w(llll + llll1 ).

(4.22)

Section 4.4

Other Constructions

Because llll + 1ill1 is a nonzero codeword in C1, w(llll that

+ lill

1

Reed-1\/luller Codes )

2': d1.

'l U

it follows from

(4.23)

If,; cf 'I/, we have

+ v') -- w +

+ --1= w(7 + v').

y) 2':

Since

Y

W(illl

+ w' is a nonzero codeword in

+ ',\f\

>

From

vve have

>

J/ -

that

and

y) 2':

Because 'X and y are two different codewords in C. the minimum distance d 111 ; 11 ( C) must be lower bounded as follows: d111i11CC) ?.':

Let nuo and Yo be two codewords m C1 and Then. w(uno) = d1 and w(vo) -= The vector ITI110ITI11ol is a codeword in C with w(l11.!1olnnol) = 2d1. The vector li[])lvol is also a codeword in C with weight w(li[])lvol) = cfi. From (4.27) we see that (C) must be either 2d1 or we conclude that 0

dmin(C)

=

( 4.28)

The construction of a code from two the lnnltIB + wl-construction [12. 13] which is a long codes from short codes.

LetC1 be the

-~''"~·~c,,·.,u

codes is called for constructing

(8, 4) linear code of minimum distance 4 1

1

1

1

1

1

1

0 0 0 0 1 1 1

Gi=

0 0 1 1 0 0 1 1 0 1 0 1 0

[ 0

Let C2 be the (8, 1) repetition code of minimum distance 8 generated G2

=[

1

1 1

1 1 1 1 1 ].

118

Important Linear Block Codes

Chapter 4

Using lulu+ vi-construction, we obtain a (16, 5) binary linear code of minimum distance 8 with the following generator matrix, G-[ G1 0

~[~

G1 G2 1 0 0 1 0

J

1 0 1 0

1 0 1 1 0 0

1 1 0 0 0

1 1 1 1 1 1 1 1 0 0 0 1 1 0 0 1 0 1 0 1 0 0 0 1 1

1 0 1 0 1

1 0 1 1 1

1 1 0 0 1

1 1 0 1 1

1 1 1 0 1

n

RM codes of length 2111 can be constructed from RM codes of length 2111 - 1 using the lulu+ vi-construction [12]. Form :::_ 2, the rth-order RM code in lulu+ viconstruction is given as follows: RM(r, m) ={lulu+ vi: u

E

RM(r, m - 1) and

v

E

RM(r -1,

(4.29) 111 -

l)}

with generator matrix GRM(r, m)

=[

GRM(r, 111 O

-

1)

GRM(r, 111 -1) GRM(r - 1, 111 - 1)

J

.

(4.30)

The matrix of (4.30) shows that a RM code can be constructed recursively from short RM codes by a sequence of lulu+ vi-constructions. For example, the rth-order RM code RM(r, m) of length 2111 can be constructed from RM codes RM(r, 111 - 2), RM(r l, m - 2), and RM(r - 2, m - 2) of length 2111 - 2. The generator matrix in terms of component codes is given as follows:

G-

1

GRM(r, m - 2) GRM(r, 111 - 2) GRM(r, m - 2) GRM(r, 111. - 2) · 0 GRM(r-l,m-2) 0 GRM(r-l,m-2) 0 0 GRM(r-l,111-2) GRM(r-l,111-2) 0 0 0 GRM(r - 2,111 - 2)

l

.

(4.31)

The recursive structure of RM codes is very useful in analyzing and constructing their trellises [15, 16]. This structure also allows us to devise multistage soft-decision decoding schemes for RM codes that achieve good error performance with reduced decoding complexity. This topic will also be discussed in a later chapter. Consider a Boolean function f(X1, X2, ... , X111 ) of 111 variables, X1, X2, ... , X 111 , that take values ofO or 1 [12, 14]. For each combination of values of X1, X2, ... , and X 111 , the function f takes a truth value of either O or 1. For the 2111 combinations of values of X1, X2, ... , X 111 , the truth values off form a 2111 -tuple over GF(2). For a nonnegative integer l less than 2111 , let (b 11 , b12, .... b1 111 ) be the standard binary representation of l, such that I = b11 + b122 + b1322 + · · · + b1 111 2111 - 1. For a

Section 4.5

The Squaring Construction of Codes

~19)

given Boolean function f(X1, X2, ... , X 111 ), we form the following 2111 -tuple vector): 'i!(f)

=

(4.32)

(vo, v1 .... , v1, .... v2111-1)

where (4.33)

and (b11, b12, ... , b1 111 ) is the standard binary representation of the index integer/. We say that the Boolean function f (X1, ... , X111 ) represents the vector 'ii. We use the notation w(f) for the vector represented by f (X1, X2, ... , ). For 1 :s i :s m, consider the Boolean function ... , Xm)

=

(4.34)

H is easy to see that this Boolean function represents the vector 'ii; defined For 1 :s i, j :s m, the function

represents the logic h(X1, X2, ... , Xm)

=

of •1; and 'ilj, represented by g(X1,

... ,

X111 )

=

and

X 1, respectively. For 1 :S i1 < i2 < · · · < i,. :Sm, the Boolean

function (4.36)

represents the logic product of 'ii; 1 , 'i/; 2 , ••• , and 'ii;,. Therefore, the generator vectors of the rth-order RM code of length n = 2111 (the rows in GRM(r, m)) are represented the Boolean functions in the following set: (4.37)

... up to all products of r variables}. Let P (r, m) denote the set of all Boolean functions (or polynomials) of degree r or less with m variables. Then, RM(r, m) is given the following set of vectors [12]: RM(r, m)

= {'i!(.f):

f

E

P(r, m)}.

(4.38)

The Boolean representation is very useful in studying the weight distribution of RM codes [18, 20]. itn.• 5

lllllE '.SQQJ1ll:Jt!WIJG (O~l§l!RQJ(l!ON Of (OIQ)IE§

Consider a binary (11, k) linear code C with generator matrix G. For O :s ki :s k., let C1 be an (11, lq) linear subcode of C that is spanned by k1 rows of G. Partition C into 2k-tq cosets of C 1· This partition of C with respect to C 1 is denoted by C /Cl· As shown in Section 3.5, each coset of C1 is of the following form: (4.39)

with 1 :s l :s 2k-k1, where for 'ii/ I= @, v1 is in C but not in C1 , and for v1 = «Jl, the cos et «Jl EB C1 is just the subcode C 1 itself. The codeword 'ilt is called the leader ( or representative) of the coset 'ii/ EB Cl· We also showed in Section 3.5 that any codeword

120

Chapter 4

Important Linear Block Codes

in a coset can be used as the coset representative without changing the composition of the coset. The all-zero codeword O is always used as the representative for C1. The set of representatives for the cosets in the partition C / C1 is denoted by [C / C1], which is called the cos et representative space for the partition C / C1 · Because all the cosets in C / C1 are disjoint, C1 and [C / C1] have only the all-zero codeword O in common; that is, C1 n [C / C1] = {O}. Then, we can express C as the sum of the coset representatives in [C/C1] and the codewords in C1 as follows: (4.40)

The preceding sum is called the direct-sum of [C / C1] and C1. Let G1 be the subset of /q rows of the generator matrix G that generates the linear subcode C1. Then, the 2k-k1 codewords generated by the k - k1 rows in the set G \ G 1 can be used as the representatives for the cosets in C / C1 · These 2k-ki codewords form an (n, k - k1) linear subcode of C. Let C2 be an (n, k2) linear subcode of C1 with O ::c k2 ::c k1. We can further partition each coset v1 El) C 1 in the partition C / C1 based on C2 into 2'q -k 2 cosets of C2; each coset consists of the following codewords in C: (4.41)

with 1 ::: I ::: 2k-k1 and 1 ::c q ::c 2k1-k 2 , where for wq f- 0, wq is a codeword in C1 but not in C 2. We denote this partition C / Ci/ C2. This partition consists of 2k-k2 cosets of C2. Now, we can express C as the following direct-sum: (4.42)

Let C1 . C2, ... , C111 be a sequence of linear subcodes of C with dimensions k1, k2, ... , k 111 , respectively, such that (4.43)

and k ~ k1 ~ k2 ~ · · · ~ k 111 ~ 0.

(4.44)

Then, we can form a chain of partitions, (4.45)

and can express C as the following direct-sum: ( 4.46)

We now present another method for constructing long codes from a sequence of subcodes of a given short code. This method is known as the squaring construction [15]. Let Co be a binary (n, ko) linear block code with minimum Hamming distance do. Let C1, C2 ..... C111 be a sequence of subcodes of Co such that (4.47)

Section 4.5

The Squaring Construction of Codes

121

For O :S i :S m, let G;, k;, and d; be the generator matrix, the dimension, and the minimum distance of the subcode VVe form a chain of as follows: (4.48)

Co/C1, Co/Ci/C2, ... , Co/Ci/··· /Cm. For O :S i < m, let G,;;+1 denote the generator matrix for the coset space [C; /C;+1J. The rank of G;;;+1 is Rank(G;;;+1)

= Rank(G;) -

Without loss of generality, we assume that Go :) G 1 :)

:)

...

:)

. Then, for

0 :Si < m,

G;;;+1

=

(4.50)

\G;+l·

One-level squaring construction is based on C1 and the 1ll = (ao. 01, .... a 11 _1) and \bJ = (bo, b1 ..... b11 _1) be two binary (ru, ibJ) denote the 211-tuple (ao, a1, ... , a11 _1, bo, b1 ..... bn-1). We form the set of 211-tuples: and x

E

[Co/C1]}.

(4.51)

Then, ICo/ C11 2 is a (2n, ko + k1) linear block code with minimum Hamming distance . 2 D1 =L. mm{ do. di).

(4.52)

The generator matrix for ICo/ C11 2 is given by

Let M1 and M2 be two matrices with the same number of columns. The matrix

is called the direct-sum of M1 and M2, denoted by M = M1 EB Then. we can express the generator matrix for the one-level squaring construction code IC0/C11 2 in the following form: (4.54) G = h@ G1 EB (1, 1)@ Go;1,

where@ denotes the Kronecker product, EB the direct-sum, h the identity matrix of dimension 2, (4.55)

and (1, 1)@ Go;1

= [Go;1 Go;1].

(4.56)

122

Chapter 4

Important Linear Block Codes

Now, we extend the one-level squaring construction to a two-level squaring construction. First, we form two codes, U ~ IC0/C11 2 and V ~ IC1/C21 2, using onelevel squaring construction. It is easy to see that Vis a subcode of V. The two-level squaring construction based on the partitions Co/C1, Ci/C2, and Co/Ci/C2 gives the following code: IC0/Ci/C21 4 ~{(a+ x, b + x): a, b EV and x E [U/V]} ={(a+ x, b + x): a, b E ICi/C21 2

(4.57)

and x E [IC0/C11 2 I ICi/C21 2]}, which is simply the code obtained by one-level squaring construction based on V and V / V. This code is a (4n, ko + 2k1 + k2) linear block code with minimum Hamming distance D2 ~ min{4do, 2d1, d2}. (4.58) Let Gu, Gv, and Gu;v denote the generator matrices for U, V, and [U/V], respectively. Then, the generator matrix for IV/ V 12 is Gv

G=

[

0 Gu;v

0 ] . Gv Gu;v

(4.59)

We put Gv and Gu in the form of (4.53) and note that Gu;v = Gu\Gv. Then, we can put the generator matrix G of ICo/ Cii C2 l4 = IV/ V 12 given by (4.59) in the following form: 0 0 0 G2 0 0 0 G2 0 0 0 G2 0 0 0 G2 (4.60) G= Go;1 Go;1 Go;1 Go;1 G1;2 G1;2 G1;2 G1;2 0 0 G1;2 G1;2 0 0 G1;2 G1;2 We can express this matrix in the following compact form: G

~ l4 0 G2 Ell (1111) 0 Go;1 Ell [ ~

1 1 1 0 1 1 ] 0G112 1 0 1

Note that (1111) and

1 [ 1 1 1 O O 1 1 0 1 0 1

(4.61)

]

are the generator matrices of the zeroth- and first-order RM codes of length 4. Higher-level squaring construction can be carried out recursively in a similar manner. Form :::: 2, let

Section 45

The

Construction of Codes

U3l

and Vm

/:.

=

denote the two codes obtained obtained m-level

iCo/

I··· IC,, I2'"

The

I

I

2111-I

construction. The code

(m -

I:.

-

= { (ru + ~c, lbJ

matrix is

... /Cm!

+ lY:)

and 1f

: ru, !ill E 0

E

[Um/

(4.62)

[15, 16]:

G=

(4.63)

® G,.;,-+1, O:::r<m

denotes the matrix of dimension (r, and is the 2111. matrix of the rth--order RM RM codes are of the construction. Long RM codes can be constructed from short RM codes the construction [15, 16]. From the construction of RM codes O
=[

. m--1)

GRM(r, m - 1)

"J .

-1, m -1)

0

( 4.65)

We define I:.

,m -1

-1.m-1)=

-- l, m -- 1).

(4.66)

Note that t,RM(r/r - L m -- 1) consists of those rows in in GRM(r - 1, m - 1) and it spans the coset ,PrwP•,P11 RM(r -- 1, m - 1)]. Now, we can GRM(r. m - 1) in the (4.67) 1-'"''-'""'F,

GRIV!(r, m - 1) in

with the GRM (r, m) in the

operations, we

GRM(r -

GRM(r, m)

=

L

,c,v,n·,·a,cc,

m - 1)

0 [ DRM(r/r -1, m --1)

of

and

-°i.

GRM(r m - 1) DRM(r/r -1. m - 1)

row

J.

(4.68)

This is exactly the generator matrix form of one-level squaring construction. Therefore, the rth-order RM code of length 2111 can be constructed from the rth-order and (r - l)th order RM codes of length 2111 -1; that RM(r, m)

=

IRM(r. m -- 1)/RM(r - 1, m - 1)1 2

.

(4.69)

124

Chapter 4

Important Linear Block Codes

Because RM(r, m - 1) = IRM(r, m - 2)/RM(r - 1,111 - 2)1 2

and RM(r -1,

111

-1) = IRM(r -1, 111

-

2)/RM(r -

then we can construct RM(r, 111) from RM(r, m - 2), RM(r 2, m - 2) using two-level squaring construction; that is, RM(r, 111)

= IRM(r, 111 - 2)/RM(r -1, 111 - 2)/RM(r -

2,111 -

1, 111

2,111 -

-

2)1 2

,

2), and RM(r -

2)1

22

.

(4.70)

Repeating the preceding process, we find that for 1 :::: µ :::: r, we can express the RM(r, m) code as a µ-level squaring construction code as follows: RM(r, m)

= IRM(r, m - µ)/RM(r

-1, m - µ)/ · · · /RM(r - µ, m - µ)I 2'' . (4.71)

A problem related to the construction of codes from component codes is code decomposition. A code is said to be decomposable if it can be expressed in terms of component codes. A code is said to be µ-level decomposable if it can be expressed as a µ-level squaring construction code from a sequence of subcodes of a given code, as shown in (4.62). From (4.71) we see that a RM code is µ-level decomposable. A µ-level decomposable code can be decoded in multiple stages: component codes are decoded sequentially one at a time, and decoded information is passed from one stage to the next stage. This multistage decoding provides a good trade-off between error performance and decoding complexity, especially for long codes. RM codes also can be constructed from Euclidean geometry, which is discussed in Chapter 8. This construction reveals more algebraic and geometric structures of these codes, especially the structures and the number of minimum-weight codewords. There is a one-to-one correspondence between a minimum-weight codeword of the rth-order RM code, RM(r, m), and an (m - r)-dimensional flat in m-dimensional Euclidean geometry, EG(m, 2), over GF(2). This correspondence gives the number of minimum-weight codewords of the RM(r, m) code [12, 17]

A211,-,

=2

n

r m-r-1 (

. .

2m-i _ l ) 2m-r-1

-1

(4.72)

i=O

In fact, these minimum-weight codewords span (or generate) the code; that is the linear combinations of these minimum-weight codewords produce all the codewords of the RM(r, m) code. The weight distribution of several subclasses of RM codes and all RM codes of lengths up to 512 have been enumerated [12, 18-21 ]. The first-order RM code, RM(l, m), has only three weights, 1, 2111 - 1, and 2111 • The number of codewords of these weights are Ao= A2111 = 1,

(4.73)

The

Section 4.6

12) Golay Code

125

The second-order RM code, RM(2. m) has the following weight distribution: Ao= A2,,, = 1,

_ 21(1+]) n:'~m-21+1 (i -1) A 211,-1±2",-1-, n'.= c22 i - l) 1

m

for 1 -< 1 -< L-J 2

L J 2 2 "iU+l) L I//

' 1 = 2(111-+m+2)/2

_ 2 _

1nmi=m-21+1 (i -

1-1

2

I

ni=l c2'

(4.74) 1)

.

- 1)

RM code, RM(m - r - 1, m), is the dual code of Because the (m - r the rth-order RM RM(r, m), the weight distributions of the RM(m - 2. m) and RM(m - 3, m) codes can be derived from (4.73), (4.74), and the MacWiHiams of

A;

0

8

12

16

20

24

1

620

13888

36518

13888

620

1

32

1

This code is a self-dual code. RM codes form a remarkable class of linear block codes. Their rich structural properties make them very easy to decode by either hard- or soft-decision decoding. Various soft-decision decoding algorithms for these codes have been devised, and some will be discussed in later chapters. Other classes of codes are more powerful than RM codes-for the same minimum distance, these codes have higher rates: however, the low decoding complexity of RM codes makes them very attractive in practical applications. fo fact, in terms of both error performance and decoding complexity, RM codes often outperform their corresponding more powerful codes. The (m - 2)th-order RM code of length 2111 is actually the distance-4 extended Hamming code obtained by adding an overall parity bit to the Hamming code of length 2111 - 1. 4U6i

THIE {2.4, 12.)
Besides the Hamming codes, the only other nontrivial binary perfect code is the (23. 12) code constructed by M. J. E. Golay in 1949 [3]. This code has a minimum distance of 7 and is capable of correcting any combination of three or fewer random errors in a block of 23 digits. The code has abundant and beautiful algebraic structure, and it has become a subject of study by many coding theorists and mathematicians; many research papers have been published on its structure and decoding. The Golay code is the most extensively studied single code. fo addition to having beautiful structure. this code has been used in many real communication systems for error control. This code in its cyclic form will be studied in Chapter 5. The (23. 12) Golay code can be extended by adding an overall parity-check bit to each codeword. This extension results in a (24, 12) code with a minimum distance

126

Chapter 4

Important Linear Block Codes

of 8. This code is capable of correcting all error patterns of three or fewer errors and detecting all error patterns of four errors. It is not a perfect code anymore; however, it has many interesting structural properties and has been widely used for error control in many communication systems, especially in the U.S. space program. It served as the primary Voyager error-control system, providing clear color pictures of Jupiter and Saturn between 1979 and 1981. In this section we study the (24, 12) Golay code and its decoding. A generator matrix in systematic form for this code is as follows [12, 23, 24]: G

=[

lP'

112 ] ,

where 112 is the identity matrix of dimension 12 and

P=

100011101101 000111011011 001110110101 011101101001 111011010001 110110100011 101101000111 011010001111 110100011101 101000111011 010001110111 111111111110

(4.75)

The P matrix has the following properties: (1) it is symmetrical with respect to its diagonal; (2) the ith column is the transpose of the ith row; (3) P. pT = 112, where pT is the transpose of JP; and ( 4) the submatrix obtained by deleting the last row and last column is formed by cyclically shifting the first row to the left 11 times (or cyclically shifting the first column upward 11 times). It follows from the second property that pT = F'. Consequently, the parity-check matrix in systematic form for the (24, 12) extended Golay code is given by H

=[

112

pT ]

= [ 112

lP' ] .

( 4.76)

It can be proved that the code is self-dual [see Problem 4.18].

A simple decoding algorithm for the (24, 12) Golay code can be devised using the properties of the JP' matrix [23]. For O :::: i :::: 11, let p; denote the ith row of P and IDI(i) the 12-tuple in which only the ith component is nonzero. For example, IDI(S) = (0 0 0 0 0 10000 0 0). We readily see that p; = u
(4.77)

Let e = (x, y) be an error vector, where x and y are binary 12-tuples. Suppose a codeword vis transmitted, and a correctable error pattern e = (x, y) occurs. Then,

Section 4.6

The

Code

12)

U:7

+ ie. The syndrome of n· is §=Ir· IHIT = (v + e) · IHIT = e. IHIT.

the received vector is Ir = v

H follows from (4.76) and

]lllT

=

]lll

§

that = (lI, y) . [ Il~2

J

= ):_( · ll12 + y · ]lll = '!I+Y· W'. Using the property

]lll · ]lllT

(4.78)

= ll12, we can express yin terms of x,

§,

and

]lll

as follows: (4.79)

In the following, we first show that a correctable error pattern for the (24, 12) Golay code can be expressed in terms of W', JPl;, unU l, and§. We then present a~~,,~~'""~ algorithm for the code. For any correctable error pattern with w(e) :S 3, we have the foHowing four possibilities: (1) (2) (3) (4)

w(x) :S w(x) :S w(x) :S w(x) =

3 and 2 and 1 and 0 and

w(y) w(y) w(y) w(y)

= = = =

0, 1, 2, 3.

Theses four possibilities define four different types of correctable error patterns. For O :s j :s 3, let eUl = (;-r, y), for which w(y) = j, and w(I,) :s 3 - j. Suppose e = e(O). It follows from (4.78) that§ = x and w(5) = w(x) :s 3. In this case, e

=

(§, I[])),

where I[]) is the all-zero 12-tuple. Suppose e = eC 1l and y = unUl, Then, it follows from (4.78) that § = 1l + 11l) ·IP'= lX +JP);. Hence, x = § + JP);, and w (5 + ]lll;) = w (i:,)

:s 2. fo this case, e = (5 + ]lll;, llll(il).

Suppose e =

(1:/ 2l

or e< 3l, and w(x) = 0. H follows from ( 4.79) that

y=

§.

and w(§ · IP') = w(y) = 2 or 3. For this case, we can express e as follows: e =

(I[]),

§ . IP').

Now, suppose e = e< 2l, and w(x) = 1. H the nonzero component of xis at the ith position, then x = llll(i). H follows from (4.79) that

+ unUl) . JP' · JP' + llU(i) · JP'

y = (5 =

§

=§.JP+ ]Pl;

128

Chapter 4

and w(s · P

Important linear Block Codes

+ p;) = w(y) = 2. Consequently, we can express e as follows: e=(l.Il
A decoding algorithm can be devised for the (24, 12) Golay code based on the preceding analysis and expressions of correctable error patterns. The decoding consists of the following steps: Step 1. Compute the syndrome s of the received sequence r. Ste)lll 2, If w(§) :s 3, then set e = (§, 0) and go to step 8. Ste)lll 3, If w(§ +)Ill;) :s 2 for some row p; in IP', then set e = (§ + p;, l.Il(il) and go

to step 8. Step 41, Compute s . JP. Step§, If w(s · P) = 2 or 3, then set e = (0, § · P) and go to step 8. Step «ii, If w(§ · P + p;) = 2 for some row ]Ill; in P, then set e = (llll
+ p;)

and go to step 8. Step 7. If the syndrome does not correspond to a correctable error pattern, stop the decoding process, or request a retransmission. (This represents a decoding failure.) Step 8. Set the decoded codeword v* = r + e and stop.

EXAMPLE 4.1 Suppose the (24, 12) Golay code is used for error control. Let r = (100 0 0 0 1101 0 0 11000 0 0 0 0 0 0 1) be the received sequence. To decoder, we first compute the syndrome § of r: § = r · HT = (111011111100). Because w(s) > 3, we go to decoding step 3. We find that s + Jlllll = (1 1 1 0 1 1 1 1 1 1 0 0)

+ (1 1 1 1 1 11 1 1 1 1 0)

= (0 0 0 1000 0 0 0 1 0), and w(s

+ Jlll11) = 2. So we set e = (s + Pll,

llll(ll))

= (0 0 0 100 0 0 0 0 100 0 0 0 0 0 0 0 0 0 0 1) and decode r into v* = r + e

= (100100110110110000 0 0 0 0 0 0).

4.1

PRODUCT CODIES

Besides the !an!1!Il + vi and squaring constructions of codes, another technique for constructing long, powerful codes from short component codes is the product coding technique.

Section

1.u

Product Codes

12:~

be an and let an (11111.2, k1k2) linear code can be formed such that each codeword is a array of 111 columns and n2 rows in which every row is a codeword in and every column is a codeword in as shown in 4.3. This two-dimensional code is called the direct the of , and [25]. The lcik2 in the upper corner of the array are information The in the upper left comer of this array are rules for C1 on rows, and the in the lower fron1 the corner rules for for C1 on rows? H turns out that either way check (see Problem and it is to have all row codewords in C1 and all colunm codewords in C2 The code C1 x C2 is encoded in two A message of lqk2 informa-tion is first as shown in the upper corner of 4.3. At the first step of encoding, each rovv of the information array is encoded into a codeword in This row results in an array of k2 rows and n1 as shown in the upper part of Figure 4.3. At the second of each of the 111 columns of the array formed at the first encoding is encoded into a codeword in C2. This results in a code array of 112 rovvs and n 1 as shown in Figure 4.30 This code array also can be formed by first performing the column and then the row encoding. Transmission can be carried out either column column or row row. H code C1 has minimum weight d1 and code C2 has minimum weight d2, the mm1mum of the code is d1d2. A codeword in the product code is formed by (1) a C1 and a codeword in C2 and forming an array in which all columns corresponding to zeros in the codeword from C 1 are zeros, and an columns corresponding to ones in the codeword from C1 are the minimum-weight codeword chosen from C2.

-,\-,-

Checks 011

rows

Information digits

k,

lh

_1_

Checks on checks

Checks on columns __v_

FIGURE 4.3: Code array for the product code C1 >< C20

130

Chapter 4

Important Linear Block Codes

It is not easy to characterize the correctable error patterns for the product code; this depends on how the correction is done. One method involves using a two-step decoding. The decoding is first performed on rows and then on columns. In this case a pattern will be correctable if and only if the uncorrectable patterns on rows after row correction leave correctable patterns on the columns. It generally improves the correction to decode by rows, then columns, then columns and rows again. This method, of course, increases the decoding delay. This type of decoding is called iterative decoding [25]. The product code is capable of correcting any combination of [(d1 d2 - 1) /2] errors, but the method described will not achieve this. For example, consider the product code of two Hamming single-error-correcting codes. The minimum distance of each is 3, so the minimum distance of the product is 9. A pattern of four errors at the corners of a rectangle gives two errors in each of the two rows and two columns and is therefore not correctable by simple correction on rows and columns. Nevertheless, simple correction on rows and columns, although nonoptimum, can be very effective. The complexity of the two-step decoding is roughly the sum of the complexities of the two component code decodings. EXAMPLE 4.8

Consider the product of the (5, 4) SPC code C1 with itself. The product code C1 x C1 is a (25, 16) linear block code C with a minimum distance of 4. Suppose the message u = (l O 1 1 0 0 0 1 0 1 0 1 1 1 0 1) is to be encoded. This message is read into a storage buffer and arranged into a 4 x 4 information array, as shown in Figure 4.4. The first four information symbols form the first row of the information array, the second four information symbols form the second row, and so on. At the first step of encoding, a single (even) parity-check symbol is added to each row of the information array. This results in a 4 x 5 array, as shown in Figure 4.4. In the second step of encoding a single (even) parity-check symbol is added to each of the five columns of the array, as shown in Figure 4.4. Suppose this code array is transmitted column by column. At the received end, the received sequence is rearranged into a 5 x 5 code array column by column, called the received array. Suppose a single error occurs at the intersection of a row and a column. The erroneous row and column containing this single error are indicated by parity-check failures, then the error is corrected by complementing the received symbol (i.e., 0 to 1, and 1 to 0) at the intersection. All the single-error patterns can be corrected in this manner. Checking the row and column parity failures cannot correct any double-error pattern, but it can detect all the double-error patterns. When a double-error pattern occurs, there 0

0 0

0

0

0 0 0

0

0

0

FIGURE 4.4: A code array of the product of the (5, 4) SPC code with itself.

Section 4.8

Interleaved Codes

Bi

1 1~----:==========- -=--=--=--k-1===============::1 11

-

Checks on rows

Information digits

n,

Checks on columns

are three distributions of the two errors: (1) they are in the same row; (2) they are in the same or (3) are in different rows and different columns. In the first case, there are two column parity failures but no row parity failure. Hence, errors are detected but they cannot be located. In the second case, there are two row parity failures but no column parity failure. Again, errors are detected but cannot be located. In the third case, there are two row parity failures and two column failures, so there are four intersecting locations. The two errors are situated at two opposite diagonal positions, but we cannot determine the positions. In constructing a two-dimensional product code, if we do not form the we obtain an incomplete code array, as shown in Figure 4.5. This incomplete product of two codes results in a (k1112 + k2n1 - lqk2, /qk2) linear block code with a minimum distance of d1 + d2 - 1 (see Problem 4.22). The code has a higher rate but smaller minimum distance than the complete product code. (n 1 - k1) x (112 - k2) checks on checks in the lower left corner of Figure

Given an (11, k) linear block code C, it is possible to construct a (An, Ak) linear block code (i.e., a code A times as long with A times as many information by interleaving, that is, simply by arranging A codewords in C into A rows of a rectangular array and then transmitting the array column by column, as shown in Figure 4.6. The resulting code, denoted by cJc, is called an interleaved code. The parameter A is referred to as the interleaving depth ( or degree). H the minimum distance of the base code C is dmin, the minimum distance of the interleaved code is also dmin• The obvious way to implement an interleaved code is to set up the code array and operate on rows in encoding and decoding. In this way, a pattern of errors can be corrected for the whole array if and only if the pattern of errors in each row

132

Chapter 4

Important Linear Block Codes Transmission

,....

,-..

,,....

,,...

,-..

;\

••

1

...

e@ e

¢10 0

•••

..• ..

• •

0

0

t

r 000

000

T.

,,_

..

•• •

...

... '-'

'--'

'-'

.......

'-'

• • 0

~

----n - k----a>---------k-----------l><

FIGURE 4.6:

Transmission of an interleaved code.

is a correctable pattern for the original code C. The interleaving technique is very effective for deriving long, powerful codes for correcting errors that cluster to form bursts. This topic will be discussed in a later chapter. Interleaving a single code can easily be generalized to interleaving several different codes of the same length. For 1 s i s A, let C; be an (n, k;) linear block code. Take A codewords, one from each code, and arrange them as A rows of a rectangular array as follows:

[

"1.0

v2,0,

VA,0,

v1,1, v2,1,

"1"-1

vA,l,

VA,n-1

v2,n-l

J (4.80)

Then, transmit this array column by column. This interleaving of A codes results in an (An, ki + k2 + · ·, + kA) linear block code, denoted by CA = C1 * C2 *···*CA. Each column of the array given in (4.80) is a binary A-tuple. If each column of ( 4.80) is regarded as an element in Galois field GF(2A), then CA may be regarded as a linear block code with symbols from GF(2A). The interleaving technique presented here is called block interleaving. Other types of interleaving will be discussed in later chapters and can be found in [26].

PROBLEMS 4.1 Form a parity-check matrix for the (15, 11) Hamming code. Devise a decoder for the code. 41,2 Show that Hamming codes achieve the Hamming bound (see Problem 3,15). 4.3 Show that the probability of an undetected error for Hamming codes of length 2111 - 1 on a BSC with transition probability p satisfies the upper bound 2- 111 for p s 1/2, (Hint: Use the inequality (1 - 2p) s (1 - p) 2 .)

Problems

'i 33

4lA!, Compute the probabiiity of an undetected error for the 11) code on a BSC with transition probability p = 10- 2 . ,ts Devise a decoder for the (22, 16) SEC-OED code whose parity-check matrix is given in Figure 4.l(a). 4Ll1i Form the generator matrix of the first-order RM code RM(l, 3) of length 8. What is the minimum distance of the code? Determine its parity-check sums and devise a majority-logic decoder for the code. Decode the received vector rr = (0 10 0 0 1 0 4L7 Form the generator matrix of the first-order RM code RM(l, 4) of length 16. What is the minimum distance of the code? Determine its parity-check sums and devise a majority-logic decoder for the code. Decode the received vector ir = (0 0 1100 100 11 100 1 1). 4Li Find the parity-check sums for the second-order RM code RM(2, 5) of length 32. What is the minimum distance of the code? Form the parity-check sums for the code. Describe the decoding steps. 4lollJl Prove that the (m - r - l)th-order RM RM(m - r - 1, m ), is the dual code of the rth-order RM code, RM(r, m ). 4loUD Show that the RM(l, 3) and RM(2, 5) codes are self-dual. 4lJL]. Find a parity-check matrix for the RM(l, 4) code. ,:jLJL2 Construct the RM(2, 5) code of length 32 from RM codes of length 8 using llllllllll + vi-construction. 4loB Using the llllllun + -11-construction, decompose the RM(2, 5) code into component codes that are either repetition codes of dimension 1 or even parity-check codes of minimum distance 2. 4U_4l Determine the Boolean polynomials that give the codewords of the RM(l, 3) code. 4loli5 Use Boolean representation to show that the RM(r, m) code can be constructed from RM(r, m - 1) and RM(r - 1, m - 1) codes. 4lolil1i Construct the RM(2, 4) code from the RM(2, 3) and RM(l, 3) codes using one-level squaring construction. Find its generator matrix in the form of (4.53) or ( 4.68). 4loli7 Using two-level squaring construction, express the generator matrix of the RM(2, 4) code in the forms of (4.60) and (4.61). 4!,J_0\ Prove that the (24, 12) Golay code is self-dual. (Hint: Show that CG· CG 7 = 0.) 4lo]_ll]) Design an encoding circuit for the (24, 12) Golay code. 4!,J;@ Suppose that the (24, 12) Golay code is used for error correction. Decode the following received sequences: filo 11' = (1 0 11 0 11 1 0 0 1 Q O Q Q O 1 1 0 0 0 Q 1 1), lbio 11' = (0 0 11111100 100 Q O O OO O O O O O 1). 4lJ:,]_ Show that the digits for checking the parity-check digits of a product code array shown in Figure 4.3 are the same no matter whether they are formed by using the parity-check rules for C2 on columns or the parity-check rules for C 1 on rows. 4LZZ Prove that the minimum distance of the incomplete product of an (11 1 , ki. d1 ) linear code and an (112, k2, d2) linear code is d1 + d2 - 1. ,iloZ3 The incomplete product of the (111,111 - 1, 2) and the (112, n2 - 1, 2) even paritycheck codes has a minimum distance of 3. Devise a decoding algorithm for correcting a single error in the information part of a code array.

134

Chapter 4

Important Linear Block Codes

BIBUOGRAPHY

1. R. W. Hamming, "Error Detecting and Error Correcting Codes," Bell Syst. Tech. I., 29: 147-60, April 1950. 2. W. W. Peterson and E. J. Weldon, Jr., Error Correcting Codes, 2d ed., MIT Press, Cambridge, 1972. 3. M. J.E. Golay, "Notes on Digital Coding," Proc. IEEE, 37: 657, June 1949. 4. A. Tietavainen, "On the Nonexistence of Perfect Codes over Finite Fields," SIAM I. Appl. Math, 24: 88-96, 1973. 5. V. Pless, Introduction to the Theory of Error Correcting Codes, 2d ed., John Wiley, New York, 1989. 6. S. K. Leung-Yan-Cheong and M. E. Hellman, "Concerning a Bound on Undetected Error Probability," IEEE Trans. Inform. Theory, IT-22: 235-37, March 1976. 7. T. Klove, Error Detecting Codes, Kluwer Academic, Boston, Mass., 1995. 8. M. Y. Hsiao, "A Class of Optimal Minimum Odd-Weight-Column SEC-DED Codes," IBM I. Res. Dev., 14, July 1970. 9. D. E. Muller, "Applications of Boolean Algebra to Switching Circuits Design and to Error Detection," IRE Trans., EC-3: 6-12, September 1954. 10. I. S. Reed, "A Class of Multiple-Error-Correcting Codes and the Decoding Scheme," IRE Trans., IT-4: 38-49, September 1954. 11. J. L. Massey, Threshold Decoding, MIT Press, Cambridge, 1963. 12. F. J. MacWilliams and N. J. A. Sloane, The Theory of Error-Correcting Codes, North Holland, Amsterdam, 1977. 13. M. Plotkin, "Binary Codes with Specific Minimum Distance," IEEE Trans. Inform. Theory, IT-6: 445-50, November 1960. 14. V. P. Nelson, H. T. Nagle, B. D. Carroll, and J. D. Irwin, Digital Logic Circuit Analysis & Design, Prentice Hall, Englewood Cliffs, N.J., 1995. 15. G. D. Forney Jr. "Coset Codes II: Binary Lattices and Related Codes," IEEE Trans. Inform. Theory, IT-34: 1152-1187, September 1988. 16. S. Lin, T. Kasami, T. Fujiwara, and M. Fossorier, Trellises and Trellis-Based Decoding Algorithms for Linear Block Codes, Kluwer Academic, Boston, Mass., 1998. 17. S. Lin, "Some Codes Which Are Invariant under a Transitive Permutation Group and Their Connection with Balanced Incomplete Block Designs," chap. 24 in Combinatorial Mathematics and Its Applications, ed. R. C. Bose and T. A. Dowling, University of North Carolina Press, Chapel Hill, 1969.

Bibliography

BS

mt T. Kasami, "The Weight Enumerators for Several Classes of Subcodes of the Second-Order Binary Reed-Muller Codes," Inform. and Control, 18: 369-94, 1971. lil9lo M. Sugino, Y. Ienaga, N. Tokura, and T. Kasami, "Weight Distribution of (128, Reed-Muller Code," IEEE Trans. Inform. Theory, IT-17: 627-28, September 1971. 21lllo T. Kasami, N. Tokura, and S. Azumi, "On the Weight Enumeration of Weight Less than 2.5d of Reed-Muller Codes," Inform. and Control, 30: 380-95, 1976.

2lio T. Sugita, T. Kasami, and T. Fujiwara, ''The Weight Distributions of the ThirdOrder Reed-Muller Code of Length 512," IEEE Trans. Inform. Theory, ff-42: 1622-25, September 1996. 220 R. G. Gallager, Low-Density Parity-Check Codes, MIT Press, Cambridge, 1960.

22t S. A. Vanstone and P. C. van Oorschot, An Introduction to Error Correcting Codes with Applications, Kluwer Academic. Boston, Mass., 1989. 24L S. B. Wicker, Error Control Systems for Digital Communication and Storage, Prentice Hall, Englewood Cliffs, N.J., 1995.

2:i P. Elias. "Error-Free Coding," IRE Trans. Inform. Theory, PGIT-4: 29-37, September 1954. 2
CHAPTER

5

Cyclic Codes Cyclic codes form an important subclass of linear block codes. These codes are attractive for two reasons: first, encoding and syndrome computation can be implemented easily by employing shift registers with feedback connections (known as linear sequential circuits); and second, because they have considerable inherent algebraic structure, it is possible to devise various practical methods for decoding them. Cyclic codes are widely used in communication systems for error control. They are particularly efficient for error detection. Cyclic codes were first studied by Eugene Prange in 1957 [1 ]. Since then, progress in the study of cyclic codes for both random-error correction and bursterror correction has been spurred by many algebraic coding theorists. Many classes of cyclic codes have been constructed over the years, including BCH codes, Reed-Solomon codes, Euclidean geometry codes, projective geometry codes, quadratic residue codes, and Fire codes, which will be discussed in later chapters. Excellent expositions of cyclic codes can be found in [2-5]. References [6-9] also provide good coverage of cyclic codes. 5.1

DESCmPliON Of CVCUC CODES If we cyclically shift the components of an n-tuple v

=

(vo, v1, · · · , v11 _1) one place

to the right, we obtain another n-tuple, 'V(l)

= (vn-1, VO,··· , Vn-2),

which is called a cyclic shift of v. If the components of v are cyclically shifted i places to the right, the resultant n-tuple is V(i)

= (V11-i, V11 -i+l, · · · , Vn-1, VO, V1, · · · , V11-i-1).

Clearly. cyclically shifting v i places to the right is equivalent to cyclically shifting v n - i places to the left.

5.JL An (n, k) linear code C is called a cyclic code if every cyclic shift of a codeword in C is also a codeword in C.

DEFKNHWN

The (7, 4) linear code given in Table 5.1 is a cyclic code. To develop the algebraic properties of a cyclic code, we treat the components of a codeword v = (vo, v1, · · · , v11 _1) as the coefficients of a polynomial as follows: v(X)

= vo + v1X + v2X 2 + ... + V11-1x 11 - 1 .

Thus, each codeword corresponds to a polynomial of degree n - l or less. If v11 _1 f= 0, the degree of v(X) is n - 1; if v11 _1 = 0, the degree of v(X) is less than n - 1. The correspondence between the codeword v and the polynomial v(X) is one-to-one. We 136

Section 5.1

of Cyclic Codes

TABLE 5.'I: A (7, 4)

11;(X)

CC
(0010) (1010) (0110) (1110) (0001) (1001) (0101) (1101) (0011) (1011)

0000000 1 01000 0110100 1011100 0011010 110010 0101 10 1000110 0001101 1100] 01 011100 1010001 0010111 1111111

(0111) (1111)

0100011 1001011

(0 0 0 0)

(1000) (0100) (110 0)

'113l7

= 1 + X + X3 .

CC
0

= 0 · 11;(X)

1+X

+ X3 = x+ + +

1·

. ~(X) (1 + X 2 ) · ~(X) + X 5 = (X + · ~(X) = (1 + X + · ~(X) =

X + 1+

+ x 4 + x 6 = x 3 -11;(X) = (J + X 3 ) · ~(X) v + v3) ·(X) ·r v6 == 2 6 1 + X + x = (1 + X + X 3 ) · g(X) + X 4 + x 5 + x 6 = (X 2 + X 3 ) · ~(X) 1+ x + + x3 + + x5 + x 6 2 = (1 + X + · !!;(X) X + X 5 + X6 = + + X 3 ) · !!;(X) 1+ + x5 + x 6 = (1 + X + X 2 + X 3 ) · ~(X) ..!l _J!-

_J

A

..tl

· ~

!L

shall call v(X) the code of v. Hereafter, we shall use the terms codeword and code interchangeably. The code that corresponds to codeword V(i)

= Vn-i + V11--i+l )7I.

_J

T...

+ V11-lAvi-1 + VQAvi+ v1 xi+l T . . . + V11-i-l x11-l . J

There exists an un•d""''""·"'· algebraic relationship between v(X) and vU\X). Multi'i"(X) by , ·we obtain _

-

vi

VOA

+ v1 xi+l + · · · ,- V11-i-lAv11-l + · · · + V11-1A?11+i-l . I

The preceding

+ v,1-i+lX + ... + Vn-l -l + voXi + ... + v,1-i-1x 11 - 1 11 + V11-i(X + 1) + V11-i+1X(X 11 + 1) + ... + V11-1Xi-l(X + 1) = (ij\(X)(X" + 1) + wU\x),

w(X) = Vn-i

11

where (ij\(X)

=

Vn-i

(5.1)

+ Vn-i+lX + · · · + v11 _1xi-l.

,,,,,,,,,~, vU)(X) is Xiw(X) by x 11 + 1.

From (5.1) we see that the code the remainder resulting from dividing the polynomial

Next, vve prove a number of algebraic properties of a cyclic code that make possible the simple implementation of encoding and syndrome computation. §JL The nonzero code polynomial of minimum degree in a code C is unique.

1I'IllIJEl[J)JRJEM

138

Chapter 5

Cyclic Codes

Proof. Let g(X) =go+ g1X + · · · + gr-lxr-l + xr be a nonzero code polynomial of minimum degree in C. Suppose that g(X) is not unique. Then, there exists another code polynomial of degree r, say g' (X) = gb + g~ X + ·. · + g;._ 1xr- 1 + xr. Because C is linear, g(X) + g'(X) =(go+ gb) + (g1 + gi)X + · · · + (gr-l + g;__ 1)xr- 1 is a code polynomial of degree less than r. If g(X)+g' (X) =I- 0, then g(X)+g' (X) is a nonzero code polynomial of degree less than the minimum degree r. This is impossible. Therefore, g(X) + g' (X) = 0. This implies that g'(X) = g(X). Hence, g(X) is unique. Q.E.D. THEOREM 5.2 Let g(X) = go+ g1X + ... + gr-lxr-l + xr be the nonzero code polynomial of minimum degree in an (n, k) cyclic code C. Then, the constant term go must be equal to 1.

Proof. Suppose that go

= 0. Then,

= X(g1 + g2X + · · · + gr-lxr- 2 + xr- 1). If we shift g(X) cyclically n - 1 places to the right (or one place to the left), we obtain a nonzero code polynomial, g1 + g2X + · · · + gr-l xr- 2 + xr-l, of degree less than r. This is a contradiction to the assumption that g(X) is the nonzero code polynomial with minimum degree. Thus, go =I- 0. Q.E.D.

It follows from Theorem 5.2 that the nonzero code polynomial of minimum degree in an (n, k) cyclic code C is of the following form: r l r g(X)= 1 +g1X+g2X 2 +···+gr-1X+X.

(5.2)

Consider the (7, 4) cyclic code given in Table 5.1. The nonzero code polynomial of minimum degree is g(X) = 1 + X + x 3 . Consider the polynomials Xg(X), X 2g(X), ... , xn-r- 1g(X), of degrees r + 1, r+2, · · · , n-1, respectively. It follows from (5.1) that Xg(X) = g<1l(x), X 2g(X) = g<2l(X), ... , xn-r-lg(X) = g
(5.3) = (uo + u1X + ... + Un-r-lxn-r-l)g(X).

is also a code polynomial, where u; = 0 or 1. The following theorem characterizes an important property of a cyclic code. THEOREM 5.3

Let g(X) = l+g1X + · · · +gr-lxr-l +xr be the nonzero code polynomial of minimum degree in an (n, k) cyclic code C. A binary polynomial of degree n - 1 or less is a code polynomial if and only if it is a multiple of g(X).

Proof. Let v(X) be a binary polynomial of degree n - 1 or less. Suppose that v(X) is a multiple of g(X). Then, v(X) = (ao + a1X + · · · + a 11 _r-1Xn-r-l)g(X) = aog(X) + a1Xg(X) + · · · + a 11 _r-1X 11 -r-lg(X).

Section 5.1

Description of Cyclic Codes

B!ll

Because v(X) is a linear combination of the code polynomials !!;(X), X !!;(X), · · · , xn-r-l!!;(X), it is a code polynomial in C. This proves the first of the theorem-that if a polynomial of degree n - 1 or less is a of !!;(X), it is a code Now, let v(X) be a code polynomial in C. Dividing v(X) by !!;(X), we obtain v(X)

= m(X)!!;(X) + lbi(X),

where either fOJ(X) is identical to zero, or the degree of fr:D(X) is less than the degree of ~(X). Rearranging the preceding we have lbi(X) = v(X)

+ :m(X)!!;(X).

It follows from the first part of the theorem that ru(X)!!;(X) is a code polynomial.

Because both v(X) and ru(X)!!;(X) are code polynomials, lbi(X) must also be a code polynomial. If \bJ(X) I- 0, then !bi()() is a nonzero code polynomial whose degree is less than the degree of ~(X). This contradicts the that !!;(X) is the nonzero code polynomial of minimum Thus, \bJ(X) must be identical to zero. This proves the second part of the theorem - that a code polynomial is a multiple of !!;(X).
+ mu•V + m2Av2 + . . . + mn-k-l xn-k-l

T

I_

x11-/c ·

Summarizing the preceding results, we have the following theorem: fo an (n, k) cyclic code, there exists one and only one code polynomial of degree n - k,

'JflHlJEOJF?.JEM §,4)

m(X)

= 1 + !!;1X + 1!;2X 2 + · · · + 1/;

11

11

-k-1X -k-l

+ xn-k.

(5.4)

Every code polynomial is a multiple of g(X), and every binary polynomial of degree n - 1 or less that is a multiple of !!;(X) is a code polynomial. H follows from Theorem 5.4 that every code polynomial v(X) in an (n, k) cyclic code can be expressed in the following form: v(X) = TUI(X)!!;(X)

=

(uo

+ u1X + · · · + u1c-1Xk-l)!!;(X).

If the coefficients of 1lll(X), uo, u1, · · · , uk-l, are the k information digits to be

encoded, v(X) is the corresponding code polynomial. Hence, the encoding can be achieved by multiplying the message un(X) by ~(X). Therefore, an (n, k) cyclic code is completely specified by its nonzero code polynomial of minimum degree, g(X),

140

Chapter 5

Cyclic Codes

given by (5.4). The polynomial g(X) is called the generator polynomial of the code. The degree of g(X) is equal to the number of parity-check digits of the code. The generator polynomial of the (7, 4) cyclic code given in Table 4.1 is g(X) = 1 + X + X 3. We see that each code polynomial is a multiple of g(X). The next important property of a cyclic code is given in the following theorem. THEOREM 5.5 The generator polynomial g(X) of an (n, k) cyclic code is a factor of X 11 + 1.

Proof Multiplying g(X) by xk results in a polynomial Xkg(X) of degree n. Dividing Xkg(X) by X 11 + 1, we obtain (5.5)

where gCkl(X) is the remainder. It follows from (5.1) that g
Thus, g(X) is a factor of X 11 (n, k)

+ 1 = {X" + a(X)}g(X).

11

+ 1.

Q.E.D.

At this point, a natural question is whether, for any n and k, there exists an cyclic code. This question is answered by the following theorem. 5.6 If g(X) is a polynomial of degree n - k and is a factor of X 11 + 1, then g(X) generates an (n, k) cyclic code.

THEOREM

Proof. Consider the k polynomials g(X), Xg(X), ... , xk- 1g(X), all of degree n - 1 or less. A linear combination of these k polynomials, v(X)

= aog(X) + a1Xg(X) + · · · + a1c-1Xk-lg(X) =

(ao

+ a1X + · · · + ak-lxk-l)g(X),

is also a polynomial of degree n - 1 or less and is a multiple of g(X). There are a total of 2k such polynomials, and they form an (n, k) linear code. Let v(X) = vo + v1X + · · · + v11 _1x 11 Multiplying v(X) by X, we obtain Xv(X)

1

be a code polynomial in this code.

+ v1X 2 + ... + V11-2X 1 + Vn-1X = Vn--1 (X + 1) + (vn-1 + voX + ... + V11-2X"- 1) = Vn-1 (X + 1) + v< 1\X), =

11

voX

11

-

11

11

where vCl) (X) is a cyclic shift of v(X). Since both X v(X) and X 11 + 1 are divisible by g(X), v< 1l(X) must be divisible by g(X). Thus, vC 1\X) is a multiple of g(X) and is a linear combination of g(X), Xg(X), ... , xk- 1g(X). Hence, vCll(X) is also a code polynomial. It follows from Definition 5.1 that the linear code generated by g(X), Xg(X), ... , x"- 1g(X) is an (n, k) cyclic code. QJJ:.D.

Section 5.1

Description of Cyclic Codes

Uli

Theorem 5.6 says that any factor of X" + 1 with degree n - k generates an + 1 may have many factors of degree n - k. Some of these polynomials generate good codes, and some generate bad codes. How to select generator polynomials to produce good cyclic codes is a very difficult problem, and coding theorists have expended much effort in searching for good cyclic codes. Several classes of good cyclic codes have been discovered, and they can be practically implemented. (n, k) cyclic code. For large n, X 11

IEKAM!Fl'LIE §"1

The polynomial X 7

+ 1 can be factored as follows:

There are two factors of degree 3, and each generates a (7, 4) cyclic code. The (7, 4) cyclic code given by Table 5.1 is generated by 11:(X) = 1 + X + X 3. This code has a minimum distance of 3 and it is a single-error-correcting code. Notice that the code is not in systematic form. Each code polynomial is the product of a message polynomial of degree 3 or less and the generator polynomial g(X) = 1 + X + X 3 . For example, let 11ll = (1 0 1 0) be the message to be encoded. The corresponding message polynomial is illl(X) = 1 + X 2 . Multiplying llll(X) by 11:(X) gives us the following code polynomial:

+ X 2 )(1 + X + X 3 ) = 1 + x + x 2 + x5,

v(X) = (1

or the codeword (11100 1 0). Given the generator polynomial g(X) of an (11, k) cyclic code, we can put the code into systematic form (i.e., the rightmost k digits of each codeword are the unaltered information digits, and the leftmost n - k digits are parity-check digits). Suppose that the message to be encoded is llil = (uo. u1, · · · . uk-1). The corresponding message polynomial is 11II(X)

Multiplying llll(X) by

xn-k,

= uo + iqX + · · · + u1c-1Xk-l_

we obtain a polynomial of degree

11 -

1 or less:

Dividing xn-kllll(X) by the generator polynomial g(X), we have

x

11

-"iu(X)

= lll(X)g(X) + 11:D(X)

(5.6)

where lll(X) and lb(X) are the quotient and the remainder, respectively. Because the degree of g(X) is n - k, the degree of lbi(X) must be n k - 1 or less; that is,

142

Chapter 5

Cyclic Codes

Rearranging (5.6), we obtain the following polynomial of degree n - 1 or less: b(X)

+ xn-kllll(X) = a(X)g(X).

(5.7)

This polynomial is a multiple of the generator polynomial g(X) and therefore it is a code polynomial of the cyclic code generated by g(X). Writing out b(X) + x 11 -ku(X), we have b(X)

+ xn-klJl!(X) = bo + b1X + · · · + bn-k-1Xn-k-l + uoxn-k + u1X11-k+l + ... + Uk-lxn-1,

(5.8)

which corresponds to the codeword (bo, b1, · · · , bn-k-1, UO, Ul, · · · , UJc-1).

We see that the codeword consists of k unaltered information digits (uo, u1, · · · , u1c-1) followed by n -- k parity-check digits. The n -- k parity-check digits are simply the coefficients of the remainder resulting from dividing the message polynomial xn-kID!(X) by the generator polynomial g(X). The preceding process yields an (n, k) cyclic code in systematic form. In connection with cyclic codes in systematic form, the following convention is used: the first n - k symbols, the coefficients of 1, X, ... , xn-k-l, are taken as parity-check digits, and the last k symbols, the coefficients of xn-k, x 11 -1c+1, ... , x 11 -1, are taken as the information digits. In summary, encoding in systematic form consists of three steps: Step 1. Premultiply the message u(X) by xn-k. Step 2. Obtain the remainder b(X) (the parity-check digits) from dividing x 11 -k11ll(X) by the generator polynomial g(X). Step 3. Combine lbi(X) and xn-kM(X) to obtain the code polynomial b(X) + xn-ku:(X).

IEXAMPllE 5.2

Consider the (7, 4) cyclic code generated by g(X) = 1 + X + x 3 . Let u(X) be the message to be encoded. Dividing x 3u(X) = X 3 + x 6 by g(X),

= 1 + X3

x 3 +x (quotient) x 3 + x + 11 x 6

x6

x3

+x4+ x3 x4

x4

+x 2 +x

x 2 +X

(remainder)

= X+ the code polynomial is v(X) = b(X) + X31lll(X) = X + x 2 + X 3 + x 6 , and the corresponding codeword is v = (0111001),

we obtain the remainder b(X)

X 2 . Thus,

where the four rightmost digits are the information digits. The 16 codewords in systematic form are listed in Table 5.2.

Generator and

Section 5.2 TABLE 52: A

Matrices of

form generated by

code in

4)

1-+X+

§,;:/2

Mes§21me

(C([J)iilJew([J)rr@

(0 0 00) (1000) (0100) (1100) (0010) (1010) (0110) (111 0) (0001) (1001) (0101) (1101) (0011) (1011) (0111) (1111)

(0 0 0 0 0 0 0) (1101000) (0110100) (1011100) (1110010) (0011010) (1000110) (0101110) (1010001) (0111001) (1100101) (0001101) (0100011) (1001011) (0010111) (1111111)

= i?;(X) x+x 2 + =Xi?;(X) 1+ + x 3 + x 4 = (1 + X)i?;(X) 1+ X + + X 5 = (1 + X 2 )11;(X) + + = X 211;(X) 5 1+ + X = (1 + X + X 2 )i(X) X + X 3 -1+ -1- X 2)g(X) 2 6 1 + X -1- x = (1 -1- X -1- X 3 )ii;(X) X + -1+ = (X + X 3 )it(X) 4 1+X +X + = (1 + X

+

+x.6 = 6 +X =

+

+ X5 -1- X 6 = (1 + X + + X 3 )ii;(X) -1- x4 + -1- x6 = (X2 + X3)g(X) 1 -1- X -1- X. 2 + + + + X6 5 = (1 + + x )!!;(X)

1+

GIENIEIFi,t\YO[,~ [~ND iPJfo:J~J1rV~(MHJ( M/Ji.Im(IE§ OIF (V(U(

((Q)[J)[E§

Consider an (n, k) code C with polynomial 11;(X) =go+ g1X + · · · + . J[n Section 5.1 we showed that the k code ii;(X), Xii;(X), · · · , span C. If the to these k code polynomials are used as the rows of a k x n generator matrix for C: go

g1

0 0

go 0

0

0

g2 g1

go

O

gn--k

g2 g1

g11-k

g2

O O gn-k

O O O

0 0 0


g11-k

(5.9)

(Note that go = g11 __ k =, 1.) In general,
r~

-l~

1 0 1 0 0 1 1 0 1 0 0 1 1 0 l 0 0 1 1 0

n

144

Chapter 5

Cyclic Codes

l

Clearly, G is not in systematic form. If we add the first row to the third row, and if we add the sum of the first two rows to the fourth row, we obtain the following matrix: 1 1 0 1 0 0 0 G'- 0 1 1 0 1 0 0 1110010' [ 1 0 1 0 0 0 1 which is in systematic form. This matrix generates the same code as G. Recall that the generator polynomial g(X) is a factor of X 11 + 1, say X 11 + 1

= g(X)h(X),

(5.10)

where the polynomial h(X) has degree k and is of the following form: h(X)

=ho+ h1X + · · · + hkx"

with ho = hk = 1. Next, we want to show that a parity-check matrix of C may be obtained from h(X). Let v = (vo, v1, · · · , v11 _1) be a codeword in C. Then, v(X) = a(X)g(X). Multiplying v(X) by h(X), we obtain v(X)h(X)

= a(X)g(X)h(X) = a(X)(X + 1) 11

(5.11)

= a(X) + X 11 a(X).

Because the degree of a(X) is k 1 or less, the powers xk, xk+ 1 , ... , xn-l do not appear in a(X) + X 11 a(X). If we expand the product v(X)h(X) on the left-hand side of (5.11), the coefficients of xk, xk+1, ... , xn-l must be equal to zero. Therefore, we obtain the following n - k equalities: k

"I:,h;Vn-i-j=O

(5.12)

forl~j~n-k.

i=O

Now, we take the reciprocal of h(X), which is defined as follows: 6

kn. - 1)= h k+ h 1c-1X+ h k-2X 2 +···+ h o Xu(X

x" .

(5.13)

We can easily see that Xkh(x- 1) is also a factor of X 11 + 1. The polynomial Xklln(x- 1 ) generates an (n, n - k) cyclic code with the following (n - k) x n matrix as a generator matrix:

hk h1c-1 hk-2 0 hk hk-1 hk-2 0 0 hk hk-1 hk-2 H= 0

0

ho

0 hk

h1c-1

h1c-2

0 ho

0 ho

0 0 0

ho (5.14)

Section 5.2

Generator and Parity-Check Matrices of Cyclic Codes

145

H follows from then - k

of (5.12) that any codeword •1 in C is orthogonal code C, and to every row of JBI. Therefore, lHI is a parity-check matrix of the the row space of lHI is the dual code of C. Since the parity-check matrix IHI is obtained from the polynomial lb(X), we call Iln(X) the of C. a cyclic specified by its code is also a parity-check matrix for a code, we have also property, which is stated in the foHowing theorem. 'fJHIJE(())lP!.JEJ.VJ! §.7 Let C be an (n, k) The dual code of C is also and is where lln(X) = (X 11 + 1)/g(X).

IE}{AM!Jl'U: 5.3l

Consider the (7, 4) cyclic code in Table 5.1 with generator polynomial + x 3 . The polynomial is

1];(X) = 1 + X

x7 + 1

lln(X)= ~ 1];(X)

= 1+ x + x 2 +x

4

.

The reciprocal of lln(X) is

+ x- 1 + x- 2 + x- 4 ) = 1 + x2 + x3 + divides X 7 + 1 : + 1)/ X 4 Iln(X- 1 )

x 4 1lncx- 1 ) = X 4 (1

This polynomial X 4 1ln(X- 1 ) = 1 + X 2 + X 3. H we construct all the codewords of the (7, 3) code generated by X 4 lln(X- 1 ) = 1 + X2 + x3 + , we will find that it has a minimum distance of 4. Hence, it is capable of correcting any single error and simultaneously detecting any combination of double errors. We also can easily form the generator matrix in systematic form. Dividing the generator polynomial 1];(X) for i = 0, 1, ... , k - 1, we obtain

xn-k+i

xn-k+i = &11;(X)1];(X)

+ lbi;(X),

(5.15)

where \bi; (X) is the remainder with the following form: fOJi(X) = bio

+ bi1X + · · · + bi,11-1c-1X 11 -k-l_

Because lbii (X) + xn-k+i for i = 0, 1, ... , k - 1 are multiples of g(X), they are code polynomials. Arranging these k code polynomials as rows of a k x n matrix, we obtain bo2 b12

bo,,,-k-1

b20

bo1 b11 b21

b22

b2.11-k-l

b1c-l,O

b1c-1,1

b1c-1,2

b1c-l,11-k-l

boo b10

IG=

b1,n-k-l

1 0 0 1 0 0

0 0 1

0

0

0

0 0 0 ...

1

(5.16)

146

Chapter 5

Cyclic Codes

which is the generator matrix of C in systematic form. The corresponding paritycheck matrix for C is

H=

1 0 0 0 1 0 0 0 1 0 0 0

0 0

...

boo

b10

b11

b20 b21

bk-1,0

0

bo1 b02

b12

b22

bk-1,2

1

bo,11-k-1

b1,11-k-l

b2,n-k-l

bk-I,11-k-l

bk-1,1

(5.17)

EXAMIPllE 5.4

= 1 + X + X 3 . Dividing

Again, consider the (7, 4) cyclic code generated by g(X) x3, x4, x 5 , and x 6 by g(X), we have 3

= g(X) + (1 + X), 4 X = Xg(X) + (X + X 2),

X

+ (1 + X + X 2), 6 2 X = (X 3 + X + l)g(X) + (1 + X ). X

5

=

(X 2 + l)g(X)

Rearranging the preceding equations, we obtain the following four code polynomials: v 0 (X) v1 (X) v2(X)

=1 + X + x3 , 2 = x + x + x4 , = 1 + X + x2 xs,

+ x2

v3(X) = 1

+ x 6.

l

Taking these four code polynomials as rows of a 4 x 7 matrix, we obtain the following generator matrix in systematic form for the (7, 4) cyclic code:

G=

r

1 0 1 1

1 0 1 0 1 1 0 1 1 1 0 0 0 1 0 0

0 0

0 0 1 0 0 1

.

which is identical to the matrix G' obtained earlier in this section.

5.3

ENCODING OF CYCUC CODES

As shown in Section 5.1, encoding of an (n, k) cyclic code in systematic form consists of three steps: (1) multiplying the message polynomial u(X) by x 11 -"; (2) dividing X"-"u(X) by g(X) to obtain the remainder b(X); and (3) forming the codeword b(X) + x 11 -ku(X). All three steps can be accomplished with a division circuit that is a linear (n - k)-stage shift register with feedback connections based on the generator polynomial g(X) = 1 + g1X + g2X 2 + · · · + gn-k-1X 11 -k-l + x 11 -k. Such a circuit is shown in Figure 5.1. The encoding operation is carried out as follows:

r

----y

l~ cb

Section 5.3

x

Encoding of Cyclic Codes

1.lil1

1 {~}

y

g, '

~{~~}4>~0B->

·--Q

o o o

-0~~ 1

X" - 1'nn(X)

Message

+

~ ~ a . . _ _ Codeword

--~-o ~ -

Parity-check digits

FIGURE 5.1:

Encoding circuit for an (n.

iJ;(X) = 1 + g1X2

k)

cyclic code with generator polynomial

+ ... + gn-k-lxn-k-1 + xn-k.

lio Tum on the gate. The k information digits uo, u1, · · · , uk-1 [or llll(X) = uo + iqX + · · · + ttk-lxk-l in polynomial form] are shifted into the circuit and simultaneously into the communication channel. Shifting the message llll(X) into the circuit from the front end is equivalent to premultiplying illl(X) by xn-k. As soon as the complete message has entered the circuit, then - k digits in the register form the remainder, and thus they are the parity-check digits. §1tie]lll Zo Break the feedback connection by turning off the gate. §tlie]Pl :lo Shift the parity-check digits out and send them into the channel. These n - k parity-check digits bo. b1. · · · . bn-k-1, together with the k information digits, form a complete codeword. §1tie]Pl

IEKAMIP'llE 5.5

Consider the (7, 4) cyclic code generated by 1:,1;(X) = 1 + X + X 3 . The encoding circuit based on g(X) is shown in Figure 5.2. Suppose that the message 1ill = (1011) is to be encoded. As the message digits are shifted into the register the contents of the register change as follows: Illlil]P!illltl

~l!.':1:,';Il§tlien· IC
1 1 0 1

0 0 0 (initial state) 110 (first shift) 1 0 1 (second shift) 1 0 0 ( third shift) 100 (fourth shift)

After four shifts, the contents of the register are (10 0). Thus, the complete codeword is (1001011). and the code polynomial is 1 + x 3 + x 5 + x 6 . Encoding of a cyclic code can also be accomplished by using its parity polynomial lhl(X) =ho+ h1 X + · · ·+ h1.;Xk. Let w = (vo. v1. · · · . v11 _1) be a codeword. We have shown in Section 5.2 that the components of w satisfy then - k equalities

148

Chapter 5

Cyclic Codes

X" - ku(X) --+------i><:,___ Codeword

Message

Parity-check digits

FIGURE 5.2:

Encoder for the (7, 4) cyclic code generated by g(X) = 1 + X

'----<>-D

FIGURE 5.3:

+ X 3.

Output to channel

Encoding circuit for an (n, k) cyclic code based on the parity polynomial

h(X) = 1 + h1X

+ · · · + X".

of (5.12). Since h1c

= 1, the equalities of (5.12) can be put into the following form: k-1

Vn-k-j

= "5:,hiV11-i-j

for 1:::: j:::: n - k

(5.18)

i=O

which is known as a difference equation. For a cyclic code in systematic form, the components Vn-k, Vn-k+l, · · · , v11 _1 of each codeword are the information digits. Given these k information digits, (5.18) is a rule for determining the n - k paritycheck digits, vo, v1, · · · , vn-k-1· An encoding circuit based on (5.18) is shown in Figure 5.3. The feedback connections are based on the coefficients of the parity polynomial lhl(X). (Note that ho = hk = 1.) The encoding operation can be described in the following steps: Step lo Initially, gate 1 is turned on and gate 2 is turned off. The k information digits llll(X) = uo + u1X + · · · + uk-lxk-l are shifted into the register and the communication channel simultaneously. Step 2o As soon as the k information digits have entered the shift register, gate 1 is turned off and gate 2 is turned on. The first parity-check digit, Vn-k-1

= hovn-1 + h1 Vn-2 + ·' · + hk-1 V11-k

is formed and appears at point P.

Section 53 §1tief[l)

:t The

is shifted once. The first the channel and is also shifted into

= Uk-2

-I-

is shifted into the second

-I-··· -I-

h1uk-3

1~-21

of Cyclic Codes

hk-2UO

-I- hk-1 Vn-k-l,

is formed at P.

have been formed and 2 is turned gate 1 is turned on and to be shifted into the register.

3 is shifted into the channeL

a k-stage shift we can make the the ... u,.,v,.uuF, circuit is preferable.

code

The Lln(X)

form v

=

(vo,

v1.

vo, 111,

and

v2

are

=

-I- l 1 -I- X -I- X 3

v2. v3. v4. vs,

~(X)

= 1+X

--1-

= 1 -I- X

is

-I-

-I-

-

v6), where

v3, v4, vs, and The difference

V6

are message and that determines the

lS

113-j

= 1 · V7-j

-I- 1 · V6-j -I- 1 · VS-j -I- Q · VJ.-j for 1 :": j :": 3.

= V7-j -I- V6-j -I- VS-j

Suppose that the message to be encoded is (1 0 1 1, V6 = 1. The first is V2

The second

= V6

1 -I- 1 -I- 0

=

1,

v4

=

0,

vs

= 0.

-I-

V5

-I-

V4

=.c

=

V5

-I-

VJ.

-I-

V3

= 1 -I- 0 -I- 1 = 0.

T=--Ga,, 1

v3

digit is VJ

'"":,1

Then,

.·J

&-~~&~-

LLtJ-{~--

f--f,,>[1---~[l1. -·-----.., o Output to channel

FIGURE 5.4: Encoding circuit for the (7, 4) lln(X) = 1 + X --1--1-

code based on its

polynomial

150

Chapters

Cyclic Codes

The third parity-check digit is VO= V4

+ V3 + V2

= Q + 1 + Q = 1.

Thus, the codeword that corresponds to the message (1011) is (1001011).

5.4

SYNDROME COMPUTATION AND ERROR DETECTION

Suppose that a codeword is transmitted. Let r = (ro, r1, · · · , r 11 _1) be the received vector. Because of the channel noise, the received vector may not be the same as the transmitted codeword. In the decoding of a linear code, the first step is to compute the syndromes = r . HT, where His the parity-check matrix. If the syndrome is zero, r is a codeword, and the decoder accepts r as the transmitted codeword. If the syndrome is not identical to zero, r is not a codeword, and the presence of errors has been detected. We have shown that for a linear systematic code the syndrome is simply the vector sum of the received parity-check digits and the parity-check digits recomputed from the received information digits. For a cyclic code in systematic form, the syndrome can be computed easily. The received vector r is treated as a polynomial of degree n - 1 or less, r(X)

= ro + riX + r2X 2 + · · · + r

11

_1 X

11 -

1

.

Dividing r(X) by the generator polynomial g(X), we obtain r(X)

= a(X)g(X) + §(X).

(5.19)

The remainder s(X) is a polynomial of degree n - k - 1 or less. Then - k coefficients of §(X) form the syndrome §. It is clear from Theorem 5.4 that s(X) is identical to zero if and only if the received polynomial r(X) is a code polynomial. Hereafter, we will simply call s(X) the syndrome. The syndrome can be computed with a division circuit as shown in Figure 5.5, which is identical to the (n - k)-stage encoding circuit, except that the received polynomial r(X) is shifted into the register from the left end. The received polynomial r(X) is shifted into the register with all stages initially set to 0. As soon as the entire r(X) has been shifted into the register, the contents in the register form the syndrome s(X).

r(X) Received vector

FIGURE 5.5: An (n - k )-stage syndrome circuit with input from the left end.

Section 5.4

and Error Detection

structure of the code, the syndrome §(X) has the following

Because of the property.

of a received the remainder §(1) ()() X§(X) the generator polynomial !§;(X) is the syndrome of shift of ir(X).

1I'l!lIJEOJRJEMI §,~

1!Sii

Let §(X) be the

ro + r1X + · · · + r11 _1 x 11 - 1.

Ir(X) =

IPr({))({))J: H follows from (5.1) that Ir(X) and ll"(l\X) satisfy the following rela-

Xrr(X) = r,,_1 (X 11

+ l) + n,(1\X).

(5.20)

Rearranging (5.20), we have Ir(lJ (X) = r,,_1 (X" Dividing both sides of (5.21) by 1§;(X) and we obtain
+ p(X)

+ l) + Xrr(X). the fact that X11 + 1 = %§(X)lhl(X),

= r 11 _1~(X)lhl(X) -1- X[21(X)~(X)

+ §(X)],

where p(X) is the remainder resulting from dividing Ir(ll (X) by ~(X). Then, p(X) is the syndrome otrr< 1\X). Rearranging (5.22), we obtain the and X§(X): X§(X) = [ir(X)

+r

11

_1lhl(X)

relationship between p(X)

+ Xm(X)]~(X) + p(X).

(5.23)

From (5.23) we see that p(X) is also the remainder resulting from dividing X§(X) by ~(X). Therefore, p(X) = §(ll (X). This completes the proof. QJEJI]), It follows from Theorem 5.8 that the remainder r/il (X) resulting from dividing X; §(X) by the generator polynomial g(X) is the syndrome of rrUl (X), which is the ith cyclic shift of n·(X). This property is useful in decoding codes. We can obtain the syndrome §Cl) (X) of ll"Cl) (X) shifting (or the syndrome register once with 1l(X) as the initial contents and with the gate disabled. This is because shifting the syndrome register once with §(X) as the initial contents is equivalent to dividing X§(X) by g(X). Thus, after the shift, the register contains §C1l(X). To obtain the syndrome §Ul(x) of rrUl (X), we simply shift the syndrome register i times with §(X) as the initial contents. IE}{AM~LIE !5.1

A syndrome circuit for the (7, 4) cyclic code generated by g(X) 1 + X + X3 is shown in Figure 5.6. Suppose that the received vector is rr = (0010110). The syndrome of rr is § = (101). Table 5.3 shows the contents in the register as the received vector is shifted into the circuit. At the end of the seventh shift, the register contains the syndrome§= (101). H the register is shifted once more with the input gate disabled, the new contents will be §(l) = (1 0 0), which is the syndrome of rrC 1l (X) = (0001011), a cyclic shift of rr.

152

Chapter 5

Cyclic Codes

FIGURE 5.6: Syndrome circuit for the (7, 4) cyclic code generated by g(X) = 1

+

X +X3. 5.3: Contents of the syndrome register shown in Figure 5.6 with r = (0 0 1 0 11 0) as input.

TABLE

Register contents

Shift 1 2 3 4 5 6 7

0 1

1 0 1 0 0

8 9

000 (initial state) 000 100 110 011 011 111 101 (syndromes) 100 (syndrome s(l)) 0 1 0 (syndrome s< 2l)

We may shift the received vector r(X) into the syndrome register from the right end, as shown in Figure 5.7; however, after the entire r(X) has been shifted into the register, the contents in the register do not form the syndrome of r(X); rather, they form the syndrome s<11 -"l(X) of r< 11 -kl(X), which is the (n - k)th cyclic shift of r(X). To show this, we notice that shifting r(X) from the right end is equivalent to premultiplying r(X) by xn-k. When the entire r(X) has entered the register, the register contains the remainder p(X) resulting from dividing x 11 -kr(X) by the generator polynomial g(X). Thus, we have

x

11

-"r(X)

= a(X)g(X) + p(X).

(5.24)

It follows from (5.1) that r(X) and r< 11 -"\X) satisfy the following relation:

x

11

-"r(X)

= b(X)(X + 1) + r
Combining (5.24) and (5.25) and using the fact that X 11 r< 11 -"\X) = [lb(X)h(X)

+1 =

(5.25) g(X)h(X), we have

+ a(X)]g(X) + p(X).

This equation says that when r
Section 5.4

Syndrome Computation and Error Detection

153i

Received vector

FIGURE 5.7:

An (n - k)-stage syndrome circuit with input from the right end.

Let w(X) be the transmitted codeword, and let ie(X) be the error pattern. Then, the received polynomial is ir(X)

= w(X) + ie(X).

=

eo+e 1 X + · · ·+e11 _1xn-l

(5.26)

Because Y(X) is a multiple of the generator polynomial ;!g(X), combining (5.19) and (5.26), we have the following relationship between the error pattern and the syndrome: (5.27) ie(X) = [m(X) + lbi(X)]g(X) + s(X), where lbi(X)g(X) = w(X). This shows that the syndrome is equal to the remainder resulting from dividing the error pattern by the generator polynomial. The syndrome can be computed from the received vector; however, the error pattern e(X) is unknown to the decoder. Therefore, the decoder has to estimate e(X) based on the syndrome s(X). If ie(X) is a coset leader in the standard array and if table-lookup decoding is used, e(X) can correctly be determined from the syndrome. From (5.27) we see that s(X) is identical to zero if and only if either the error pattern ie(X) = 0 or ie(X) is identical to a codeword. U e(X) is identical to a code polynomial, ie(X) is an undetectable error pattern. Cyclic codes are very effective for detecting errors, random or burst. The error-detection circuit is simply a syndrome circuit with an OR gate whose inputs are the syndrome digits. H the syndrome is not zero, the output of the OR gate is 1, and the presence of errors has been detected. Now, we investigate the error-detecting capability of an (n, k) cyclic code. Suppose that the error pattern e(X) is a burst of length n k or less (i.e., errors are confined ton - k or fewer consecutive positions). Then, we can express e(X) in the following form: e(X) = X 11B(X),

where O ::: j ::: n - l, and IB(X) is a polynomial of degree n - k - 1 or less. Because the degree of JE(X) is less than the degree of the generator polynomial g(X), Iffi(X) is not divisible by g(X). Since g(X) is a factor of X 11 + 1, and X is not a factor of g(X), g(X) and xJ must be relatively prime. Therefore, e(X) = XlJE,(X) is not

154

Chapter 5

Cyclic Codes

divisible by g(X). As a result, the syndrome caused by e(X) is not equal to zero. This implies that an (n, k) cyclic code is capable of detecting any error burst of length n - k or less. For a cyclic code, an error pattern with errors confined to i high-order positions and l - i low-order positions is also regarded as a burst of length l or less. Such a burst is called an end-around burst. For example, e

( 1

0

1

0

0

0

0

0

0

0

0

1

1

0

1

is an end-around burst of length 7. An (n, k) cyclic code is also capable of detecting all the end-around error bursts of length n - k or less (the proof of this is left as a problem). Summarizing the preceding results, we have the following property: THIEORIEM 5o9 An (n, k) cyclic code is capable of detecting any error burst of length n - k or less, including the end-around bursts.

In fact, a large percentage of error bursts of length n - k + 1 or longer can be detected. Consider the bursts of length n - k + 1 starting from the ith digit position and ending at the (i + n k)th digit position (i.e., errors are confined to digits . h e; -- e;+n-k -- 1) . Th ere are 2n-k-l sue h b ursts. A mong e;, e;+1, · · ·, e;+n-k, wit these bursts, the only one that cannot be detected is

Therefore, the fraction of undetectable bursts of length n - k + 1 starting from the ith digit position is 2-(n-k-l). This fraction applies to bursts of length n - k + 1 starting from any digit position (including the end-around case). Therefore, we have the following result THIEOJRIEM

5o1t0 The fraction of undetectable bursts of length n - k

+1

is

2-(n-k-l).

For l > n - k + 1, there are l1- 2 bursts of length l starting from the ith digit position and ending at the (i + l - l)th digit position. Among these bursts, the undetectable ones must be of the following form: e(X)

=

X; a(X)g(X),

where a(X) = ao + a1X + · · · + a1-(n-k)-1x 1- n - k + 1, the fraction of undetectable error bursts of

THIEORIEM

SoU

length l is

2-(n-k).

The preceding analysis shows that cyclic codes are very effective for burst-error detection.

of

Section 55

Codes

·i '5'5

The (7. by = 1 + X + has a minimum distance of 3. It is any combination of two or fewer random errors or any burst of length 3 or less. H also detects many bursts of than 3.

can be with a division circuit \Vhose is to the number of n - k). The error-correction step is (modulo-2) the error pattern to the received vector. This addition can be performed with a EXCLUSIVE-OR gate if correction is carried out serially at a n EXCLUSIVE-OR are required if correction is as shown in Figure 3.8. The association of the syndrome with can be a table. A straightforward circuit is via a combinational circuit ,,,~..,,~,,CV the table-lookup however., the limit to this approach of the decoding circuit tends to grow exponentially with the and with the number of errors that are going to be corrected. codes have considerable and If these ,~r.~,~.nr> aeco1:J1rnt2: circuits can be structure of a code allows us to decode a received vector n·(X) = ro + r 1 X + + · · · + r11 _1x 11 - 1 serially. The received are decoded is decoded with the same As soon as the the circuit checks whether the crnnr1,n,,n with an error at the with e11 _1

=

1

n,,,, 0 • " " are

=r the

11 _1

form the

r11 _2 of ir(X) becomes the first

=

eo

+ e1X + · · · + e

11 _

1x

11

-

1

xe11 _1 = If §(X) does not 1, the received polynomial (stored in a 11

+ roX + · · · + r

shifted once

_2x 11 - 1, and the new contents in 1 §( )(){) of rr(l\X). Now, the second of r/ 1>(X). The same decoding circuit will check 11

whether §(l) (X) corresponds to an error pattern with an error at location x 11 - 1 . If the s(X) of ir(X) does correspond to an error with an error at location xn-l e11 _1 = the first received digit r11 _ 1 is an erroneous and it must be corrected. The correction is carried out the smn r11 _1 EB e11 _ 1 . This correction results in a modified received polynomial, denoted by Il'1(X) = ro + r1X + · · · + r 11 _2x 11 - 2 + (r11 _1 EB e11 _1)X 11 - 1 . The effect of the error e11 _ 1 on the syndrome is then removed from the syndrome §(X), by adding the syndron1e of e' (X) = xn-l to §(X). This sum is the syndrome of the modified received ir1 (X). Now, Il'1 (X) and the syndrome register are cyclically shifted once simultaneously. This shift results in a received polynomial 1 1 ir?l(X) = (r11 _1 EB e11 _1) + roX + · · · + r 11 _2X 11 - 1 . The syndrome Eli \X) of rr\ \X) is the remainder resulting from dividing X[§(X) + xn-l] by the generator polynomial

156

Chapter 5

Cyclic Codes

g(X). Because the remainders resulting from dividing Xs(X) and X 11 by g(X) are ,P>(X) and 1, respectively, we have sill (X)

= s(l) (X) + 1.

Therefore, if 1 is added to the left end of the syndrome register while it is shifted, we obtain sill (X). The decoding circuitry proceeds to decode the received digit r11 _2. The decoding of r11 _2 and the other received digits is identical to the decoding of r11 _1. Whenever an error is detected and corrected, its effect on the syndrome is removed. The decoding stops after a total of n shifts. If e(X) is a correctable error pattern, the contents of the syndrome register should be zero at the end of the decoding operation, and the received vector r(X) has been correctly decoded. If the syndrome register does not contain all O's at the end of the decoding process, an uncorrectable error pattern has been detected. A general decoder for an (n, k) cyclic code is shown in Figure 5.8. It consists of three major parts: (1) a syndrome register, (2) an error-pattern detector, and (3) a buffer register to hold the received vector. The received polynomial is shifted into the syndrome register from the left end. To remove the effect of an error digit on the syndrome. we simply feed the error digit into the shift register from the left end through an EXCLUSIVE-OR gate. The decoding operation is as follows:

r(X)

Buffer register

Received vector

ri

~~~~~----<~

+ 1--~--Corrected vector

Feedback connection

Syndrome register

••• Error pattern detection circuit

Syndrome modification

FIGURE 5.8: General cyclic code decoder with received polynomial r(X) shifted into the syndrome register from the left end.

Section 5.5 §i11Bl[ll 1Lo

§tliB![ll

'.t

§1tiBJJJl

2lo

§!IB]Jll

4lo

§tt<el[ll

5,

Decoding of

Codes

157

The syndrome is formed by the entire received vector into stored the register. The received vector is in the buffer register. The syndrome is read into the detector and is tested for the corresponding error pattern. The detector is a combinational logic circuit that is designed in such a way that its is 1 if and if the syndrome in the syndrome register corresponds to a correctable error with an error at the highest-order x11 - 1. That if a 1 appears at the of the detector. the received in the rightmost stage of the buffer register is assumed to be erroneous and must be corrected; if a O appears at the output of the detector, of the buffer is the received symbol at the assumed to be error-free, and no correction is necessary. Thus, the output of the detector is the estimated error value for the to come out of the buffer. The first received symbol is read out of the buffer. At the same time, the syndrome register is shifted once. If the first received is detected to be an erroneous it is then corrected the output of the detector. The of the detector is also fed back to the syndrome register to modify the syndrome to remove the error effect from the syndrome). This operation results in a new which corresponds to the altered received vector shifted one the right. The new syndrome formed in step 3 is used to detect whether the at the rightmost stage of the buffer second received symbol register) is an erroneous symbol. The decoder steps 2 and 3. The second received symbol is corrected in exactly the same manner as the first received symbol was corrected. The decoder decodes the received vector by symbol in the manner outlined until the entire received vector is read out of the buffer register.

The preceding decoder is known as a Meggitt decoder [11 ], which in principle to any cyclic code. But whether it is practical depends on its error-pattern detection circuit. In some cases the error-pattern detection circuits are simple. Several of these cases are discussed in subsequent chapters.

Consider the decoding of the (7, 4) cyclic code generated by ii;(X) = 1 + X + X 3 . This code has a minimum distance of 3 and is capable of correcting any single error over a block of seven digits. There are seven single-error patterns. These seven error patterns and the all-zero vector form all the coset leaders of the decoding table. Thus, they form all the correctable error patterns. Suppose that the received polynomial 1r(X) = ro + r1X + r2X 2 + r3X 3 + r4X 4 + rsX 5 + r6X 6 is shifted into the syndrome register from the left end. The seven single-error patterns and their corresponding syndrome are listed in Table 5.4.

158

Chapter 5

Cyclic Codes

Error patterns and their syndromes with the received polynomial r(X) shifted into the syndrome register from the left end. TABLE 5.4:

Enor pruttern e(X) 6

=X e5(X) = X 5 e4(X) = X 4 e3(X) = X 3 e2(X) = X 2 e 1 (X) = X 1 e 0 (X) = x 0 e 6 (X)

§(X) §(X) s(X) s(X) s(X) s(X) 1,(X)

Symdlrom.e

Sylllldrrnnme nctor

§(X)

(so, s1, s2)

= 1+X = 1 + X + X2 = x + x2 = 1+X = X2 =X =1 2

(101) (111) (011) (110) (001) (010) (100)

We see that e6(X) = X 6 is the only error pattern with an error at location X 6 . When this error pattern occurs, the syndrome in the syndrome register will be (1 0 1) after the entire received polynomial r(X) has entered the syndrome register. The detection of this syndrome indicates that r6 is an erroneous digit and must be corrected. Suppose that the single error occurs at location Xi [i.e., e; (X) = Xi] for 0 _s: i < 6. After the entire received polynomial has been shifted into the syndrome register, the syndrome in the register will not be (101); however, after another 6 - i shifts, the contents in the syndrome register will be (101), and the next received digit to come out of the buffer register will be the erroneous digit. Therefore, only the syndrome (101) needs to be detected, and this can be accomplished with a single three-input AND gate. The complete decoding circuit is shown in Figure 5.9. Figure 5.10 illustrates the decoding process. Suppose that the codeword v = (1001011) [or v(X) = 1 + x 3 + x 5 + X 6 ] is transmitted and ll' = (1011011) [or r(X) = 1 + X 2 + X3 + X 5 + x 6 ] is received. A single error occurs at location X2 . When the entire received polynomial has been shifted into the syndrome and buffer registers, the syndrome register contains (001). In Figure 5.10, the contents in the syndrome register and the contents in the buffer register are recorded after each shift. Also, there is a pointer to indicate the error location after each shift. We see that after four more shifts the contents in the syndrome register are (101), and the erroneous digit '2 is the next digit to come out from the buffer register. The (7, 4) cyclic code considered in Example 5.9 is the same code considered in Example 3.9. Comparing the decoding circuit shown in Figure 3.9 with the decoding circuit shown in Figure 5.9, we see that the circuit shown in Figure 5.9 is simpler than the circuit shown in Figure 3.9. Thus, the cyclic structure does simplify the decoding circuit; however, the circuit shown in Figure 5.9 takes a longer time to decode a received vector because the decoding is carried out serially. fo general, there is a trade-off between speed and simplicity, as they cannot be achieved at the same time. The Meggitt decoder described decodes a received polynomial r(X) = ro + r1X + · · · + r11 _1x 11 - 1 from the highest-order received digit r 11 _1 to the lowest-order received digit ro. After decoding the received digit r;, both the buffer and syndrome

Section 55

f~--.-- +

t ·- .

of

Codes

i~!ll

Buffer register

~\~\~\~~

~Multiplexerf--1~~~\

v

I --

areJ

~-·--------~~_J___·--·--·

FIGURE 5.9: Decoding circuit for the (7, 4) cyclic code generated by i?;(X)

= 1 + X + X3 .

registers are shifted once to the right. The next received digit to be decoded is 1\-1. His possible to implement a Meggitt decoder to decode a received polynomial in the reverse order (i.e., to decode a received polynomial from the lowest-order received digit ro to the highest-order received digit r11 _ 1 ). After decoding the received digit r;, both the buffer and registers are shifted once to the left. The next received digit to be decoded is r;+l· The details of this decoding of a received polynomial in reverse order are left as an exercise. To decode a cyclic code, the received polynomial rr(X) may be shifted into the syndrome register from the right end for computing the syndrome. When ll'(X) has been shifted into the syndrome register, the register contains §(n-kl(X), which is the syndrome of ir(n-kl(X), the (n - k)th cyclic shift of ll'(X). If 1/ 11 -kl(X) corresponds to an error pattern e(X) with e11 _1 = 1, the highest-order digit r11 _1 of ir(X) is erroneous and must be corrected. In ir(n-kl(X), the digit r11 _ 1 is at the location xn-k-l_ When r 11 _1 is corrected, the error effect must be removed from §(n-kl(X). The new syndrome, denoted by §i"-k) (X), is the sum of §(n-kl(X) and the remainder p(X) resulting from dividing xn-k-l by the generator polynomial g(X). Because the degree of xn-k-l is less than the degree of !§(X),

Therefore, 5f1-kJ (X)

=

,,
+ xn-k-1,

160

Cyclic Codes

Chapter 5

Buffer register Pointer

i

Syndrome register Initial I o I o I i 1

Correction 0

1 o I i I i I o 1 1 1 i

r1

0

i 1st shift

\~~~~-0~

I i I O I

r1

I i I O 11

0

ri1iJ1joj

~

rl°l1J1l1loi

i 4th shift 1 1

joj1l1J1Jol

o

r1

o

I

o

0

I

1

~ 0

Error corrected 5th shift I o

~ 0

i 3rd shift I 1 I 1 I 1 I

~ 0

i 2nd shift I O I i I 1 I

~

0

0

0 0

6th shift

I

o

o

I

o 1

~

O I O I i I I

r Corrected word

7th shift

Io Io Io

0

0

0

0

1

FIGURE 5.10: Error-correction process of the circuit shown in Figure 5.9.

which indicates that the effect of an error at the location xn-l on the syndrome can be removed by feeding the error digit into the syndrome register from the right end through an EXCLUSIVE-OR gate, as shown in Figure 5.11. The decoding process of the decoder shown in Figure 5.11 is identical to the decoding process of the decoder shown in Figure 5.8.

Section 5.5

Decoding of Cyclic Codes

161

F~~~~~~~~~<~~~~~~

~r(_X)~-o-cC-Q;;~~~~~-B-uf-fe-r-re-g-is-te-r~~~~~~Received ~vector

<>

Syndrome register

-->

vector

+ /;

Syndrome_) modification 0

0

0

l_:'.rror pattern detection circuit

e;

FIG URE 5.11: General code decoder with received polynomial rr(X) shifted into the syndrome register from the right end.

IE}{AMIPLIE 5. 1lOJ

Again, we consider the decoding of the (7, 4) cyclic code generated by g(X) = 1+ X + . Suppose that the received polynomial rr(X) is shifted into the syndrome register from the right end. The seven single--error patterns and their corresponding are listed in Table 5.5. We see that only when e(X) = X 6 occurs, the syndrome is (001) after the entire n·(X) has been shifted into the syndrome register. If the single error occurs at the location with i -/=- 6, the syndrome in the register will not be TABLE 5.5: Error patterns and their syndromes with the

received polynomial rr(X) shifted into the syndrome register from the right end. §yrrni!llmmru<e wed(J])Jr

JErrrr(J])Jrll)l1J1tl1tem

e(X) e(X) = x 6 e(X) = X 5 e(X) = x 4 <e(X) = X 3 e(X) = X 2 e(X) = X e(X) = x 0

(so, s1, sz)

ri 3 >(X) = x 2 §( 3l(X) = X §< 3l(X) = 1 §< 3l(X) = 1 + X 2 ,Pl(x) = 1 + x + x2 §<3 l(X) = X + X 2 §< 3l(X) = 1 + X

(001) (010) (100)

(101) (111) (011)

(110)

162

Chapter 5

Cyclic Codes Buffer register

[(X) Input

FIGURE 5.12:

Multiplexer

----l"'l

+

['(X) Output

Decoding circuit for the (7, 4) cyclic code generated by g(X)

1+x+x3 . (001) after the entire received polynomial ll'(X) has been shifted into the syndrome register; however, after another 6 - i shifts, the syndrome register will contain (001). Based on this result, we obtain another decoding circuit for the (7, 4) cyclic code generated by g(X) = 1 + X + X3 , as shown in Figure 5.12. We see that the circuit shown in Figure 5.9 and the circuit shown in Figure 5.12 have the same complexity.

5.6

CYCLIC HAMMING CODES The Hamming codes presented in Section 4.1 can be put in cyclic form. A cyclic Hamming code of length 2111 - 1 with m ~ 3 is generated by a primitive polynomial p(X) of degree m. In the following we show that the cyclic code defined previously is indeed a Hamming code, by examining its parity-check matrix in systematic form. The method presented in Section 5.2 is used to form the parity-check matrix. Dividing xm+i by the generator polynomial p(X) for O ::S i < 2"' - m - 1, we obtain xm+i

= a; (X)p(X) + b; (X),

(5.28)

where the remainder b; (X) is of the form b; (X)

= b;o +

bilX + · · · + b;, 111 -1X 111 - 1 .

Because X is not a factor of the primitive polynomial p(X), xm+i and p(X) must be relatively prime. As a result, b; (X) -I 0. Moreover, b; (X) consists of at least two

Section 5.6

Cyclic Hamming Codes

H:i3

terms. Suppose that !bl; (X) has only one term. say x.i with O :s: j < m. H follows from (5.28) that x 111 +; = 21; CX)n:»(X) + xi. Rearranging the preceding

we have

xi (X 111 +; -i

+ 1) =

21; (X)lJll(X).

the foregoing that RJJ(X) divides x 111 +i-i + l; this is impossible since m + i - j < 2111 -1, and lJll(X) is a polynomial of degree m. [Recall that the smallest n such that lJll(X) divides X 11 + 1 is 2111 -1.] Therefore. for O :s: i < 2111 - m -1, the remainder !bi; (X) contains atleast two terms. Next we show that for i f:: j. !bi; (X) f:: fmj(X). From (5.28) we obtain

+ x +; = 21; (X)lJll(X), !bii (X) + xm+J = (X)jp)(X). lbi; (X)

111

that lbi; (X) = (X). Assuming that i < j and combining the equations above we obtain the following relation: xm+i (xi-i

two

+ 1) = [2l;(X) + 2lj(X)][PJ(X).

-i + 1. but this is since i f:: j This equation implies that ]Jll(X) divides and j - i < 2m - 1. Therefore. lbJ;(X) f:: lbij(X). Let IHI = [Il111 (Ql] be the parity-check matrix form) of the cyclic code generated by fill(X). where Il 111 is an nz x m matrix and (Ql is an m x (2m - m - 1) matrix. Let lbJ; = (b;o. b;i. · · · . b;. 111 -1) be them-tuple ,._,,,.r"''J::'·s•.r:.'""''J-o to lbJ; (X). H follows from (5.17) that the matrix (Ql has the 2111 - m 1 lbJ; 's with 0 :s: i < 2111 - 111 - 1 as all its columns. H follows from the preceding analysis that no two columns of (Ql are alike, and each column has at least two 1's. Therefore, the matrix IHI is indeed a parity-check matrix of a Hamming and 1[j)(X) generates this code. The polynomial ]Jll(X) = 1 + X + is a pnm1tlve the (7. 4) cyclic code generated by ]Jll(X) = 1 + X + X 3 is a I-!amming code. A list of primitive polynomials with degree:=::: 3 is given in Table 2.7. Cyclic Hamming codes can easily be decoded. To devise the decoding circuit. all we need to know is how to decode the first received digit. AH the other received digits will be decoded in the same manner and with the same circuitry. Suppose that a - 2 • of the received vector single error has occurred at the highest-order position. 2 ir(X) [i.e .. the error polynomial is e(X) = - ]. Suppose that ir(X) is shifted into the syndrome register from the right end. After the entire ir(X) has entered the register, the syndrome in the register is equal to the remainder resulting from dividing X 111 • x 2"' - 2 ( the error polynomial preshifted m times) by the generator polynomial [ll)(X). Because ]PJ(X) divides x2"'-l + 1. the syndrome is of the following form:

§(X)

= x111-1.

Therefore. if a single error occurs at the highest-order location of ir(X). the resultant syndrome is (0, 0, · · · , 0. 1). U a single error occurs at any other location of

164

Chapter 5

Cyclic Codes

Output Input ~ L~~,___B_u_ff_e_r_re_g1_·st_e_rr_(_X) _ _,___ _ _ _ _ _ _......, +

~~~~~~~~-~-+

AND

FIGURE 5.13:

Decoder for a cyclic Hamming code.

Ir(X), the resultant syndrome will be different from (0, 0, · · · , 0, 1). Based on this result only a single m-input AND gate is needed to detect the syndrome pattern (0, 0, · · · , 0, 1). The inputs to this AND gate are sb, s{, · · · , s;11 _ 2 and s111 -1, wheres; is a syndrome digit and denotes its complement. A complete decoding circuit for a cyclic Hamming code is shown in Figure 5.13. The decoding operation is described in the following steps:

s;

Step 1. The syndrome is obtained by shifting the entire received vector into the syndrome register. At the same time, the received vector is stored in the buffer register. H the syndrome is zero, the decoder assumes that no error has occurred, and no correction is necessary. If the syndrome is not zero, the decoder assumes that a single error has occurred. Step 2. The received word is read out of the buffer register digit by digit. As each digit is read out of the buffer register the syndrome register is shifted cyclically once. As soon as the syndrome in the register is (0, 0, 0, ... , 0, 1) the next digit to come out of the buffer is the erroneous digit, and the output of the m-input AND gate is 1. Step 3. The erroneous digit is read out of the buffer register and is corrected by the output of the 111-input AND gate. The correction is accomplished by an EXCLUSIVE-OR gate. Step 4!. The syndrome register is reset to zero after the entire received vector is read out of the buffer. The cyclic Hamming code presented here can be modified to correct any single error and simultaneously to detect any combination of double errors. Let

Section 5.6

Cyclic Hamming Codes

1155

!Ii(X) = (X + l)[pi(X), where JPl(X) is a m. Because both X + 1 and JPl(X) divide x 2"'-l + 1 and since are !Ii(X) must also divide x2"' -l + 1. A single-error-correcting and double-error-detecting ,m"'""'"'" code of 2111 - 1 is ~(X) = + l)JPl(X). The code has m + 1 parity-check digits. We show next that the minimum distance of this code is 4. For Hamming code by C1 and denote the ~()() = (X + l)JPl(X) by C2. In C2 consists of the codewords of C1 as aH its because any code in C1 does not have X + 1 as a factor. an odd-weight code of C1 is not divisible by !Ii(X) = + l)JPl(X), and it is not a code polynomial of an code polynomial of C1 has X + 1 as a factor. Therefore, it is divisible by ~(X) = (X + l))Pl(X), and it is also a code polynomial in C2. As a result, the minimum weight of C2 is at least 4. Next, we show that the minimum weight of C2 is 4. Let i, j, and k be three distinct integers less than 2 111 - l such that Xi + x.i + is not divisible JPl(X). Such do exist. For example, we first choose i and j. + x.i by JPl(X), we obtain

+

= m(X)LOJ(X) + lbJ(X),

where lbJ(X) is the remainder with degree m - 1 or less. Because Xi + x.i is not divisible !)ll(X), lbJ(X) cfa 0. Now, we choose an integer k such that when x" is divided by JPl(X), the remainder is not to lbi(X). Therefore, Xi + + is not divisible by [p>(X). this polynomial JI»(X), we have +x1c =

Next, we choose a JPl(X), the remainder is ocTI(X); that

+ cdl(X).

(5.29)

/ less than 2.111 -1 such that when X 1 is divided

= if(X)JPl(X) + cdl(X).

(5.30)

The integer l cannot be equal to any of the three integers i. j, or k. Suppose that l = i. From (5.29) and (5.30) we would obtain x.i

+

= [(('.(X) + if(X)]JPl(X).

This result implies that JPl(X) divides xk--.f + 1 (assuming that j < k), which is since k -- j < 2"' - L and JPl(X) is a polynomial. Therefore, l cfa i. Similarly, we can show that l cfa j and I cfa k. Using this fact and combining (5.29) and (5.30), we obtain xi+ x.i + x1c + x 1 = [(('.(X) + if(X)]JPl(X).

Because X + 1 is a factor of Xi + xk + X1 and it is not a factor of)Pl(X), (('.(X) +if(X) must be divisible by X + 1. As a result, Xi + x.i + Xk + X 1 is divisible by ~(X) = (X + l)JPl(X). Therefore, it is a code polynomial in the code generated by ~(X). It has weight 4. This proves that the cyclic code C2 generated by !Ii(X) = (X + l)]PJ(X) has a minimum weight ( or distance) of 4. Hence, it is capable of correcting any single error and simultaneously detecting any combination of double errors.

166

Chapter 5

Cyclic Codes

Because the distance-4 Hamming code C2 of length 2111 - 1 consists of the even-weight codewords of the distance-3 Hamming code C1 of length 2111 - 1 as its codewords, the weight enumerator A2 (z) for C2 can be determined from the weight enumerator A1(z) for C1. A2(z) consists of only the even power terms of A1(z). Therefore, (5.31) (see Problem 5.8). Since A1 (Z) is known and is given by (4.1), A2(z) can be determined from (4.1) and (5.31): A ( ) 2

1

z = 2(n + 1)

{(1 + )11 + (1

z

z) 11 + 2n(l - z2)(n-ll/2},

(5.32)

where n = 2111 -1. The dual of a distance-4 cyclic Hamming code is a (2111 -1, m + 1) cyclic code that has the following weight distribution:

Therefore, the weight enumerator for the dual of a distance-4 cyclic Hamming code is B2(z) = 1 + (2111 -1)2211,-1_1 + (2111 -1)2211,-1 +z2111-1. (5.33) If a distance-4 cyclic Hamming code is used for error detection on a BSC, its probability of an undetected error, Pu (E), can be computed from (3.33) and (5.32) or from (3.36) and (5.33). Computing Pu(£) from (3.36) and (5.33), we obtain the following expression: 11 Pu(E)=T(m+l){l + 2(21/1 -1)(1 - p)(l - 2p) 2m-l_ 1+(1- 2p)2' - 1} - (1- p)2'"- 1 .

(5.34) Again, from (5.34), we can show that the probability of an undetected error for the distance-4 cyclic Hamming codes satisfies the upper bound 2-Cn-k) = 2-Cm+ll (see Problem 5.21). Distance-3 and distance-4 cyclic Hamming codes are often used in communication systems for error detection. 5.7

ERROR-TRAPPING DECODING

In principle, the general decoding method of Meggitt's applies to any cyclic code, but refinements are necessary for practical implementation. If we put some restrictions on the error patterns that we intend to correct, the Meggitt decoder can be practically implemented. Consider an (n. k) cyclic code with generator polynomial g(X). Suppose that a code polynomial v(X) is transmitted and is corrupted by an error pattern e(X). Then, the received polynomial is r(X) = v(X) + e(X). We showed in Section 5.4 that the syndrome s(X) computed from r(X) is equal to the remainder resulting from dividing the error pattern e(X) by the generator g(X) [i.e., e(X) = a(X)g(X) + s(X) ]. Suppose that errors are confined to then k high-order . . Xk , xk+l , · · · , xn-1 o f r (X) ['1.e., e (X) -- ek X" +ek+l xk+l +· · ·+en-1 xn-1] . positions, If r(X) is cyclically shifted n - k times, the errors will be confined ton - k low-order parity positions, x 0. X 1, · · · , xn-k-l of r< 11 -kl(X). The corresponding error pattern is then e (11-k)(X) _- e1c + ek+l X + · · · + e11-l xn-k-1 .

Section 5.7

Error-Trapping Decoding

Ui7

Because the syndrome §(n-kl (X) of 1rCn-kJ (X) is to the remainder resulting from e(n-k) (X) iJ:(X) and since the degree of e
ri

11

-k\X)

§
= e<11 -kl(X) = ek + e1c+1X + · · · + e

by

Xk,

11

_1X 11 -k-l.

we have

Xk§Cn-k\X) = e(X) vk = ek,,,

+ ek+l.t'vk+l + · · · + en-lAvn-1 .

This result says that if errors are confined to the n - k high-order positions of the received the error e(X) is identical to Xk §(X); however, if the errors are confined ton -k consecutive positions (including end-around) other than then -k high-order positions of Jr(X), after the entire 1r(X) has been shifted into the register, the register must be shifted a certain number of times before its contents are identical to the error digits. This shifting of the syndrome register until its contents are identical to the error is called error trapping [14]. H errors are confined ton - k consecutive positions of Il'(X) and if we can detect when the errors are trapped in the syndrome register, errors can be corrected simply adding the contents of the syndrome register to the received digits at the 11 - k proper positions. Suppose that at-error-correcting cyclic code is used. To detect the event that the errors are trapped in the syndrome register, we may simply test the weight of the syndrome after each shift of the syndrome register. As soon as the weight of the syndrome becomes t or less, we assume that errors are trapped in the syndrome register. If the number of errors in 1r(X) is t or less and if they are confined to 11 - k consecutive positions, the errors are trapped in the syndrome register only when the weight of the syndrome in the register becomes t or less. This result can be shown as follows. An error pattern e(X) with t or fewer errors that are confined to 11 - k consecutive positions must be of the form e(X) = XilE,(X), where lffi(X) has t or fewer terms and has degree n - k - 1 or less. (For the end-around case, the same form would be obtained after a certain number of cyclic shifts of e(X).) Dividing e(X) by the generator polynomial g(X), we have xi!ffi(X)

= m(X)g(X) + §(X),

168

Chapter 5

Cyclic Codes

where s(X) is the syndrome of XiB(X). Because s(X) + XlJR(X) is a multiple of g(X), it is a code polynomial. The syndrome s(X) cannot have weight t or less unless s(X) = xiB(X). Suppose that the weight of s(X) is t or less, and s(X) # X iB(X). Then, s(X) + XlB(X) is a nonzero code polynomial with weight less than 2t + 1. This is impossible, since a t-error-correcting code must have a minimum weight of at least 2t + 1. Therefore, we conclude that the errors are trapped in the syndrome register only when the weight of the syndrome becomes t or less. Based on the error-trapping concept and the test just described, an errortrapping decoder can be implemented as shown in Figure 5.14. The decoding operation can be described in the following steps: Step 1. The received polynomial r(X) is shifted into the buffer and syndrome registers simultaneously with gates 1 and 3 turned on and all the other gates turned off. Because we are interested only in the recovery of the k information digits, the buffer register has to store only the k received information digits. Step 2. As soon as the entire r(X) has been shifted into the syndrome register, the weight of the syndrome in the register is tested by an (n - k)-input threshold gate whose output is 1 when t or fewer of its inputs are 1; otherwise, it is zero. a. If the weight of the syndrome is t or less, the syndrome digits in the syndrome register are identical to the error digits at then - k highorder positions x", xk+ 1 , ... , xn-l of r(X). Now, gates 2 and 4 are turned on and the other gates are turned off. The received vector is read out of the buffer register one digit at a time and is corrected by the error digits shifted out from the syndrome register. r(X) k-bit buffer register

Input

Output

Syndrome register

1------------'

... Threshold gate

FIGURE 5.14:

Error-trapping decoder.

Section 5.7

Error-Trapping Decoding

116iSI

lhio H the weight of the syndrome is greater than t, the errors are not confined to the n - k positions of r(X), and they have

not been trapped in the syndrome register. Go to step 3. §ttll::jpJ :t The register is cyclically shifted once with gate 3 turned on and other gates turned off. The of the new is tested. If it is t or less, the errors are confined to the locations xk- 1, , ... , x 11 - 2oh(X), and the contents in the syndrome register are identical to the errors at these locations. Because the first received digit r11 _ 1 is error-free, it is read out of the buffer with 2 turned on. As soon as r 11 _1 has been read out, gate 4 is turned on and gate 3 is turned off. The contents in the register are shifted out and are used to correct the next n - k received digits to come out from the buffer register. (b) H the of the syndrome is greater than t, the syndrome register is shifted once more with gate 3 turned on. §tt<ejp! 4lo The syndrome register is shifted until the of its contents to t or less. If the weight goes down to t or less at the end of the ith for 1 ::: i ::: k, the first i received r11 _;,r 11 _;+1, ··· ,r11 -1, in the buffer are error-free, and the contents in the syndrome register are identical to the errors at the locations xk-i, xk-i+l, ... , xn-i-l_ As soon as the i error-free received digits have been read out of the buffer register, the contents in the register are shifted out and are used to correct the next n - k received digit to come out from the buffer When the k received information digits have been read out of the buffer register and have been corrected, gate 2 is turned off. nonzero digits left in the syndrome register are errors in the parity of ir(X), and will be ignored. §ill::Jll> !!!lo H the weight of the syndrome never goes down to t or less by the time the syndrome register has been shifted k either an error pattern with errors confined to n - k consecutive end-around locations has occurred or an uncorrectable error pattern has occurred. The syndrome register keeps being shifted. Suppose that the weight of its contents becomes t or less at the end of k + I shifts with 1 ::: l ::: n - k. Then, errors are confined to then - k consecutive endaround locations, x 11 - 1, x 11 - 1+1, ... 'x 11 -1, x 0 , x 1 , ... 'xn-k-l-l of r(X). The l digits in the l leftmost stages of the register match the errors at the l high-order locations x 11 - 1, xn-l+l, ... 'xn-1 of rr(X). Because the errors at the n - k - l parity locations are not needed, the syndrome register is shifted n -k-l times with all the gates turned off. Now, the l errors at the locations x 11 - 1, x 11 - 1+1 , ... , xn-1 of ir(X) are contained in the l rightmost stages of the syndrome register. With gates 2 and 4 turned on and other gates turned off, the received digits in the buffer register are read out and are corrected by the corresponding error digits shifted out from the syndrome register. This completes the decoding operation.

170

Chapter 5

Cyclic Codes k-bit buffer register

Syndrome register

.----------'

Threshold gate

FIGURE 5.15: Another error-trapping decoder.

If the weight of the syndrome never drops to t or less by the time the syndrome register has been shifted a total of n times, either an uncorrectable error pattern has occurred or errors are not confined to n - k consecutive positions. In either case, errors are detected. Except when errors are confined to the n - k consecutive endaround positions of r(X), the received information digits can be read out of the buffer register, corrected, and delivered to the data sink after at most k cyclic shifts of the syndrome register. When an error pattern with errors confined to n - k consecutive end-around locations of r(X) occurs, a total of n cyclic shifts of the syndrome register is required before the received message can be read out of the buffer register for corrections. For large n and n - k, the number of correctable end-around error patterns becomes big, which causes an undesirably long decoding delay. It is possible to implement the error-trapping decoding in a different manner so that the error patterns with errors confined ton - k consecutive end-around locations can be corrected as fast as possible. This can be done by shifting the received vector ir(X) into the syndrome register from the left end, as shown in Figure 5.15. This variation is based on the following facts. If the errors are confined ton - k low-order parity positions x 0 , x1, · · · , xn-k-l of ir(X), then after the entire ir(X) has entered the syndrome register, the contents in the register are identical to the error digits at the locations x 0 , X 1 , · · · , xn-k-l of ir(X). Suppose that the errors are not confined to the n - k low-order positions of ir(X) but are confined to n - k consecutive locations (including the end-around case), say Xi, x;+1, ... , x
Step 1. Gates 1 and 3 are turned on and the other gates are turned off. The received vector r(X) is shifted into the syndrome register and simultaneously into the buffer register (only the k received information digits

Section 5.7

§tlie]lll

2"

§tlie]lll

2t

As soon as the entire rr(X) has been shifted into the the contents of the form the §(X) of r(X). The weight of the is tested. (a) If the the errors are confined to the (n - k) low-order xo, xi, ... , xn-k-l of the k received information in the buffer register are Gate 2 is then turned on, and the error-free information are read out of the buffer with gate 4 turned off. If the of the is than t, the 3 turned on and the register is then shifted once with other gates turned off. Go to 3. The of the new contents in the syndrome is tested. If the the errors are confined to the The leftmost register is identical to the error at the the other 11 - k - 1 in the register match the errors at x0 , oh(X). The output of the threshold gate turns gate 3 off and sets a clock to count from L The register is then shifted (in step with the clock) with gate 3 turned off As soon as the clock has counted ton - k, the contents of the syndrome register win be (0 0 · · · 0 The rightmost digit matches the error at xn-l of rr(X). The k received information digits are then read out of the buffer, and the first received information out from the register. The digit is corrected by the 1 decoding is thus completed. (b) Uthe of the contents in the syn·· drome register is greater than t, the syndrome register is shifted once again with gate 3 turned on and other gates turned off. Go to step 4. Step 3(b) repeats until the of the contents of the syndrome register drop to t or less. H the to t or less after the ith for 1 :S i :S n --k, the clock starts to count from i + 1. At the same time the syndrome is shifted with gate 3 turned off. As soon as the dock has counted to n - k, the i in the register match the errors in the first i received information digits in the buffer register. The other information are error-free. Gates 2 and 4 are then turned on. The received information digits are read out of the buffer for correction. If the weight of the contents of the syndrome register never drops to t or less by the time that the syndrome register has been shifted n - k times gate 3 turned on), gate 2 is then turned on, and the received information digits are read out of the buffer one at a time. At the same time the syndrome register is shifted with gate 3 turned on. As soon as the weight of the contents of the syndrome register drops to t or the contents match the errors in the next 11 - k digits to come out of the buffer. Gate 4 is then turned on, and the erroneous information digits are corrected the digits coming out from the syndrome register with gate 3 turned off. Gate 2 is turned off as soon as k information digits have been read out of the buffer. iuu~n,,~u

pUoOHAUH

§ttie]lll 4!,

§11:<e]lll

!!L

172

Chapter 5

Cyclic Codes

With the implementation of the error-trapping decoding just described, the received information digits can be read out of the buffer register after at most 11 - k shifts of the syndrome register. For large 11 - k, this implementation provides faster decoding than the decoder shown in Figure 5.14; however, when 11 - k is much smaller thank, the first implementation of error trapping is more advantageous in decoding speed than the one shown in Figure 5.15. The decoding of cyclic Hamming codes presented in Section 5.6 is actually an error-trapping decoding. The syndrome register is cyclically shifted until the single error is trapped in the rightmost stage of the register. Error-trapping decoding is most effective for decoding single-error-correcting block codes and burst-error-correcting codes (decoding of burst-error-correcting codes is discussed in Chapter 20). It is also effective for decoding some short double-error-correcting codes. When errortrapping decoding is applied to long and high-rate codes (small n - k) with large error-correcting capability, it becomes very ineffective, and much of the errorcorrecting capability is sacrificed. Several refinements of this simple decoding technique [12)-[19) have been devised to extend its application to multiple-errorcorrecting codes. One of the refinements is presented in the next section.

EXAMPLE 5.11 Consider the (15, 7) cyclic code generated by g(X) = 1 + x 4 + x 6 + x 7 + X 8 . This code has a minimum distance of dmin = 5, which will be proved in Chapter 6. Hence, the code is capable of correcting any combination of two or fewer errors over a block of 15 digits. Suppose that we decode this code with an error-trapping decoder.

xi"

xi x2

xi

x12

x11

X"

X'

x111

X"

x1 FIGURE 5.16: Ring arrangement of code digit positions.

Section 5.8

-~~I

Improved Error-Trapping Decoding

H3l

7-bit buffer register

1L _''-----------~---------'

TI1reshold gate w(s) ,s 2?

FIGURE 5.17: Error-trapping 6 7 8 4

1+ x

+x +x +x

decoder for the (15, 7) cyclic code generated

~(X)

=

.

Clearly, any single error is confined to n - k = 8 consecutive positions. Therefore, any single error can be trapped and corrected. Now, consider any double errors over a span of 15 digits. H we arrange the 15 digit positions x 0 to X 14 as a ring. as shown in Figure 5.16, we see that any double errors are confined to eight consecutive positions. Hence, any double errors can be trapped and corrected. An error-trapping decoder for the (15, 7) cyclic code generated by it(X) = 1 + X 4 + x 6 + X 7 + x 8 is shown in Figure 5.17.

The error-trapping decoding discussed in Section 5.7 can be to correct error patterns such that, for each error pattern, most errors are confined to 11 k consecutive positions, and fewer errors are outside the (n - k)-digit span. This improvement needs additional circuitry. The complexity of the additional circuitry depends on how many errors outside an (11 - k)-digit span are to be corrected. An improvement proposed by Kasami [17] is discussed here. The error pattern ie(X) = eo + e1X + e2X 2 + · · · + e11 _1 xn-l that corrupted the transmitted codeword can be divided into two parts:

+ eu,+···+en-k-1 xn-k-1 ie1 (X) = e11-kxn-k + · · · + e _1X 1,

iep (X) =eo

V

11

11

-

where ie 1 (X) contains the errors in the message section of the received vector, and iep (X) contains the errors in the parity section of the received vector. Dividing e1 (X) by the code generator polynomial it(X), we obtain ie1(X)

= qi(X)\g(X) + p(X),

(5.35)

174

Chapter 5

Cyclic Codes

where p(X) is the remainder with degree n - k - 1 or less. Adding ep(X) to both sides of (5.35), we obtain e(X) = ep(X)

+ e 1 (X)

= q(X)g(X)

+ p(X) + ep(X).

(5.36)

Because ep(X) has degree n - k - 1 or less, p(X) + ep(X) must be the remainder resulting from dividing the error pattern e(X) by the generator polynomial. Thus, p(X) + ep(X) is equal to the syndrome of the received vector r(X),

+ ep(X).

(5.37)

= s(X) + p(X).

(5.38)

§(X) = p(X) Rearranging (5.37), we have ep(X)

That is, if the error pattern e1 (X) in the message positions is known, the error pattern ep(X) in the parity positions can be found. Kasami's error-trapping decoding requires finding a set of polynomials [ 1 (X) J7=l of degree k - 1 or less such that, for any correctable error pattern e(X), there is one polynomial 1 (X) such that xn-k 1 (X) matches the message section of e(X) or the message section of a cyclic shift of e(X). The polynomials ef>/X)'s are called the covering polynomials. Let p 1(X) be the remainder resulting from dividing xn-k
+ p 1 (X)]::: t -

w[cp 1 (X)],

then x 11 -k4> 1(X) matches the error pattern in the message section of e(X), and s(X) + p 1(X) matches the error pattern in the parity section of e(X). Thus, e(X) = s(X)

+ pz(X) + xn-k
The correction is then performed by taking the modulo-2 sum r(X) + e(X). This step requires N (n - k)-input threshold gates to test the weights ofs(X) + p 1(X) for j = 1, 2, · · · , N. Step 4. If w[s(X) + p_/X)] > t - w[cp 1 (X)] for all j = 1, 2, · · ·, N, both syndrome and buffer registers are shifted cyclically once. Then, the new contents of the syndrome register, sC 1l(X), is the syndrome corresponding to eC 1l (X), which is obtained by shifting the error pattern e(X) cyclically one place to the right. Step 5. The weight ofsC 1l(X) + p 1 (X) is computed for j = 1, 2, · · ·, N. If, for some l,

The

Section 509

12)

Code

n'S

then x 11 -k4» 1(X) matches the errors in the message section of e< 1l(X), and §Cll (X) + p 1(X) matches the errors in the section of eCll (X)" Thus, the modulo-2 sum rrl 1l (X)+

The correction is then e< 1l(X)" If w[§C 1\X)

§!e]lll

if£o

+ p 1 (X)]

> t - w[«fli(X)]

for all j = 1, 2, " "" , N, both and buffer are shifted cyclically once again" The syndrome and buffer registers are continuously shifted until §Ul(X) (the syndrome after the ith is found such for some l,

Then, ieUl(X)

= §CiJ(X) + p 1(X) +

(X),

where eCil(X) is the ith cyclic shift of e(X)" Uthe weight w[§U1(X) + p 1 (X)] never drops to t (X)] or less for an j by the time that the syndrome and buffer registers have been shifted 11 - 1 times, an uncorrectable error pattern is detected" The complexity of a decoder that employs the decoding method described depends on N, the number of covering polynomials in (q;ii (X) 1 " The combinational logical circuitry consists of N (n -k)-input threshold gates" To find the set of covering polynomials {q;ii (X) =l for a specific code is not an easy Several methods for finding this set can be found in [17], [20], and [21 )a This improved error-trapping method is applicable to many double- and triple-error-correcting codes, however, it is still applicable only to relatively short and low-rate codes" When the code length n and error-correcting t become large, the number of threshold gates required in the error-detecting logical becomes very large and impracticaL Other variations of error-trapping decoding can be found in [15], and [18]"

}f=

}f

5.9

THIE {23, 172) GOlAV (OIDJIE

As pointed out in Section 4"6, the (23, 12) Golay code [22] with a minimum distance of 7 is the only known multiple-error-correcting binary perfect code" This code can be put in cyclic form, and hence it can be encoded and decoded based on its structure" It is generated either by g1 (X)

= 1+ X2 +

orby g2(X)

= 1+ x

+xs+x6+x10+x11

+ x5+

Both g1 (X) and g2 (X) are factors of X 23

x6 +

x7+

x9 +

1 and X 23

x 11

0

+ + 1 = (1 + X)g1 (X)g2 (X)o The encoding can be accomplished by an 11-stage shift register with feedback

116

Chapter 5

Cyclic Codes

connections according to either g1 (X) or g2 (X). If the simple error-trapping scheme described in Section 5.7 is used for decoding this code, some of the double-error patterns and many of the triple-error patterns cannot be trapped. For example, consider the double-error patterns e(X) = X11 + X22 . The two errors are never confined to n - k = 11 consecutive positions, no matter how many times e(X) is cyclically shifted. Therefore, they can never be trapped in the syndrome register and cannot be corrected. We can also readily see that the triple-error pattern e(X) = X 5 + X11 + x 22 cannot be trapped. Therefore, if the simple error-trapping scheme for decoding the Golay code is used, some of its error-correcting capability will be lost; however, the decoding circuitry is simple. There are several practical ways to decode the (23, 12) Golay code up to its error-correcting capability t = 3. Two of the best are discussed in this section. Both are refined error-trapping schemes. 5.9.1

Kasami Decoder [H]

The Golay code can easily be decoded by Kasami's error-trapping technique. The set of polynomials {(/) i (X) 1 is chosen as follows:

}f=

(/) 3 (X) =

X 6.

Let g1 (X) = 1 + X 2 + X4 + x 5 + x 6 + x 10 + X 11 be the generator polynomial. Dividing X 11 q,1 (X) by g1 (X) for j = 1, 2, 3, we obtain the following remainders: P1(X) = 0,

pz(X) = p 3 (X) =

x + x2 + x 5 + x6 + x8 + x 9, Xp 2 (X) = x 2 + x 3 + x 6 + x 7 + x 9 + x 10 .

A decoder based on Kasami's error-trapping scheme is shown in Figure 5.18. The received vector is shifted into the syndrome register from the rightmost stage; this is equivalent to preshifting the received vector 11 times cyclically. After the entire received vector has entered the syndrome register, the syndrome in the register corresponds to r< 11 >(X), which is the eleventh cyclic shift of r(X). In this case, if the errors are confined to the first 11 high-order positions x 12 , x 13 , ... , x 22 of r(X), the syndrome matches the errors in those positions. The error-correction procedure of this decoder is as follows: Step Jl.. Gates 1, 3, and 5 are turned on; gates 2 and 4 are turned off. The received vector r(X) is read into the syndrome register and simultaneously into the buffer register. The syndrome s(X) =so+ s 1 X + · · · + s10X 10 is formed and is read into three threshold gates. Step 2. Gates 1, 4, and 5 are turned off; gates 2 and 3 are turned on. The syndrome is tested for correctable error patterns as follows: ai. If the weight w[s(X)] :S 3, all the errors are confined to the 11 high-order positions ofr(X), and s(X) matches the errors. Thus, the erroneous symbols are the next 11 digits to come out of the buffer register. The output of the threshold gate To turns gate 4 on and

Section 5.9

Test for error pattern

The (23, 12) Golay Code

17/7/

(TI1reshold gates)

T0 = 1, if and only if I,)~ 0 s; ~ 3 T 1 = 1, if and only if s 0 + s 1 + s\ + s 3 + s4 + s~ + si, + s 7 + s~ + s:J + s 10 ~ 2 To= 1, if and only if s 0 + s 1 + s2 + s; + s 4 + s, + si, + s 7+ s~ + s:J + s 10 ~ 2

1--@l<-

-.-e-rGsl-· ---~

Clock

+">IColC1IC2lql> E : tttt i Counter C L _________

FIGURE 5.18:

a

An error-trapping decoder for the

12) Golay code.

gate 3 off. Digits are read out one at a time from the buffer register. The digit coming out of the syndrome register is added (modulo-2) to the digit coming out of the buffer. This corrects the errors. !bi, If w[§(X)] > 3, the weight of §(X) + p 2 (X) is tested. H w[§(X) + Pz(X)] S 2, then §(X) + pz(X) =so+ s~ X + s2X 2 + s3X 3 + s4X 4 + s~X 5 + s~X 6 + s7 X 7 + s8X 8 + s9X 9 + s10X 10 is identical to the error

178

Chapter 5

Cyclic Codes

pattern in the 11 high-order positions of the received word, and a single error occurs at location X5 , where is the complement of s;. Gate 4 is turned on, and gate 3 is turned off. The counter C starts to count from 2. At the same time, the syndrome register is shifted without feedback. The output Q, which is 1 when and only when C counts 3 and 4, is fed into the syndrome register to form the error pattern s(X) + p 2(X). When the counter C counts 8, its output E is L and the leftmost stage of the syndrome register is set to 1. This 1 is used for correcting the error at location X 5 in the received vector r(X). The digits coming out of the buffer are then corrected by the digits coming out of the syndrome register. re. If w[ s(X)] > 3 and w[s(X) + p2 (X)] > 2, the weight of s(X) + p3 (X) is tested. If w[s(X) + p 3 (X)] ::; 2, then s(X) + p 3 (X) =so+ s1X + s~X 2 + s~X 3 + s4X 4 + s5 X 5 + s~X 6 + s~X 7 + ssX 8 + s~X 9 + s~ 0 X 10 is identical to the error pattern in the 11 high-order positions of the received word and a single error occurs at positions X 6 . The correction is the same as step (b ), except that counter C starts to count from 3. If w[s(X)] > 3, w[s(X) + p 2 (X)] > 2, and w[s(X) + p 3 (X)] > 2, then the decoder moves to step 3.

s;

§tep 3. Both the syndrome and buffer registers are cyclically shifted once with gates 1. 4, and 5 turned off and gates 2 and 3 turned on. The new contents of the syndrome register are s< 1l (X). Step 2 is then repeated. Step 4. The decoding operation is completed as soon as the buffer register has been cyclically shifted 46 times. Gate 5 is then turned on and the vector in the buffer is shifted out to the data sink. If there are three or fewer errors in the received vector, the vector in the buffer at the end of decoding will be the transmitted codeword. If there are more than three errors in the received vector, the vector in the buffer at the end of decoding will not be the transmitted codeword. 5.9.2

Systematic Search Decoder [23]

This decoding method is based on the fact that every pattern of three or fewer errors in a block of 23 digits can be cyclically shifted so that at most one of the errors lies outside a specified 11-digit section of the word. The decoding procedure is as follows: Step 1. Compute the syndrome from the received vector. Step 2. Shift the syndrome and the received vector 23 times, checking whether the weight of the syndrome ever falls to 3 or less. If it does, the syndrome with weight 3 or less matches the error pattern and correction can be made. Step 3. If it does not, the first received information digit is inverted and step 2 is repeated, checking for a syndrome of weight of 2 or less. If one is found, the first received information digit was incorrect and the other two errors are specified by the syndrome. This completes the decoding.

Section 5. 10

Shortened Cyclic Codes

H'.91

§tlieJPl 4lo H no syndrome of weight 2 or less is found in step 3, the first

information digit was originally correct. In this case, this bit must be reinverted. §tlieJPl !5o Repeat step 3 by inverting the second, · · · , and twelfth information digits. Because not all the errors are in the parity-check section, an error must be corrected in this manner. In every pattern of three or fewer errors, there is at least one error that if corrected, will leave the remaining error or errors within 11 successive positions. When the digit corresponding to this error is inverted, the remaining errors are corrected as in ordinary error trapping. Compared with the Kasami decoder, the systematic search decoder has the advantage that only one weight-sensing (threshold) gate is required; however, it has the disadvantage that the clock and timing circuitry is more complex than that of the Kasami decoder, since 12 different digits must be inverted sequentially. Also, the Kasami decoder operates faster than the systematic search decoder. This systematic search technique can be generalized for decoding other multiple-error-correcting cyclic codes. The weight enumerator for the (23, 12) Golay code is A(Z)

= 1 + 253z 7 + 506z 8 + 1288z 11 + 1288z 12 + 506z 15 + 253z 16 + z23 .

H this code is used for error detection on a BSC, its probability of an undetected error (£) can be computed from (3.19). Moreover, P,, (E) satisfies the upper bound 2- 11 [Le., Pu(E) .:s 2- 11 ] [24]. Therefore, the (23, 12) Golay code is a good error-detecting code.

fa system design, if a code of suitable natural length or suitable number of infor-

mation digits cannot be found, it may be desirable to shorten a code to meet the requirements. A technique for shortening a cyclic code is presented in this section. This technique leads to simple implementation of the encoding and decoding for the shortened code. Given an (11, k) cyclic code C consider the set of codewords for which the l leading high-order information digits are identical to zero. There are 2k-l such codewords, and they form a linear subcode of C. If the l zero information digits are deleted from each of these codewords, we obtain a set of 2k-l vectors of length n - L These 2"- 1 shortened vectors form an (n - l, k - l) linear code. This code is called a shortened cyclic code (or polynomial code), and it is not cyclic. A shortened cyclic code has at least the same error-correcting capability as the code from which it is derived. The encoding and decoding for a shortened cyclic code can be accomplished by the same circuits as those employed by the original cyclic code. This is so because the deleted/ leading-zero information digits do not affect the parity-check and syndrome computations; however, in decoding the shortened cyclic code after the entire received vector has been shifted into the syndrome register, the syndrome register must be cyclically shifted l times to generate the proper syndrome for decoding the first received digit r11 _ 1_1. For large!, these extra l shifts of the syndrome register

um

Chapter 5

Cyclic Codes

cause undesirable decoding delay; they can be eliminated by modifying either the connections of the syndrome register or the error-pattern detection circuit. Let r(X) = ro + r1X + · · ·+ r 11 _1-1Xn-l-l be the received polynomial. Suppose that r(X) is shifted into the syndrome register from the right end. If the decoding circuit for the original cyclic code is used for decoding the shortened code, the proper syndrome for decoding the received digit r11 _ 1_1 is equal to the remainder resulting from dividing xn-k+1r(X) by the generator polynomial g(X). Because shifting r(X) into the syndrome register from the right end is equivalent to premultiplying r(X) by xn-k, the syndrome register must be cyclically shifted another l times after the entire r(X) has been shifted into the register. Now, we want to show how these extra l shifts can be eliminated by modifying the connections of the syndrome register. Dividing xn-k+1r(X) by g(X), we obtain (5.39)

where §(n-k+l) (X) is the remainder and the syndrome for decoding the received digit rn-1--1· Next, we divide xn-k+l by g(X). Let p(X) =Po+ p1X + · · · + Pn-1c-1Xn-k-l be the remainder resulting from this division. Then, we have the following relation: p(X)

=

xn-k+t

+ a2(X)g(X).

(5.40)

Multiplying both sides of (5.40) by ir(X) and using the equality of (5.39), we obtain the following relation between p(X)r(X) and §Cn-k+I) (X): p(X)r(X)

= [lll1 (X) + 212 (X)r(X) ]g(X) + §(s-k+t) (X).

(5.41)

The foregoing equality suggests that we can obtain the syndrome §(n-k+l) (X) by multiplying r(X) by p(X) and dividing the product p(X)r(X) by g(X). Computing §(n-k+l) (X) in this way, we can avoid the extra l shifts of the syndrome register. Simultaneously multiplying r(X) by p(X) and dividing p(X)r(X) by g(X) can be accomplished by the circuit shown in Figure 5.19. As soon as the received polynomial r(X) has been shifted into the register, the contents in the register form the syndrome ,,Cn-k+t) (X), and the first received digit is ready to be decoded.

Input r(X)

FIGURE 5.19: Circuit for multiplying r(X) by p(X) =Po+ P1X

and dividing p(X)r(X) by g(X)

= 1 + g1 X + ... + x

11

-k.

+ · · · + Pn-k-lxn-k-l

Section 5.10

q>~-->\ K(X)

~-'

l~ -

[

Shortened Cyclic Codes

31-bit buffer register

'HH

L_

~

{(}~

I

~~

Gate

-0

°[=[~{JL~ !.

V

0

0

0 0

AND

t FIGURE 5.20: Decoding circuit for the (31, 26) cyclic Hamming code generated 11:(X)=l+x2 +x 5 .

IEJ{AMf'LIE 5. U

For m = 5, there exists a (31, 26) cyclic Hamming code generated by 11:(X) 1 + X 2 + X5 . Suppose that it is shortened by three digits. The resultant shortened code is a (28, 23) linear code. The decoding circuit for the (31, 26) cyclic code is shown in Figure 5.20. This circuit can be used to decode the (28, 23) shortened code. To eliminate the extra shifts of the syndrome register, we need to modify the connections of the syndrome register. First, we need to determine the polynomial p(X). Dividing xn-k+ 3 by 11;(X) = 1 + X 2 + X 5 , we have

x3 + xs +

1

+ 11 xs xs+xs+x3 xs +x3

x5

+ x2 + 1 x 3 + x2 + 1

and p(X) = 1 + X2 + X3 . The modified decoding circuit for the (28, 23) shortened code is shown in Figure 5.21. The extra l shifts of the syndrome register for decoding the shortened cyclic code can also be avoided by modifying the error-pattern detection circuit of the decoder for the original cyclic code. The error-pattern detection circuit is redesigned to check whether the syndrome in the syndrome register corresponds to a correctable error pattern e(X)

182

Chapter 5

Cyclic Codes r'(X)

Input r(X)

Gate

f---l~H-

28-bit buffer register .(X) --- -~

+

Gate

FIGURE 5.21: Decoding circuit for the (28, 23) shortened cyclic code generated by g(X)=l+x 2 +x 5 .

with an error at position xn-Z-l (i.e., e11 _1-1 = 1). When the received digit rn-1-l is corrected, the effect of the error digit e 11 _1-1 on the syndrome should be removed. Suppose that the received vector is shifted into the syndrome register from the right end. Let p(X) = Po+ P1X + · · · + Pn-k-1X 11 -k-l be the remainder resulting from dividing xn-l-l . xn-k = x 2n-ic-t-l by the generator polynomial g(X). Then, the effect of the error digit e11 _ 1_1 on the syndrome is removed by adding p(X) to the syndrome in the syndrome register. EXAM/Pl.IE 5.13

Consider the (28, 23) shortened cyclic code obtained by deleting three digits from the (31, 26) cyclic Hamming code generated by g(X) = 1 + X 2 + X 5 . Suppose that in decoding this code the received vector is shifted into the syndrome register from the right end. If a single error occurs at the position x 27 [or e(X) = X 27 ], the syndrome corresponding to this error pattern is the remainder resulting from dividing X 5 e(X) = x 32 by g(X) = 1 + x 2 + x 5 . The resultant syndrome is (01000). Thus, in decoding the (28, 23) shortened Hamming code, the error-pattern detection circuit may be designed to check whether the syndrome in the syndrome register is (0100 0). Doing so avoids the extra three shifts of the syndrome register. The resultant decoding circuit with syndrome resetting is shown in Figure 5.22. Shortened cyclic codes for error detection in conjunction with ARQ protocols are widely used for error control, particularly in computer communications. In these applications they are often called cyclic redundancy check (CRC) codes. A CRC code is, in general, generated by either a primitive polynomial p(X) or a polynomial g(X) = (X + l)p(X). A number of CRC codes have become international standards for error detection in various contexts. A few standard CRC codes follow:

Section 5.10

Shortened Cyclic Codes

-==ni 0

0

Ul3i

V

0

AND ~--!>I~

FIGURE 5.22: Another decoding circuit for the (28. 23) shortened Hamming code generated by ~(X) = 1 + X 2 + x5 .

CCITT X-25 (Consultative Committee for International Telegraphy and Telephony, Recommendation X-25)

g(X) =ex+ l)(X 15 + x 14 + x 13 + x 12 + x 4 + x 3 + =X16+x12+xs+1.

+ x + 1)

ANSI (American National Standards Institute)

g(X) = (X + l)(Xl5 + X + 1) = x16 + x15 + x2 + 1. IBM-SD LC (IBM Synchronous Data Link Control)

g(X) =ex+ 1)2cx14 + xB + x12 + xio + xs + x6 + xs + x4 + x3 + x + 1) = x16 +xis+ xB + x1 + x4 + x2 + x + 1. !EC TC57

g(X) =ex+ 1) 2(x 14 + x 10 + x 9+ x 8 + x 5 + x 3 + x 2+ x + l) = x16 + x14 + x11 + xs + x6 + xs + x4 + 1, IEEE Standard 802.3

g(X) = x32 + x26 + x23 + x22 + x16 + x12 + xn + xio + xs + x1 + xs + x4 + x2 + x + 1.

184

5.11

Chapter 5

Cyclic Codes

CYCUC PRODUCT CODES

The obvious way to implement a product code is to set up the code array and operate on rows and then on columns (or columns and then rows) in encoding and decoding, but there is an alternative that can be extremely attractive. In many cases the product code of cyclic codes is cyclic, and cyclic code implementation is much simpler. If the component codes C1 and C2 are cyclic, and if their lengths, 111 and 112, are relatively prime, the product code C1 x C2 is cyclic if the code digits of a code array are transmitted in a proper order [3, 25, 26]. We start with the upper right corner and move down and to the left on a 45° diagonal, as shown in Figure 5.23. When we reach the end of a column, we move to the top of the next column. When we reach the end of a row, we move to the rightmost digit of the next row. Because 111 and n2 are relatively prime, there exists a pair of integers a and b such that a111 + b112 = 1. Let g1 (X) and h1 (X) be the generator and parity polynomials of the (111, k1) cyclic code C1, and let g2(X) and h2(X) be the generator and parity polynomials of the (112, k2) cyclic code C2. Then, it is possible to show [25, 26] that the generator polynomial g(X) of the cyclic product code of C1 and C2 is the greatest common divisor (GCD) of x 11 1112 - 1 and g 1 (Xb112 )g2(Xa 11 1 ); that is, (5.42) and the parity polynomial lhl(X) of the cyclic product code is the greatest common divisor of lhl1 (Xb 112 ) and lhl2 (Xa 111 ); that is, (5.43) The complexity of the decoder for cyclic product codes is comparable to the complexity of the decoders for both the (111, k1) code and the (112, k2) code. At the receiving end of the channel, the received vector may again be rearranged as a rectangular array. Thus, the decoder can decode each of the row (or column) codewords separately and then decode each of the column (or row) codewords.

I I I

I

r I I I

-----<1----r-----1 I

I

I I I

t I I I

- - - - - - - - -,- - - - - - I I

y

y

I I I

I I

B

FIGURE 5.23: Transmission of a cyclic product code.

Section 5.12

Quasi-Cyclic Codes

'rnS

in the transmitted codeword, the set of n1 digits formed every (n2)th are the n1 of a codeword of C1 permuted in a fixed way. can be permuted back to their original form and corrected by a Meggitt-type decoder. The in the form are a codeword in a related code and can be decoded directly in this form by a Meggitt-type decoder. the column code C2 can be corrected by selecting every (n1)th from the large codeword. Thus, the total required is roughly that to decode the two individual codes. 5. U

QIUA§!-(VCUC (0\Dl!E§

codes possess full cyclic that shifting a codeword any either to the right or to the results in another symmetry structure makes it to implement the encoding and of codes with simple shift registers and logic circuits. There are other linear block codes that do not possess full cyclic symmetry but do have structure, quasi-cyclic codes. A quasi-cyclic code is a linear code for which cyclically shifting a codeword a fixed number no -f. 1 ( or a multiple of no) of symbol positions either to the right or to the left results in another codeword. It is clear that for no = 1, a quasi-cyclic code is a cyclic code. The integer no is called the shifting constraint. It is clear that the dual of a quasi-cyclic code is also quasi-cyclic. As an example, consider the (9, 3) code generated by the following generator matrix: cG =

111 100 110] 110 111 100 . [ 100 110 111

The eight codewords of this code are listed in Table 5.6. Suppose we cyclically shift the fifth codeword in Table 5.6, (00101101 0) three symbol positions to the right. This shift results in the seventh codeword (010001 011) in Table 5.6. If we cyclically shift the fifth codeword one and two positions to the right, we obtain two vectors, (0 0 0 1 0 11 0 1) and (1 0 0 0 1 0 1 1 0), respectively, which are not codewords in Table 5.6. Therefore, the (9, 3) code given in Table 5.6 is a quasi-cyclic code with shifting constraint no = 3. This code also can be encoded with a shift register as shown in Figure 5.24. Let (co, c1, c2) be the message to be encoded. As soon as the 5.6: The codewords of the (9, 3) quasi-cyclic code.

TABLE

000 111

110 100 001 011 010 101

000 100 111 110 011

010 001 101

000 110

100 111

010 001 011 101

186

Cyclic Codes

Chapter 5

.------; Gate 2 14-----~ Input

Output

FIGURE 5.24: An encoding circuit for a (9, 3) quasi-cyclic code. three information symbols have been shifted into the register, gate 1 is deactivated and gate 2 is activated. The information symbol C2 and two parity-check symbols and appear at the output terminals and then are shifted into the channel. The two parity-check symbols are given by

Pil)

pf)

(1)

P2 =co+ c2, (2)

P2

= co

+ ci + c2.

Next, the register is shifted once. The content of the register is now (c2, co, c1). The information symbol c1 and two parity-check symbols and 2 ) appear at the output terminals, and they are shifted into the channel. The parity-check symbols and are given by

p?)

p?)

Pi

p?)

(l)

P1

(2)

P1

= c1 + c2,

= co

+ q + c2.

At this point, the register is shifted once again. The content of the register is now (ci, c2, co), and the information symbol co and two parity-check symbols p~l) and 2 p~ ) appear at the output terminals. These three symbols are then shifted into the channel. The two parity-check symbols are given by (l)

Po =co+c1, (2)

Po =co+ ci

+ c2.

This completes the encoding. The codeword has the form V

=

(2) ( Po ,

(1)

(2)

(1)

(2)

(1)

C )

Po , co, P1 , P1 , CI, P2 , P2 , 2 ,

which consists of three blocks, each of which consists of one unaltered information symbol and two parity-check symbols. This form may also be regarded as a systematic form.

Section 5.12

Quasi-Cyclic Codes

iU

For an (mno. mko) quasi-cyclic code with shifting constraint no, the matrix in systematic form is

cG = [

Jll'oll lP"m-1®

Jll'1@

···

lPoil

···

...

...

.

lP1®

Jll'2@

JPloil

lP"m-1® ] lPm-2@ '

(5.44)

where Il and @ represent the ko >< ko identity and zero respectively, and JP; is an arbitrary ko x (no - ko) matrix. Each row ( as a sequence of m ko >< no matrices) is the cyclic shift (to the right) of the row immediately above it, and the row at the is the cyclic shift of the bottom row. Each column of G is the downward cyclic shift of the column on its left (or the upward cyclic shift of the column on its right). A message for the code consists of m ko-bit blocks, and a codeword consists of m no-bit blocks. Each of these m no-bit blocks consists of ko unaltered information symbols and no - ko parity-check symbols. The parity-check matrix corresponding to the generator matrix given by (5.44) is IllP5

@JP;_l

@lP'f

IllP'5

(5.45)

lHI=

where IT and @ represent the (no - ko) >< (no - ko) identity and zero matrices, respectively, and JP"! is the transpose of lP';. Consider the (9, 3) quasi-cyclic code given previously for which ko = 1 and no = 3. The parity-check matrix in systematic form is

lHI=

101 011 001 000 001 001

001 001 101 011 001 000

001 000 001 001 101 011

A more general form for the generator matrix of an (mno, mko) quasi-cyclic code is Go Gm-1

G1 Go

cG,n-l Gm-2

(5.46)

G= G2 G1

G3 G2

G1 Go

where each G; is a ko x no submatrix. We see that G given by (5.46) displays the cyclic structure among the rows and columns in terms of the submatrices G; 's. For 0 ::=: j < m, let M1 ~ [G1 , GJ-1, · · · , GJ+1f denote the }th column of G (with G 111 = Go). M 1 is an mko x no submatrix. Then, we can put Gin the following form: G = [Mo, M1, · · · , M111-1].

188

Chapter 5

Cyclic Codes

For O :S l < no, let Qz be the mko x m subrnatrix that is formed by taking the Ith columns from Mo, M1, · · · , Mm-1· Then, we can put Gin the following form:

Ge= [Qo, Q1, · · · , Qno-1]. Each column of Qz consists of mko bits that are regarded as m ko-bit bytes (a byte is a group of ko binary digits). In terms of bytes, Qz is regarded as an m x m matrix that has the following cyclic structure: (1) each row is the cyclic shift (to the right) of the row immediately above it, and the top row is the cyclic shift of the bottom row; (2) each column is the downward cyclic shift of the column on its left, and the leftmost column is the downward cyclic shift of the rightmost column. The matrix Q 1 is called a circulant. Therefore, Ge consists of no circulants. Most often, quasi-cyclic codes are studied in circulant form. !EXAMPLE 5.14

Consider the (15, 5) quasi-cyclic code with parameters m = 5, no = 3, and ko = 1 that is generated by the following generator matrix:

G

=

[ 001 110 110 010 100

100 001 110 110 010

010 100 001 110 110

110 010 100 001 110

110 110 010 100 001

Mo

M1

M2

M3

M4

l .

This quasi-cyclic code has a minimum distance of 7. In circulant form, the generator matrix takes the following form:

G

=

[ 01011 10101 11010 01101 10110 Qo

00111 10011 11001 11100 01110

01000 10000 00100 00010 00001

Q1

Q2

l .

IPROIBllEMS 5.1 Consider the (15, 11) cyclic Hamming code generated by g(X) a. Determine the parity polynomial h(X) of this code.

= 1 + X + X 4.

b. Determine the generator polynomial of its dual code. c. Find the generator and parity matrices in systematic form for this code. 5.2 Devise an encoder and a decoder for the (15, 11) cyclic Hamming code generated by g(X) = 1 + X + X 4 .

Problems

Us9

.:.t3 Show that g(X) = 1 + X 2 + X 4 + X 6 + X 7 + X 10 generates a (21, 11) cyclic code. Devise a syndrome computation circuit for this code. Let ir(X) = 1 + X 5 + be a received polynomial. Compute the syndrome of r(X). Display the contents of the syndrome register after each digit of ir has been shifted into the syndrome computation circuit. !:loi:tl Shorten this 11) cyclic Hamming deleting the seven leading high-order message digits. The resultant code is an (8, 4) shortened cyclic code. Design a decoder for this code that eliminates the extra shifts of the syndrome register. !:lo§ Shorten the (31, 26) cyclic Hamming code by deleting the 11 leading high-order message digits. The resultant code is a (20, 15) shortened cyclic code. Devise a decoding circuit for this code that requires no extra shifts of the syndrome register. §J'i Let g(X) be the generator polynomial of a binary cyclic code of length 11. mo Show that if g(X) has X + 1 as a factor, the code contains no codewords of odd weight. lbio Un is odd and X + 1 is not a factor of g(X), show that the code contains a codeword consisting of all l's. «:o Show that the code has a minimum weight of at least 3 if n is the smallest integer such that g(X) divides X" + L §:7 Consider a binary (n, k) cyclic code C generated by g(X). Let

be the reciprocal polynomial of g(X). Show that g*(X) also generates an (11, k) cyclic code. li:Do Let C* denote the cyclic code generated by g*(X). Show that C and C* have the same weight distribution. (Hint: Show that

in,

is a code polynomial in C if and only if

§J§

is a code polynomial in C*.) Consider a cyclic code C of length n that consists of both odd-weight and evenweight codewords. Let g(X) and A(z) be the generator polynomial and weight enumerator for this code. Show that the cyclic code generated by (X + l)g(X) has weight enumerator A1(z)

=

HA(z)

+ A(-z)].

Suppose that the (15, 10) cyclic Hamming code of minimum distance 4 is used for error detection over a BSC with transition probability p = 10~ 2 . Compute the probability of an undetected error, Pu (E ), for this code. :'L].IO Consider the (2 111 - 1, 2111 - 111 - 2) cyclic Hamming code C generated by g(X) = (X + l)p(X), where I\Jl(X) is a primitive polynomial of degree m. An error pattern of the form §oil))

190

5,11

5.12 5.13

5.14

Chapter 5

Cyclic Codes

is called a double-adjacent-error pattern. Show that no two double-adjacent-error patterns can be in the same coset of a standard array for C. Therefore, the code is capable of correcting all the single-error patterns and all the double-adjacent-error patterns. Devise a decoding circuit for the (7, 3) Hamming code generated by g(X) = (X + l)(X 3 + X + 1). The decoding circuit corrects all the single-error patterns and all the double-adjacent-error patterns (see Problem 5.10). For a cyclic code, if an error pattern e(X) is detectable, show that its ith cyclic shift e
= v(X).

Show that if l /:- 0, l is a factor of n. 5.15 Let g(X) be the generator polynomial of an (n, k) cyclic code C. Suppose C is interleaved to a depth of Jc. Prove that the interleaved code c>- is also cyclic and its generator polynomial is g(X,. ). 5.16 Construct all the binary cyclic codes of length 15. (Hint: Using the fact that X 15 + 1 has all the nonzero elements of GF(24 ) as roots and using Table 2.9, factor X 15 + 1 as a product of irreducible polynomials.) 5.17 Let f3 be a nonzero element in the Galois field GF(2111 ), and f3 I- 1. Let if>(X) be the minimum polynomial of (3. Is there a cyclic code with (X) as the generator polynomial? If your answer is yes, find the shortest cyclic code with if>(X) as the generator polynomial. 5.18 Let f3i and f3z be two distinct nonzero elements in GF(2111 ). Let 1 (X) and 2 (X) be the minimal polynomials of f3i and (3 2 , respectively. Is there a cyclic code with g(X) = 1 (X) · ef; 2 (X) as the generator polynomial? If your answer is yes, find the shortest cyclic code with g(X) = ¢ 1(X) · 4> 2 (X) as the generator polynomial. 5.19 Consider the Galois field GF(2111 ), which is constructed based on the primitive polynomial p(X) of degree m. Let ex be a primitive element of GF(2111 ) whose minimal polynomial is p(X). Show that every code polynomial in the Hamming code generated by p(X) has ex and its conjugates as roots. Show that any binary polynomial of degree 2111 - 2 or less that has ex as a root is a code polynomial in the Hamming code generated by p(X). 5.20 Let C 1 and C2 be two cyclic codes of length n that are generated by g1 (X) and g2(X), respectively. Show that the code polynomials common to both C 1 and C2 also form a cyclic code C3. Determine the generator polynomial of C3. If d1 and d2 are the minimum distances of C 1 and C2 , respectively, what can you say about the minimum distance of C3 ? 5,21 Show that the probability of an undetected error for the distance-4 cyclic Hamming codes is upper bounded by 2-<111 +1!. 5.22 Let C be a (2m -1, 2111 -m-l) Hamming code generated by a primitive polynomial p(X) of degree m. Let Cc1 be the dual code of C. Then, Cc1 is a (2 111 - 1, m) cyclic code generated by

Problems

1~1

where lhl(X)

= - - -+1 JP)(X)

m, Let v(X) be a codeword in Cd and let vUl (X) be the ith Show that for 1::: 2111 - 2, vCil(X) f v(X).

i:::

shift of v(X).

IbJ, Show that Cc1 contains the all-zero codeword and 2111 - 1 codewords of weight 2111-l. (Hint: For part (a), use (5.1) and the fact that the smallest integer n such that X 11 + 1 is divisible JlD(X) is 2111 - 1. For part (b ), use the result of Problem 3.6(b ).) 5,::23 For an (n, k) cyclic code, show that the syndrome of an end-around burst of length n - k cannot be zero. 5,::24\ Design a Meggitt decoder that decodes a received polynomial rr(X) = ro + r1 X + ·. ·+r11 _ 1 x 11 - 1 from the lowest-order received digit ro to the highest-order received digit r11 _ 1. Describe the decoding operation and the syndrome modification after each correction. 5:l§ Consider the 5) cyclic code generated by the following polynomial: 1l';(X)

5,Z([ii

5:27

5,W

5,Z
=1+X +

+x4+xs+xs+x10.

This code has been proved to be capable of correcting any combination of three or fewer errors. Suppose that this code is to be decoded by the simple error-trapping decoding scheme. m, Show that all the double errors can be trapped. lbJ, Can all the error patterns of three errors be trapped? If not, how many error patterns of three errors cannot be trapped? lC, Devise a simple error-trapping decoder for this code. ii, Devise a simple error-trapping decoder for the (23, 12) Golay code. IbJ, How many error patterns of double errors cannot be trapped? ll:o How many error patterns of three errors cannot be trapped? Suppose that the (23, 12) Golay code is used only for error correction on a BSC with transition probability p. If Kasami's decoder of Figure 5.18 is used for decoding this code, what is the probability of a decoding error? (Hint: Use the fact that the (23, 12) Golay code is a perfect code.) Use the decoder of Figure 5.18 to decode the following received polynomials: m, rt(X) = X 5 + x 19 IbJ, rr(X) = x4 + xn + x21 At each step in the decoding process, write down the contents of the syndrome register. Consider the following binary polynomial: 1l';(X)

= (X 3 + l)Jlll(X),

where (X 3 +1) and Jlll(X) are relatively prime, and JlD(X) is an irreducible polynomial of degree m with m 2: 3. Let n be the smallest integer such that !!;(X) divides X 11 + 1. Thus, g(X) generates a cyclic code of length n.

192

Chapter 5

Cyclic Codes

a. Show that this code is capable of correcting all the single-error, doubleadjacent-error, and triple-adjacent-error patterns. (Hint: Show that these error patterns can be used as coset leaders of a standard array for the code.) b. Devise an error-trapping decoder for this code. The decoder must be capable of correcting all the single-error, double-adjacent-error, and triple-adjacenterror patterns. Design a combinational logic circuit whose output is 1 when the errors are trapped in the appropriate stages of the syndrome register. c. Suppose that p(X) = 1 + X + X 4 , which is a primitive polynomial of degree 4. Determine the smallest integer n such that g(X) = (X 3 + l)p(X) divides

xn + l.

5.30 Let C1 be the (3, 1) cyclic code generated by g1(X) = 1 + X + X 2 , and let C2 be the (7, 3) maximum-length code generated by g2 (X) = 1 + X + X 2 + X 4 . Find the generator and parity polynomials of the cyclic product of C1 and C2. What is the minimum distance of this product code? Discuss its error-correcting capability. 5.31 Devise an encoding circuit for the (15, 5) quasi-cyclic code given in Example 5.14. BIBUOGRAPHY

1. E. Prange, "Cyclic Error-Correcting Codes in Two Symbols," AFCRC-TN-57, 103, Air Force Cambridge Research Center, Cambridge, Mass., September 1957. 2. E. R. Berlekamp, Algebraic Coding Theory, McGraw-Hill, New York, 1968. (Rev. ed., Aegean Park Press, Laguna Hills, Calif., 1984.) 3. W. W. Peterson and E. J. Weldon, Jr., Error-Correcting Codes, 2d ed., MIT Press, Cambridge, Mass., 1972. 4. F. J. MacWilliams and N. J. A. Sloane, The Theory of Error-Correcting Codes, North Holland, Amsterdam, 1977. 5. R. E. Blahut, Theory and Practice of Error Control Codes, Addison-Wesley, Reading, Mass., 1984. 6. R. J. McEliece, The Theory of Information and Coding, Addison-Wesley, Reading, Mass., 1977. 7. G. Clark and J. Cain, Error-Correction Codes for Digital Communications, Plenum, New York, 1981. 8. S. A. Vanstone and P. C. van Oorschot, An Introduction to Error Correcting Codes with Applications, Kluwer Academic, Boston, Mass., 1989. 9. S. B. Wicker, Error Control Systems for Digital Communication and Storage, Prentice Hall, Englewood Cliffs, N.J., 1995. 10. W. W. Peterson, "Encoding and Error-Correction Procedures for the BoseChaudhuri Codes," IRE Trans. Inform. Theory, IT-6, 459-70, September 1960. 11. J.E. Meggitt, "Error Correcting Codes and Their Implementation," IRE Trans. Inform. Theory, IT-7: 232-44, October 1961.

Bibliography

11Bl

U, T. Kasami, "A Decoding Method for Multiple-Error-Correcting Cyclic Codes by Using Threshold Logics," Conf Rec. Process. Soc. (in Japanese), Tokyo, November 1961.

B, M. E. Mitchell et al., "Coding and Decoding Operation Research," G. E. Advanced Electronics Final on Contract AF 19 (604)-6183, Air Force Cambridge Research Labs., Cambridge, Mass., 1961. ]AL M. E. Mitchell, "Error-Trap Decoding of Codes," G. E. Report No. 62MCD3, General Electric Communications Dept., Oklahoma

December 1962. JL§, E. Prange, "The Use of Information Sets in Decoding

Codes," IEEE

Trans. Inform. Theory, ff-8: 85-80, September 1962. Jliiii, L. Rudolph, "Easily Implemented Error-Correction Encoding-Decoding," G. E. Report No. 62MCD2, General Electric Corporation, Oklahoma City, December 1962.

Jl7. T. Kasami, "A Decoding Procedure For Multiple-Error-Correction Cyclic Codes," IEEE Trans. Inform. Theory, H-10: 134-39, April 1964. ]li;t

F. J. MacWilliams, "Permutation Decoding of Systematic Codes," Bell Syst. Tech. 43 (p. 1): 485-505, January 1964.

Jll!)), L. Rudolph and M. E. Mitchell, "Implementation of Decoders for Cyclic Codes," IEEE Trans. Inform. Theory, H-10: 259-60, July 1964. 21[]), D. C. Foata, "On a Program for Ray-Chaudhuri's Algorithm for a Minimum Cover of an Abstract Complex," Commun. ACM, 4: 504-6, November 1961. 2Jl, I. B. Pyne and E. J. McCluskey, ·'The Reduction of Redundancy in Solving Prime Implicant Tables," IRE Trans. Electron. Comput., EC-11: 473-82, August 1962.

22, M. J.E. Golay, "Notes on Digital Coding," Proc.

37: 657, June 1949.

Z3\ E. J. Weldon, Jr., "A Comparison of an Interleaved Golay Code and a ThreeDimensional Product Code," Final Report, USNELC Contract N0095368M5345, San Diego, CA, August 1968. Z~, S. K. Leung-Yan-Cheong, E. R. Barnes, and D. U. Friedman, "On Some Properties of the Undetected Error Probability of Linear Codes," IEEE Trans. Inform. Theory, IT-25 (1): 110-12, January 1979. 2§, H. 0. Burton and E. J. Weldon, Jr.,"Cydic Product Codes," IEEE Trans. Inform. Theory, ff-11: 433-40, July 1965. 2(6, S. Lin and E. J. Weldon, Jr., "Further Results on Cyclic Product Codes," IEEE Trans. Inform. Theory, IT-16: 452-59, July 1970. Z7, W. C. Gore, "Further Results on Product Codes," IEEE Trans. Inform. Theory, IT-16: 446-51, July 1970. 2§, N. M. Abramson, "Cascade Decoding of Cyclic Product Codes:' IEEE Trans. Commun. Technol., COM-16: 398-402, 1968.

CHAPTER

Bina

6

BCH Codes

The Bose, Chaudhuri, and Hocquenghem (BCH) codes form a large class of powerful random error-correcting cyclic codes. This class of codes is a remarkable generalization of the Hamming codes for multiple-error correction. Binary BCH codes were discovered by Hocquenghem in 1959 [1] and independently by Bose and Chaudhuri in 1960 [2]. The cyclic structure of these codes was proved by Peterson in 1960 [3]. Binary BCH codes were generalized to codes in pm symbols (where p is a prime) by Gorenstein and Zierler in 1961 [4]. Among the nonbinary BCH codes, the most important subclass is the class of Reed-Solomon (RS) codes. The RS codes were discovered by Reed and Solomon in 1960 [5] independently of the work by Hocquenghem, Bose, and Chaudhuri. The first decoding algorithm for binary BCH codes was devised by Peterson in 1960 [3]. Then, Peterson's algorithm was generalized and refined by Gorenstein and Zierler [4], Chien [6], Forney [7], Berlekamp [8, 9], Massey [10, 11], Burton [12], and others. Among all the decoding algorithms for BCH codes, Berlekamp's iterative algorithm, and Chien's search algorithm are the most efficient ones. In this chapter we consider primarily a subclass of the binary BCH codes that is the most important subclass from the standpoint of both theory and implementation. Nonbinary BCH codes and Reed-Solomon codes will be discussed in Chapter 7. For a detailed description of the BCH codes, and their algebraic properties and decoding algorithms, the reader is referred to [9] and [13-17]. 6.1

BINARY PRIMITIVE BCH CODES

For any positive integers m(m :::: 3) and t (t < 2m-l ), there exists a binary BCH code with the following parameters: Block length: Number of parity-check digits: Minimum distance:

n = 2111 -1, n-k:S:mt. dmin ::'. 2t + 1.

Clearly, this code is capable of correcting any combination of t or fewer errors in a block of n = 2111 - 1 digits. We call this code a t-error-correcting BCH code. The generator polynomial of this code is specified in terms of its roots from the Galois field GF(2 111 ). Let a be a primitive element in GF(2111 ). The generator polynomial g(X) of the t-error-correcting BCH code of length 2m - 1 is the lowest-degree polynomial over GF(2) that has (6.1) as its roots [i.e., g(a;) = 0 for 1 :s: i .::: 2t]. It follows from Theorem 2.11 that g(X) has a, a 2 , · · · , a 21 and their conjugates as all its roots. Let ;(X) be the 194

Section 6.1

Binary Primitive BCH Codes

i!ll:S

minimal polynomial of o/. Then, g(X) must be the least common multiple (LCM) of rr/»1 (X), «J;,2(X), · · · .
= LCM{
(6.2)

,
If i is an even integer, it can be expressed as a product of the following form: i

=i'i.

where i' is an odd number, and l :::'.. 1. Then, o/ = (o/)2' is a conjugate of o/, and therefore o/ and o/ have the same minimal polynomial: that is, «f;i;(X)

=
Hence, every even power of a in the sequence of has the same minimal polynomial as some preceding odd power of a in the sequence. As a result, the generator polynomial ~(X) of the binary t-error-correcting BCH code of length 2111 - 1 given by (6.2) can be reduced to (6.3) Because the degree of each minimal polynomial is 111 or less, the degree of ~(X) is at most mt; that the number of parity-check digits, n - k, of the code is at most equal to mt. There is no simple formula for enumerating n - k, but if tis small, n - k is exactly equal to mt [9, 18]. The parameters for all binary BCH codes of length 2111 - 1 with m ::; 10 are given in Table 6.1. The BCH codes just defined are usually called primitive ( or narrow-sense) BCH codes. 6.1: BCH codes generated by primitive elements of order less than 2 10 .

TABLE

ra

k

u

ra

k

7 15

4 11 7 5 26 21 16 11 6 57 51 45 39 36 30 24

1 1 2 3 1 2 3 5 7 1 2 3 4 5 6 7

127

50 43 36 29 22 15 8 247 239 231 223 215 207 199 191 187

31

63

255

ff

13 14 14 21 23 27 31 1 2 3 4 5 6 7 8 9

UJ

1k

t

255

71 63 55 47 45 37 29 21 13 9 502 493 484 475 466 457

29 30 31 42 43 45 47 55 59 63 1 2 3 4 5 6

511

( continued overleaf)

196

Chapter 6

Binary BCH Codes TAB LE 6. 1: ( continued)

n

127

511

1023

k

t

18 16 10 7 120 113 106 99 92 85 78 71 64 57 322 313 304 295 286 277 268 259 250 241 238 229 220 211 202 193 184 175 848 838 828 818 808 798 788 778 768 758 748 738

10 11 13 15 1 2 3 4 5 6 7 9 10 11 22 23 25 26 27 28 29 30 31 36 37 38 39 41 42 43 45 46 18 19 20 21 22 23 24 25 26 27

28 29

n

511

1023

k

179 171 163 155 147 139 131 123 115 107 99 91 87 79 166 157 148 139 130 121 112 103 94 85 76 67 58 49 40 31 28 19 553 543 533 523 513 503 493 483 473 463 453 443

t

10 11 12 13 14 18 19 21 22 23 24 25 26 27 47 51 53 54 55 58 59 61 62 63 85 87 91 93 95 109 111 119 52 53 54 55 57 58 59 60 61 62 63 73

n

511 1023

1023

k

448 439 430 421 412 403 394 385 376 367 358 349 340 331 10 1013 1003 993 983 973 963 953 943 933 923 913 903 893 883 873 863 858 268 258 249 238 228 218 208 203 193 183 173 163

t

7 8 9 10 11 12 13 14 15 16 18 19 20 21 121 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 103 106 107 109 110 111 115 117 118 119 122 123

Section 6.1

Binary Prirnitive BCH Codes

1!ll1

TABLE 6.1: PE

k

t

728 718 708 698 688 678 668 658 648 638 628 618 608 598 588 578 573 563

30 31 34 35 36 37 38 39 41 42 43 44 45 46 47 49 50 51

t

k

ui

433 423 413 403 393 383 378 368 358 348 338 328 318 308 298 288 278

n

74 75 77 78 79 82 83 85 86 87 89 90 91 93 94 95 102

1k

t

153 143 133 123 121 111 101 91 86 76 66 56 46 36 26 16 11

125 126 127 170 171 173 175 181 183 187 189 191 219 223 239 147 255

From (6.3), we see that the single-error-correcting BCH code of length 2111 is generated ~(X)

-

1

=
Because ot is a element of GF(2111 ),
Let a be a primitive element of the Galois field GF(24 ) given by Table 2.8 such that 1 + ot + a 4 = 0. From Table 2.9 we find that the minimal polynomials of a, a3, and a 5 are

+ X 4. 2 3 ¢3(X) = 1 + x + x + x + x4, ¢, 1 (X) = 1 + X

and

qp5(X) = 1 + X

+ X2,

respectively. H follows from (6.3) that the double-error-correcting BCH code of length n = 24 - 1 = 15 is generated by

198

Chapter 6

Binary BCH Codes

Because <{> 1(X) and

<{> 3 (X)

are two distinct irreducible polynomials,

g(X) = 1 (X)3(X)

+ X + X 4 )(1 + X + x 2 + x 3 + X 4 ) = 1 + x4 + x 6 + x 7 + x8 .

= (1

Thus, the code is a (15, 7) cyclic code with dmin ::::: 5. Since the generator polynomial is a code polynomial of weight 5, the minimum distance of this code is exactly 5. The triple-error-correcting BCH code of length 15 is generated by g(X) = LCM{1 (X), <{>3(X), s(X)}

= (1 + X + X 4 )(1 + X + X 2 + x 3 + X 4 )(1 + X + X 2 ) = 1 + x + x 2 + x4 + xs + x8 + x10 . This triple-error-correcting BCH code is a (15, 5) cyclic code with dm; 11 ::::: 7. Because the weight of the generator polynomial is 7, the minimum distance of this code is exactly 7. Using the primitive polynomial p(X) = 1 + X + X 6 , we may construct the Galois field GF(2 6 ), as shown in Table 6.2. The minimal polynomials of the elements

TABLE 6.2: Galois field GF(2 6 ) with p(a) 0 1 a a2 a3 a4 as a6

0 1 a2 a3 a4 as

1

+

O' 7

O' O'

as

+

0'2 0'2

+

0'9

0'3

a3

0'10 all

+

a

+

al3

a2 a2

1

+

O'

+

a a a

al9

a20

1

as as

+

as

+

as

+

as

+a4 a3

alS al7

+ +

a3

O'

0'14

alS

+a4 0'4

1

al2

al6

= 1 +a+ a 6 = 0.

+a4

+ + +

a2 a2 a2 a2

+ + +

a3 a3 a3

+a4 +a4

(0 0 0 0 0 0) (100 0 0 0) (0 100 0 0) (0 0 100 0) (0 0 0 100) (0 0 0 0 1 0) (000001) (1100 0 0) (0 1100 0) (0 0 1100) (0 0 0 11 0) (000011) (110001) (10100 0) (0 10100) (0 0 101 0) (000101) (1100 1 0) (011001) (111 1 0 0) (0 111 1 0) (001111)

Section 6.1

Binary Primitive BCH Codes

TABLE 6.2: (continued)

0{21 0/22 0{23

0/24

1 + 1 1 1

0/25 0{26

+

O{

+

0/2

+a4 +

0/3

1 +

OI

+ +

0/2

+ +

0{29

0{5

+

as

0/3

+ + 0{4

O{

+

0/2 +

1

+

+

O{ O{

+

0{37

()/38 ()/39

+

a a

a40 0/41 ()/42

1 + 1

a43 a44 a45 a46 0/47 ()/48 0{49

1 + 1 1

a

+ +

0{52

a53 a54

1 1

+

0{59 Ol60

0/61 Ol62

0/3 0/3

0/2

+ +

a

+ +

0/2 0/2

1 1

+

+ + +

O{

a3

1

+ +

O{

0/

1 +

0/

0/

1 1 1 1 1

+

Ol

as

+

as

+ + + + + + +

(X3

(X2

0/2 0/2

+ + + + + +

+ + +

+ +

0{5

+ +

()/5

a4 ()/4 4 4 0/

+ + +

as 0{5 as

O{

+ +

a4 a4

+

(X4

0{3

(X3

(X2

0/2 0/2

a4

+ +

as

0/5

0{3

0/2

a

+ +

0{4

a3

(X2

Ol57

0/58

+ + +

a2

a

ass

a56

0/2

0{4

a3

0/

aso

a51

()/3

0/2 0{2

+ a3

+ + + +

O{

a

+ ()/3

()/2 0{2

1

0{5

0{4

O{

0{36

+ ()/3

0{2

0/35

as

0{3

0!3

()/33

+

0{5

a2 0{2

+

as

•1

O{

1 1 1

+ + +

+ a-·

0{28

0{32

+ 0{4

a

0/27

()/30 ()/31

0{3

+

0{4

+ + + + + +

0{4

(X3 ()/3

(X3

0/3 0/3 0{3

Ol4 0/4 ()/4 0{4 (X4

0/5 +

as

+

as

+

0{5

+ + + + + +

I (X63

()/5 0{5 a5

0/5

ots a5

= 11

(110111) (101011) (100101) (100 0 1 0) (010001) (111 0 0 0) (0 11100) (0 0 111 0) (000111) (110011) (10100 1) (100 100) (0 100 1 0) (001001) (110100) (0 1101 0) (001101) (11011 0) (011011) (111101) (1 0 111 0) (010111) (111 0 11) (101101) (100 11 0) (010011) (111001) (101100) (0 1011 0) (001011) (110101) (10101 0) (0 10101) (111 0 1 0) (011101) (111110) (0 11111) (111111) (101111) (100111) (100011) (100001)

1!lJ!lJ

200

Chapter 6

Binary BCH Codes

TABLE 6.3: Minimal polynomials of the elements in GF(2 6 ).

Minimal polynomials a, a2, a4, al6, a32 a3, a6, al2a24, a48a33 as, alO, a20, a40, al 7, a34 a7, al4, a28, as6, a49, a3S a9, a1s, a36 all, a22, a44, a2s, aso, a37

al3,a26,aS2,a41,al9,a38 alS, a30, a60, aS7, as 1, a39

a21, a42 a23,a46,a29,ass,as3,a43 a27, aS4, a4S a31, a62, a6l, aS9, ass, a47

1+ X + X6 1 + x + x2 + x4 + x 6 1 + x + x 2 + xs + x 6 1 + x3 + x 6 1 + x2 + x3 1 + x 2 + x 3 + xs + x 6 1 + x + x3 + x4 + x 6 1 + x 2 + x 4 + xs + x 6 1 + X + X2 1 + x + x 4 + xs + x 6 1 + X + X3 1 + xs + x 6

TABLE 6.4: Generator polynomials of all the BCH codes of length 63. n

k

63

57 51 45 39 36 30 24 18 16 10 7

t 1 2 3 4 5 6 7 10 11 13 15

g(X) g1(X) = 1 + X + X 6 g2(X) = (1 + X + X 6)(1 + X + x 2 + x 4 + X 6) g3(X) = (1 + X + X 2 + xs + X 6)g2(X) g4(X) = (1 + X 3 + x 6)g3(X) gs(X) = (1 + X 2 + X 3)g4(X) g 6 (X) = (1 + x 2 + x 3 + xs + x 6)gs(X) g7(X) = (1 + X + X 3 + X 4 + X 6)g6(X) g10(X) = (1 + X 2 + X 4 + xs + X 6)g7(X) g11 (X) = (1 + X + X 2)g10(X) g13(X) = (1 + X + X 4 + xs + X 6)g11 (X) g1s(X) = (1 + X + X 3)g13(X)

in GF(2 6 ) are listed in Table 6.3. Using (6.3), we find the generator polynomials of all the BCH codes of length 63, as shown in Table 6.4. The generator polynomials of all binary primitive BCH codes of length 2111 - 1 with m :s 10 are given in Appendix C. It follows from the definition of a t-error-correcting BCH code of length n = 2111 - 1 that each code polynomial has a, a 2 , ... , a 21 and their conjugates as roots. Now, let v(X) = vo + v1X + · · · + v11 _1x 11 - 1 be a polynomial with coefficients from GF(2). If v(X) has a, a 2 , .. · , a 21 as roots, it follows from Theorem 2.14 that v(X) is divisible by the minimal polynomials 1 (X),4> 2 (X), ... , 21 (X) of a, a 2 , · · · , a 21 . Obviously, v(X) is divisible by their least common multiple (the generator polynomial), g(X) = LCM {¢1 (X), ¢2(X), · · · , ¢21 (X)}.

Section 6.1

Binary Primitive BCH Codes

2.11ll1

Hence, w(X) is a code polynomial. Consequently, we may define at-error-correcting BCH code of length n = 2111 - 1 in the following manner: a binary n-tuple w = (vo, v1, v2, · · · , v11 _1) is a codeword if and only if the polynomial w(X) = vo + v1X + ... + v11 _ 1 x 11 - 1 has a, a 2 , · · · , a 21 as roots. This definition is useful in proving the minimum distance of the code. Let v(X) = vo + v1X + · · · + v11 _1x 11 - 1 be a code polynomial in a t-errorcorrecting BCH code of length n = 2111 - 1. Because o/ is a root of w(X) for 1 :S i :s 2t, then (6.4)

This equality can be written as a matrix product as follows:

=0

(vo, v1. · · · , v11-1) ·

(6.5)

for 1 :S i :s 2t. The condition given by (6.5) simply says that the inner product of · equa l t o zero. N ow. we f orm th e (vo, v1, · · · , v 11 _1 ) an d (1 , ot ;, a,2; , · · · , a (11-11;) 1s following matrix:

lHI=

1 1 1

(){

Ol2

(){3

Olll-1

(a2) (ot3)

(a2 )2 (ot3)2

(a2)3 (ot3)3

(ot2y1-l (a3y1-l

1

(a21)

(ot2t)2

(a21 )3

(0:21)'1-l

(6.6)

H follows from (6.5) that if w = (vo, v1, · · · , v11 _1) is a codeword in the t-errorcorrecting BCH code, then w. JHI']f = ill)_ (6.7)

On the other hand, if an n-tuple w = (vo, v1, · · · , v11 _1) satisfies the condition of (6.7), it follows from ( 6.5) and ( 6.4) that, for 1 :s i :S 2t, o: 1 is a root of the polynomial w(X). Therefore, w must be a codeword in the t-error-correcting BCH code. Hence, the code is the null space of the matrix lHI, and IHI is a parity-check matrix of the code. U for some i and j, ai is a conjugate of a;, then v(ai) = 0 if and only if w(a 1) = 0 ( see Theorem 2.11 ); that is, if the inner product of w = (vo, v1, · · · , v11 _ 1) and the i th row of lHI is zero, the inner product of wand the jth row of IHI is also zero. For this reason, the jth row of lHI can be omitted. As a result, the lHI matrix given by (6.6) can be reduced to the following form:

IHI=

1 1 1

a

(.¥2

Ol3

Oln-1

(a3) (as)

(a3)2 (a5)2

(a3)3 (ot5)3

(o:3)11-l (a5)'1-l

1

(a21-1)

(a2t-1 )2

(a21-l )3

(0t21-l)'1-1

(6.8)

202

Chapter 6

Binary BCH Codes

Note that the entries of Hare elements in GF(2 111 ). We can represent each element in GF(2 111 ) by an 111-tuple over GF(2). H we replace each entry of lHl with its corresponding m-tuple over GF(2) arranged in column form, we obtain a binary parity-check matrix for the code. EXAMPLE 6.2

Consider the double-error-correcting BCH code of length n = 24 - 1 = 15. From Example 6.1 we know that this is a (15, 7) code. Let a be a primitive element in GF(2 4 ). Then, the parity-check matrix of this code is

H-_[11

a a2 a3 a4 a5 a6 a 7 a8 a9 a10 all a12 a13 a14] a3 a6 a9 al2 a15 al8 a21 a24 a27 a30 a33 a36 a39 a42

[by (6.8)]. Using Table 2.8 and the fact that a 15 = 1, and representing each entry of H with its corresponding 4-tuple, we obtain the following binary parity-check matrix for the code:

lHl=

1 0 0 0 1 0 0 1 1 0 1 0 1 1 1 0 1 0 0 1 1 0 1 0 1 1 1 1 0 0 0 0 1 0 0 1 1 0 1 0 1 1 1 1 0 0 0 0 1 0 0 1 1 0 1 0 1 1 1 1 -------------------------------------1

0 0 0

0 0 0 1

0 0 1 1 0 0 0 1 1 0 0 0 1 0 1 0 0 1 1 1 1 0 1 1

0 1 1 1 1 0 0 1 0 1 1 0

0 0 0 0 0 1 0 1 0 1 1 1

1 1 1 1

Now, we are ready to prove that the t-error-correcting BCH code just defined indeed has a minimum distance of at least 2t + 1. To prove this, we need to show that no 2t or fewer columns of H given by (6.6) sum to zero (Corollary 3.2.1). Suppose that there exists a nonzero codeword v = (vo, v1, · · · , v11 _1) with weight 8::: 2t. Let Vj 1 , v.12, · · · , v.i, be the nonzero components of v (i.e., v.f 1 = v h = · · · = VJ;; = 1). Using (6.6) and (6.7), we have

a·ii ah ah

= (VJ1 , vh, ...

' V.fs) .

(a2).f1 (a2).h (a2)h

(a21).f1 (a2t)i2 (a2t)h

Section 6.1

Binary Primitive BCH Codes

aji

(ail )2

(aji )21

ah ah

(ah)2

(ah )21 (ah )21

(ah)2

2(1))3

= (1, 1, ... , 1) .

The preceding

the

equality:

(){·ii ah ah

(otjl

)2

(ah )2

(ah )2

(1, 1, . " , 1) .

=ilJJ.

(6.9)

where the second matrix on the left is a 8 x 8 square matrix. To satisfy the equality of (6.9), the determinant of the 8 x 8 matrix must be zero; that

)6

a.ii

(aji )2

(aji

ah

(ah)2 (ah )2

(ahl) (ah )8

ah

=0.

Taking out the common factor from obtain 1 1 1 aU1 +h+·+js) .

1

each row of the foregoing determinant, we (){jl

a(o-l)j1

ah ah

a
=0.

(6.10)

a<S-l).i,

The determinant in the preceding equality is a Vandermonde determinant that is nonzero. Therefore, the product on the left-hand side of (6.10) cannot be zero. This is a contradiction, and hence our assumption that there exists a nonzero codeword w of weight 8 < 2t is invalid. This implies that the minimum weight of the t-error-correcting BCH code defined previously is at least 2t + 1. Consequently, the minimum distance of the code is at least 2t + 1. The parameter 2t + 1 is usually called the designed distance of the t-errorcorrecting BCH code. The true minimum distance of a BCH code may or may not be equal to its designed distance. In many cases the true minimum distance of a

204

Chapter 6

Binary BCH Codes

BCH code is equal to its designed distance; however, there are also cases in which the true minimum distance is greater than the designed distance. Binary BCH codes with length n i= 2111 - 1 can be constructed in the same manner as for the case n = 2111 - 1. Let f3 be an element of order n in the field GF(2 111 ). We know that n is a factor of 2111 - 1. Let g(X) be the binary polynomial of minimum degree that has {3, {32, ... , f32t

as

roots.

Let

W1(X), W2(X), · · ·, W21 (X)

be

the

minimal polynomials

of

f3, {32, ... , {32t, respectively. Then,

g(X) = LCM{W1(X), W2(X), · · ·, W2 1 (X)}. Because {3 = 1, {3, {3 2 , · · · , {3 21 are roots of X" + 1. We see that g(X) is a factor of X 11 + 1. The cyclic code generated by g(X) is at-error-correcting BCH code of length n. In a manner similar to that used for the case n = 2111 - 1, we can prove that the number of parity-check digits of this code is at most mt, and the minimum distance of the code is at least 2t + 1. If f3 is not a primitive element of GF(2 111 ), n i= 2 111 - 1, and the code is called a nonprimitive BCH code. 11

EXAMPLE 6.3

Consider the Galois field GF(2 6 ) given in Table 6.2. The element f3 = a 3 has order n = 21. Lett = 2. Let g(X) be the binary polynomial of minimum degree that has

as roots. The elements {J, {3 2, and {3 4 have the same minimal polynomial, which is W1CX)

= 1 + x + x 2 + x4 + x6 .

The minimal polynomial of {3 3 is

Therefore, g(X) = W1(X)l¥3(X)

= 1 + x + x4 + x 5 + x 7 + x 8 + x 9 . We can check easily that g(X) divides X 21 + 1. The (21, 12) code generated by g(X) is a double-error-correcting nonprimitive BCH code. Now, we give a general definition of binary BCH codes. Let f3 be an element of GF(2m). Let lo be any nonnegative integer. Then, a binary BCH code with designed distance do is generated by the binary polynomial g(X) of minimum degree that has as roots the following consecutive powers of {J:

Section 6.2

Decoding o-f BCH Codes

21QlS

For O < i < do - 1, let \II; (X) and n; be the minimum polynomial and order of {3 10 +;, respectively. Then, ~(X)

= LCM{IJ!o(X), lli1 (X), · · · , \llc10 -2(X)}

and the length of the code is n

= LCM{no, 111, · • · , nc10 -2}.

The BCH code just defined has a minimum distance of at least do and has no more than m(do - 1) parity-check digits (the proof of these is left as an exercise). Of course, the code is capable of correcting L(do - 1)/2J or fewer errors. If we let lo = 1, do = 2t + 1. and f3 be a primitive element of GF(2m), the code becomes a t-error-correcting primitive BCH code of length 2111 -1. If lo = l, do = 2t + 1, and f3 is not a element of GF(2 111 ), the code is a nonprimitive t-error-correcting BCH code of length n that is the order of {3. We note that in the definition of a BCH code with designed distance do, we require that the generator polynomial ~(X) has do - 1 consecutive powers of a field element f3 as roots. This requirement guarantees that the code has a minimum distance of at least do. This lower bound on the minimum distance is called the BCH bound. Any cyclic code whose generator polynomial has cl - 1 consecutive powers of a field element as roots has a minimum distance of at least d. fo the rest of this chapter, we consider only the primitive BCH codes. 16J.2

IDJIE(OrnNG Of IS(~ (OIClllE§

Suppose that a codeword v(X) = vo + v1X + v2X 2 + · · · + v11 _1xn-l is transmitted, and the transmission errors result in the following received vector: ir(X) =

ro

+ r1X + r2X 2 + · · · + r

11

_1X

11 -

1

.

Let e(X) be the error pattern. Then, ir(X)

= v(X) + e(X).

(6.11)

As usual, the first step of decoding a code is to compute the syndrome from the received vector ir(X). For decoding a t-error-correcting primitive BCH code, the syndrome is a 2t-tuple: (6.12) where lHl is given by (6.6). From (6.6) and (6.12) we find that the ith component of the syndrome is S;

= ir(a;) = ro

+ r1a; + r2a 2; + · · · + r

11

_

1 a(n-l)i

(6.13)

for 1 :':c i :':c 2t. Note that the syndrome components are elements in the field GF(2111 ). We can compute from ir(X) as follows. Dividing ir(X) by the minimal polynomial 1/P; (X) of o/, we obtain ir(X) = 2;(X)«/»;(X) + !Tu;(X),

206

Chapter 6

Binary BCH Codes

where b; (X) is the remainder with degree less than that of 'P; (X). Because 'P; (aJ) = 0, we have (6.14) Thus, the syndrome component S; can be obtained by evaluating b; (X) with X = aJ. EXAMPLE 6.4

Consider the double-error-correcting (15, 7) BCH code given in Example 6.1. Suppose that the vector

r=(lOOOOOOOlOOOOOm is received. The corresponding polynomial is r(X)

= 1 + X 8.

The syndrome consists of four components:

The minimal polynomials for a, a 2 , and

a4

are identical, and

The minimal polynomial of a 3 is

Dividing r(X)

= 1 + X 8 by
Dividing r(X) = 1 + x 8 by q, 3 (X) = 1 + X

+ x 2 + X 3 + x4, we have the remainder

Using GF(24 ) given by Table 2.8 and substituting a, a 2 , and a 4 into b 1 (X), we obtain

Substituting a 3 into b3(X), we obtain S3

Thus,

= 1 + a9 = 1 + a + a3 =

a 7.

Decoding of BCH Codes

Section 6.2

21()7

Because ot 1 , ot 2 , ... , ot 21 are roots of each code polynomial, w(oti) = 0 for 1 ::c i ::c 2t. The following u ~ u u , u F between the syndrome and the error pattern follows from (6.11) and (6.13): (6.15) for 1 ::c i pattern e

::c

§ depends on the error has v errors at locations

2t. From (6.15) we see that the Suppose that the error ; that e(X) =

xi + 1

+···+

(6.16)

where O ::c Ji < h < · · · < j,; < n. From (6.15) and (6.16) we obtain the following set of

s1 = oti1 + oth + ... + = s3 = S2

(oti 1)2 -1- (oth )2 -1- ... (oti1 )3

+ (oti, )2

+ (ah )3 + ... -1- (ai, )3

(6.17)

where a ii. ah, · .. , ai are unknown. Any method for solving these equations is a decoding algorithm for the BCH codes. Once we have found ail, ah, · · · , ai,·, the powers Ji, h, · ·. , .iv tell us the error locations in e(X), as in (6.16). In general, the equations of (6.17) have many possible solutions (2k of them). Each solution yields a different error pattern. H the number of errors in the actual error pattern e(X) is tor fewer v ::c t), the solution that an error pattern with the smallest number of errors is the right that the error pattern corresponding to this solution is the most probable error pattern e(X) caused by the channel noise. For large t, solving the equations of (6.17) is difficult and ineffective. In the we describe an effective ai1 for l = 1, 2, · · · , v from the syndrome components S; 's. For convenience, let /31 = ai1 (6.18) 1 •

for 1 ::c l ::c v. We call these elements the error location numbers, since they tell us the locations of the errors. Now, we can express the equations of (6.17) in the following form: S1

= /31 + /32 + · · · + f3v

S2

= /3f -1- /3i + · · -1- /3~ (6.19)

S21=12 /3 21 t- 1321

+ .. + /3~/. V

208

Chapter 6

Binary BCH Codes

These 2t equations are symmetric functions in /31, /32, · · · , f3v, which are known as power-sum symmetric functions. Now, we define the following polynomial: u(X)

6

= (1 + /31X)(l + f32X) · · · (1 + f3vX)

(6.20)

The roots of u (X) are /311 , /3 21, ... , /3;; 1, which are the inverses of the error-location numbers. For this reason, u (X) is called the error-location polynomial.Note that u (X) is an unknown polynomial whose coefficients must be determined. The coefficients of u(X) and the error-location numbers are related by the following equations:

ero = 1

= /3i + /32 + · · · + f3v er2 = /31/32 + /32/33 + · · · + f3v-lf3v er1

(6.21)

The eri 's are known as elementary symmetric functions of /31 's. From (6.19) and (6.21), we see that the er; 's are related to the syndrome components SJ 's. In fact, they are related to the syndrome components by the following Newton's identities:

+ er1 = 0 S2 + er1S1 + 2er2 = 0 S3 + er1S2 + er2S1 + 3er3 = 0 S1

(6.22)

+ er1Sv-l + · · · + erv-1S1 + verv = 0 Sv+l + er1Sv + · · · + erv-1S2 + ervSl = 0 Sv

For the binary case, since 1 + 1 = 2 = 0, we have .

/(Ji=

{ 0 eri

for odd i, for even i.

If it is possible to determine the elementary symmetric functions er1, er2, · · · , er v from the equations of (6.22), the error-location numbers /31, /32, · · · , f3v can be found by

determining the roots of the error-location polynomial o-(X). Again, the equations of (6.22) may have many solutions; however, we want to find the solution that yields au (X) of minimal degree. This u (X) will produce an error pattern with a minimum number of errors. If v :::: t, this o- (X) will give the actual error pattern e(X). Next, we describe a procedure to determine the polynomial u (X) of minimum degree that satisfies the first 2t equations of (6.22) (since we know only S1 through S21 ).

Section 6.3

Iterative Algorithm for Finding the Error-Location Polynomial o- (X)

At this procedure consists of three

for BCH codes. The

the

]_,

21('J~

· · · , S2 1 ) from the received polynomial

rr(X).

Z, Determine the error-location polynomial 1:» (X) from the syndrome S1. S2, · · · , S2 1 • 3,, Determine the error-location numbers

th. th · · · . f3v

finding the roots of

1:»(X), and correct the errors in [((X).

The first algorithm that carries out these three steps was devised by Peterson [3]. 1 and 3 are quite simple; 2 is the most part of uv,,vuun, a BCH code. 15.3

rrlE~AnVIE AlGOml~M IFO~ IF!N[D)!NIG

nu IE~IF«O~-tO«:Ano~, IP'OlVNOM!Al 1:»(X) Here we present Berlekamp's iterative algorithm for finding the error-location We describe only the algorithm, without giving any The reader who is interested in details of this algorithm is referred to Berlekamp [9], Peterson and Weldon [13], and MacWilliams and Sloane [14]. The first of iteration is to find a minimum-degree polynomial 1:»0l(x) whose coefficients satisfy the first Newton's identity of (6.22). The next step is to test whether the coefficients of <»Cl \X) also satisfy the second Newton's of (6.22). H the coefficients of 1:»< 1\X) do satisfy the second Newton's identity of (6.22), we set <»( 21(x)

= u 0 \x).

If the coefficients of 1:»(1\X) do not satisfy the second Newton's identity of (6.22), we add a correction term to 1:»< 1\X) to form 11<2\X) such that u<2\x) has minimum degree and its coefficients satisfy the first two Newton's identities of (6.22). Therefore, at the end of the second step of iteration, we obtain a minimum-degree polynomial 1:»(2 l(X) whose coefficients satisfy the first two Newton's identities of (6.22). The third step of iteration is to find a minimum-degree polynomial 1:»<3\X) from u<2l(X) such that the coefficients of 1:»( 3 l(X) satisfy the first three Newton's identities of ( 6.22). Again, we test whether the coefficients of 1:»< 2 l (X) satisfy the third Newton's identity of (6.22). H they do, we set 1:»< 3l(X) = 1:»(21(X). If they do not, we add a correction term to u< 2l (X) to form 1:»(3l (X). Iteration continues until we obtain 1:»<21 l(X). Then, 1:»<21 l(X) is taken to be the error-location polynomial 1:»(X), that 1:»(X) = l:»< 21 \X).

This <» (X) will an error pattern (l';(X) of minimum weight that satisfies the equations of (6.17). If the number of errors in the received polynomial ll'(X) is tor less, then 1:»(X) produces the true error pattern. Let IY(µ,>cx) = 1 + X + x 2 + · ·· + x 1n (6.23)

u?~)

u?,)

ui;~)

be the minimum-degree polynomial determined at the µ"th step of iteration whose coefficients satisfy the first fJ Newton's identities of (6.22). To determine 1:»({~+ 1l(x),

:no

Chapter 6

Binary BCH Codes

we compute the following quantity:

=

d /1,

S µ,+l

+ (Tl(µ,ls /1, + 0'2(µ,ls 11-l + ... + 0'11,<µ,Js µ+l-11,

(6.24)

This quantity dµ, is called the µ,th discrepancy. If dµ = 0, the coefficients of u
-1- 0, the coefficients of u<µ,) (X) do not satisfy the (µ, + l)th Newton's identity, and we must add a correction term to u<11l(X) to obtain u<11+1l(X). To make this correction, we go back to the steps prior to the µ,th step and determine a polynomial uCPl(X) such that the pth discrepancy dp -1- 0, and p - lp [Ip is the degree of u
If dµ,

(6.25)

which is the minimum-degree polynomial whose coefficients satisfy the first µ, + 1 Newton's identities. The proof of this is quite complicated and is omitted from this introductory book. To carry out the iteration of finding u(X), we begin with Table 6.5 and proceed to fill out the table, where lµ is the degree of u<µ) (X). Assuming that we have filled out all rows up to and including the µ,th row, we fill out the (µ, + l)th row as follows:

1. If dµ, = 0, then u<µ+ll(X) = o-
In either case, d µ,+l

=

S /L+2

(11+1)

+ (Tl

Sµ+l

(µ,+l)

+···+(Tl1.1+[

Sµ+2-l {J. +1,

(6.27)

where the cr;(µ+l),s are the coefficients of u<µ+l\X). The polynomial u< 21 l(X) in the last row should be the required u (X). If its degree is greater than t, there are more 6.5: Berlekamp's iterative procedure for finding the error-location polynomial of a BCH code.

TABLE

µ

u
d/l

l/l

µ - l/l

-1 0 1

1 1

1

0 0

-1 0

2

2t

S1

Section 6.3

Iterative Algorithm for

the Error-lorntion Polynomial a(X)

Zn

than t errors in the received polynomial rc(X), and generaHy it is not possible to locate them.

Consider the 5) triple-error-correcting BCH code given in Example 6.1. Assume that the codeword of all zeros,

w = (0, 0, 0, 0, 0, 0, 0, 0, 0, 0. 0, 0, 0, 0, 0), is transmitted, and the vector 1r

is received. rc(X) are identical, and

=

=

(0 0 0 1 0 1 0 0 0 0 0 0 1 0 0)

X 3 + X 5 + X 12 . The minimal polynomials for

¢>1 (X)

ot, 01

2, and

01

4

= q;2(X) = qp4(X) = 1 + X + X 4 .

The elements a 3 and a 6 have the same minimal polynomial,

+ x3 + x4. The minimal polynomial for a 5 is

+ X 2.

qp5(X) = 1 + X

Dividing 1r(X) by q/) 1 (X), qp 3(X), and qp 5 (X), respectively, we obtain the following remainders: ll:»1 (X) lbi3(X) lbis(X)

Using Table 2.8 and substituting syndrome components:

2 ot, a ,

S1

Substituting

a3

and

6 01

= 1, = 1 + x2 + x3• = X2. and

= S2 =

a4

S4

(6.28)

into lbi 1 (X), we obtain the following

= 1.

into lbi3(X) . we obtain S3

= 1 + ot6 + a9 = a10,

s6 = 1 + a12 + a1s =

()is.

(6.29)

Substituting a 5 into lbJ 5 (X), we have

Using the iterative procedure described previously, we obtain Table 6.6. Thus, the error-location polynomial is

212

Binary BCH Codes

Chapter 6

TABLE 6.6: Steps for finding the error-location polynomial of

the (15,5) BCH code given in Example 6.5. u
µ

-1 0 1 2 3 4

5 6

1 1

l+X l+X 1 + X +asx 2 1 + X +a 5X 2 1 + X +a 5X 3 1 + X +asx 3

d/l

l /l

µ,-l/l

1 1 0

0 0 1 1 2 2 3

-1 0 0 (take p = -1) 1 1 (take p = 0) 2 2 (take p = 2)

as 0

alO 0

We can easily check that a 3 , a 10 , and a 12 are the roots of u(X). Their inverses are a 12 , as, and a 3 , which are the error-location numbers. Therefore. the error pattern is e(X)

= x 3 + xs + x 12 .

Adding e(X) to the received polynomial r(X), we obtain the all-zero vector. If the number of errors in the received polynomial r(X) is less than the designed error-correcting capability t of the code, it is not necessary to carry out the 2t steps of iteration to find the error-location polynomial u(X). Let uCf.ll(X) and dµ, be the solution and discrepancy obtained at the µ,th step of iteration. Let lp. be the degree of uCf.ll (X). Chen [19] has shown that if dµ, and the discrepancies at the next t - lµ, -1 steps are all zero, u<µ,J (X) is the error-location polynomial. Therefore, if the number of errors in the received polynomial r(X) is v(v ::: t), only t + v steps of iteration are needed to determine the error-location polynomial u(X). If v is small (this is often the case), the reduction in the number of iteration steps results in an increase in decoding speed. The described iterative algorithm for finding u(X) applies not only to binary BCH codes but also to nonbinary BCH codes.

6.4

SIMPUFIIED ITERATIVE ALGORHHM FOIR FINDING THE IERROR-lOCATION POlYNOMiAL u(X)

The described iterative algorithm for finding u(X) applies to both binary and nonbinary BCH codes, including Reed-Solomon codes; however, for binary BCH codes, this algorithm can be simplified to t-steps for computing u (X). Recall that for a polynomial f (X) over GF(2),

f 2 (X) = f

2

(X ),

[see (2.10)]. Because the received polynomial r(X) is a polynomial over GF(2), we have

Simplified Iterative Algorithm

Section 6A

o/ for

X in the preceding

2B

we obtain (6.30)

[see

we obtain the

bet·ween (6.31)

holds. Then,

of

This result says that (6.32) It follows from

and (6.32) that ( 6.33)

=0.

The foregoing equality is simply the l"fowton's second This result says that if the first Newton's holds, then the second Newton's also holds. suppose the first and the third i\Jewton's identities of that S1

+ cr1

= 0,

(6.34) (6.35)

The equality of

implies that the second Newton's identity holds: (6.36)

Then,

2

S2

i:;2 + al2 "1 =

0.

(6.37)

+ cr12 S2' = 0.

(6.38)

H follows from (6.31) that (6Ti) becomes

Multiplying both sides of (6.35) by o·1 .. we obtain (6.39)

Adding (6.38) and (6.39) and using the equalities of (6.31) and (6.32). we find that the fourth Newton's identity holds:

214

Chapter 6

Binary BCH Codes

This result says that if the first and the third Newton's identities hold, the second and the fourth Newton's identities also hold. With some effort, it is possible to prove that if the first, third, ... , (2t - l)th Newton's identities hold, then the second, fourth, · · · , 2tth Newton's identities also hold. This implies that with the iterative algorithm for finding the error-location polynomial a-(X), the solution o-< 2fl-ll(X) at the (2µ - l)th step of iteration is also the solution o-< 21.,l (X) at the 2µth step of iteration; that is, (6.40)

This suggests that the (2µ - l)th and the 2µth steps of iteration can be combined. As a result, the foregoing iterative algorithm for finding o-(X) can be reduced to t steps. Only the even steps are needed. The simplified algorithm can be carried out by filling out a table with only t rows, as illustrated in Table 6.7. Assuming that we have filled out all rows up to and including the µth row, we fill out the (µ + l)th row as follows: 1. If d/L = 0, then 0-<1.,+ll(X) = 0-<1.,l(x). 2. If d/L i= 0, we find another row preceding the µth row, say the pth, such that the number 2p - lp in the last column is as large as possible and dp i= 0. Then, (6.41)

In either case, l!L+l is exactly the degree of o-C!L+ll(X), and the discrepancy at the (µ + l)th step is d/L+l

=

S2/L+3

(/L+l)

+ a1

S2/L+2

(/L+l)

+ a2

S2/L+l

(!,l+l)

+ ··· + a1

11+!

S2/L+3-l fl +t.

(6.42)

The polynomial 0-< 1\X) in the last row should be the required o-(X). If its degree is greater than t, there were more than t errors, and generally it is not possible to locate them. The computation required in this simplified algorithm is half of the computation required in the general algorithm; however, we must remember that the simplified algorithm applies only to binary BCH codes. Again, if the number of errors in the 6.7: A simplified Berlekamp iterative procedure for finding the error-location polynomial of a binary BCH code.

TABLE

2µ - lµ,

-1 0

Section 6.5

Finding the Error-location Numbers and Error Correction

for the error-location binary BCH code given in Example 6.6.

of the

TABLE 6.8:

1

-2

0 1 2

3

1 l l+S1X=l+X 1 + X + ot 5 X 2 1 + X + a 5X 3

1 S1 = 1 S3 + S2S1 = ot 5 (¥10

215

-1

0 0 1 2

0

1 (take p = -~) 2 (take p = 0) 3 (take p = 2)

3

received ir(X) is less than t, it is not necessary to carry out the t steps of iteration to determine u (X) for at-error-correcting binary BCH code. Based on Chen's result if for someµ,, d1,_ and the discrepancies at the next 1(t - l µ - 1) /21 steps of iteration are zero, (FC 11 \X) is the error-location polynomial. H the number of errors in the received polynomial is v ( v :::: t), only 1(t + v) /21 steps of iteration are needed to determine the error-location polynomial or (X). IEKAMif"UE !ED.©

The simplified table for finding u(X) for the code considered in Example 6.5 is given in Table 6.8. Thus, Cf (X) = u< 3l (X) = 1 + X + a 5 X 3, which is identical to the solution found in Example 6.5.

The last a BCH code is to find the error-location numbers that are the of the roots of Cf (X). The roots of ff (X) can be found simply substituting 1. ot, a 2 • · · · , a 11 - 1 (11 = 2111 - 1) into ll"(X). Since ot 11 = l, a- 1 = 0! 11 - 1• Therefore, if o/ is a root of Cf (X). 01 11 - 1 is an error-location number, and the received digit r11 _ 1 is an erroneous digit. Consider Example 6.6. where the error-location was found to be H~HH<-H

ll"(X) = 1 + X

+a 5 X 3 .

By substituting L 0/, 01 2. · · · . 01 14 into ll"(X). we find that a 3 , 01 10 . and a 12 are roots of u (X). Therefore, the error-location numbers are a l 2 • a 5 • and 01 3 . The error pattern is e(X)

= x3 + xs + x12.

which is exactly the assumed error pattern. The decoding of the code is completed by adding (modulo-2) e(X) to the received vector !l"(X). The described substitution method for finding the roots of the error-location polynomial was first used by Peterson in his algorithm for decoding BCH codes [3]. Later, Chien [6] formulated a procedure for carrying out the substitution and error correction. Chien's procedure for searching for error-location numbers is described next. The received vector Il"(X) = ro + r1X + r2X 2 + · · · + r _1 xn-l 11

216

Chapter 6

Binary BCH Codes

is decoded bit by bit. The high-order bits are decoded first. To decode r11 _ 1, the decoder tests whether a 11 - 1 is an error-location number; this is equivalent to testing whether its inverse, a, is a root of a- (X). If a is a root, then 1 + er1a

+ er2a 2 + · · · + ervav = 0.

Therefore, to decode r11 _ 1, the decoder forms er1 a, er2a2, · · · , erva v. If the sum 1 + er1a + er2a 2 + · · · , ervav = 0, then an-l is an error-location number, and r11 _1 is an erroneous digit; otherwise, r11 _1 is a correct digit. To decode r11 _1, the decoder forms er1 a 1, er2a 21 , · · · , erv
If this sum is 0, then a 1 is a root of a (X), and r 11 _1 is an erroneous digit; otherwise,

r11 _z is a correct digit. The described testing procedure for error locations can be implemented in a straightforward manner by a circuit such as that shown in Figure 6.1 [6]. The t er-registers are initially stored with er1, er2, · · · , er, calculated in step 2 of the decoding (erv+l = erv+2 = · · · = er, = 0 for v < t). Immediately before r 11 _1 is read out of the buffer, the t multipliers® are pulsed once. The multiplications are performed, and er1 a, er2a 2, · · · , erva v are stored in the er-registers. The output of the logic circuit A is 1 if and only if the sum 1 + er1a + er2a 2 + · · · + erva" = O; otherwise, the output of A is 0. The digit r11 _1 is read out of the buffer and corrected by the output of A. Once r 11 _1 is decoded, the t multipliers are pulsed again. Now, er1a 2, er2a4. · · · , erva 2v are stored in the er-registers. The sum

is tested for 0. The digit r11 _2 is read out of the buffer and corrected in the same manner as r 11 _1 was corrected. This process continues until the whole received vector is read out of the buffer.

A cc'

a'

Input Buffer

FIGURE 6.1:

Cyclic error location search unit.

Section 6.6

Correction of Errors and Erasures

2H

The described decoding algorithm also applies to nonprimitive BCH codes. The 2t syndrome components are given by

for 1 ::: i ::: 2t.

H the channel is the binary symmetric erasure channel as shown in Figure l.6(b ), the received vector may contain both errors and erasures. H was shown in Section 3.4, that a code with minimum distance dmin is capable of correcting all combinations of v errors and e erasures provided that 2v

+e +1

:S dmin·

(6.43)

Erasure and error correction with binary BCH codes are quite simple. Suppose a BCH code is designed to correct t errors, and the received polynomial Il'(X) contains v (unknown) random errors and e (known) erasures. The decoding can be accomplished in two steps. First, the erased positions are replaced with O's and the resulting vector is decoded using the standard BCH decoding algorithm. Next, thee erased positions are replaced with 1 's, and the resulting vector is decoded in the same manner. The decodings result in two codewords. The codeword with the smallest number of errors corrected outside the e erased positions is chosen as the decoded codeword. H the inequality of (6.43) holds, this decoding algorithm results in correct decoding. To see this, we write (6.43) in terms of the error-correcting capability t of the code as (6.44) v + e/2::: t. Assume that when e erasures are replaced with O's, e* s e /2 errors are introduced in those e erased positions. As a result, the resulting vector contains a total of v + e* ::: t errors that are guaranteed to be correctable if the inequality of (6.44) ( or (6.43)) holds; however, if e* > e /2 then only e - e* < e /2 errors are introduced when the e erasures are replaced with 1's. In this case the resultant vector contains v + (e -e*) < terrors. Such errors are also correctable. Therefore, with the described decoding algorithm, at least one of the two decoded codewords is correct.

Decoding of BCH codes requires computations using Galois field arithmetic. Galois field arithmetic can be implemented more easily than ordinary arithmetic because there are no carries. fo this Section we discuss circuits that perform addition and multiplication over a Galois field. For simplicity, we consider the arithmetic over the Galois field GF(2 4 ) given by Table 2.8. To add two field elements, we simply add their vector representations. The resultant vector is then the vector representation of the sum of the two field elements. For example, we want to add a 7 and a 13 of GF(24 ). From Table 2.8 we find that their vector representations are (1 1 0 1) and (1 0 1 1), respectively. Their vector sum is (1 1 0 1) + (1 0 11) = (0 11 0), which is the vector representation of a 5 .

218

Chapter 6

Binary BCH Codes

ADD

Register A (accumulator)

FIGURE 6.2:

Galois field adder.

Two field elements can be added with the circuit shown in Figure 6.2. First, the vector representations of the two elements to be added are loaded into registers A and B. Their vector sum then appears at the inputs of register A. When register A is pulsed (or clocked), the sum is loaded into register A (register A serves as an accumulator). For multiplication, we first consider multiplying a field element by a fixed element from the same field. Suppose that we want to multiply a field element f3 in GF(24 ) by the primitive element a whose minimal polynomial is (X) = 1 + X + X4 . We can express element f3 as a polynomial in a as follows: f3

= bo + b1 a + b2a 2 + b3a 3.

Multiplying both sides of this equality by a and using the fact that a 4 = 1 + a, we obtain the following equality: af3

= b3 + (bo + b3)a + b1 a 2 + b2a 3.

This multiplication can be carried out by the feedback shift register shown in Figure 6.3. First, the vector representation (bo, b1, b2, b3) of f3 is loaded into the register, then the register is pulsed. The new contents in the register form the

FIGURE 6.3:

Circuit for multiplying an arbitrary element in GF(24) by a.

Section 6.7

o'f Galois Field Arithmetic

21£1

vector representation of af3. For let f3 = a 7 = 1 +a+ a 3 . The vector "'""ucauvu of f3 is (1 1 0 We load this vector into the of the circuit shown in Figure 6.3. After the the new contents in the register win be (101 0), which of ot7 and a. The circuit shown in Figure 6.3 can be used to all the nonzero elements of GF(2 4 ). First, we load (1 0 0 0) of a 0 = 1) into the register. Successive shifts of the will generate vector of successive powers of a, the same order as appear in Table 2.8. At the end of the fifteenth will contain (1 0 0 0) As another suppose that we want to devise a circuit to ,cn,1-r,.,,., element f3 of the element a 3 . we express p in form:

a 3 , we have

both sides of the a 3 18

= boo/'° + + b3a 6 = boot 3 + b1 (1 +a)+ b2(a + =0 b1

+ (b1-+

+

+ b3(ot2 + a 3) + b3)a 2 + (bo + b3)a 3.

Based on the we obtain the circuit shown in 6.4, which of nmltiplying any element p in a 3 . To we first load the vector (bo. b1, b2, b3) of p into the register, then we the ,.co .. w,,,,.. The new contents in the register will be the vector representation of a 3 p. Next we consider two arbitrary field elements. Again, we use GF(2 4 ) for illustration. Let 16 and y be two elements in VVe express these two elements m form: is

+ b3or. 3 , JI

Then, we can express the

PY

= co + c1 a + f3y in the

= (((qp)a

+ C30!3. form:

+ c2p)a + c1p)(.X + cop

(6.45)

FIGURE 6.4: Circuit for multiplying an arbitrary element in GF(2 4 ) by a 3 .

220

Chapter 6

Binary BCH Codes

This product can be evaluated with the following steps: 1. Multiply qf3 by a and add the product to c2f3. 2. Multiply (c3/3)a + c2f3 by a and add the product to q/3. 3. Multiply ((q/3)a + c2f3)a + ci/3 by a and add the product to co/3. Multiplication by a can be carried out by the circuit shown in Figure 6.3. This circuit can be modified to carry out the computation given by (6.45). The resultant circuit is shown in Figure 6.5. In operation of this circuit, the feedback shift register A is initially empty, and (bo, b1, b2, b3) and (co, q, c2, c3), the vector representations of f3 and y, are loaded into registers B and C, respectively. Then, registers A and C are shifted four times. At the end of the first shift, register A contains (qbo, c3b1, c3b2, c3hJ), the vector representation of c3/3. At the end of the second shift, register A contains the vector representation of (c3/3)a + c2/3. At the end of the third shift, register A contains the vector representation of ((c3/3)a + c2/3)a + q/3. At the end of the fourth shift, register A contains the product f3y in vector form. If we express the product f3y in the form f3y

=

(((co/3)

+ q/3a) + c2f3a 2) + qf3a3,

we obtain a different multiplication circuit, as shown in Figure 6.6. To perform the multiplication, f3 and y are loaded into registers B and C, respectively, and register A is initially empty. Then, registers A, B, and Care shifted four times. At the end of

Register A

Register C

FIGURE 6.5: Circuit for multiplying two elements of GF(2 4 ).

Section 6.7

Implementation of Galois Field Arithmetic

221

- - . .. 1 Keg1ster C

I

i&] ~-]· 1-©"' ,0" I Lo- lo. L-u-l I~ Register A

FIGURE 6.6: Another circuit for multiplying two elements of GF(2 4 ).

the fourth register A holds the product f3y. Both multiplication circuits shown in Figures 6.5 and 6.6 are of the same complexity and require the same amount of computation time. Two elements from can be multiplied with a combinational logic circuit with 2m and m outputs. The advantage of this implementation is its however, for m, it becomes prohibitively complex and costly. Multiplication can also be programmed in a general-purpose computer; it requires roughly Sm instruction executions. Let rr(X) be a over GF(2). We consider now how to compute Ir(o/). This type of computation is required in the first step of decoding of a BCH code. H can be done with a circuit for multiplying a field element by o/ in GF(2 111 ). Again. we use computation over GF(2 4 ) for illustration. Suppose that we want to compute rc(a)=ro+r1a+r2a 2 +···+ri4a 14 ,

(6.46)

where a is a primitive element in GF(24 ) given by Table 2.8. We can express the right-hand side of (6.46) in the form ir(a)

= (· · · (((r14)a +

r13)a + r12)a +···)a+ ro.

Then, we can compute Ir(a) by adding an input to the circuit for multiplying by a shown in Figure 6.3. The resultant circuit for computing r(a) is shown in Figure 6.7. In operation of this circuit. the register is initially empty. The vector (ro, r1. · · · . r14) is shifted into the circuit one digit at a time. After the first shift. the register contains (r14, O. O. 0). At the end of the second shift, the register contains the vector representation of r14a + r13. At the completion of the third shift. the

222

Chapter 6

Binary BCH Codes

r(X) Input FIGURE 6.7:

Circuit for computing r(a).

FIGURE 6.8:

Circuit for computing r(a 3 ).

r(X) Input

register contains the vector representation of (r14a + r13)a + r12- When the last digit ro is shifted into the circuit, the register contains r(a) in vector form. Similarly, we can compute r(a 3) by adding an input to the circuit for multiplying by a 3 of Figure 6.4. The resultant circuit for computing r(a 3 ) is shown in Figure 6.8. There is another way of computing r(ai). Let; (X) be the minimal polynomial of a;. Let b(X) be the remainder resulting from dividing r(X) by; (X). Then, r(a;)

= b(ai).

Thus, computing r(a;) is equivalent to computing b(a;). A circuit can be devised to compute b(a;). For illustration, we again consider computation over GF(24 ). Suppose that we want to compute r(a 3 ). The minimal polynomial of a 3 is cp 3 (X) = 1 + X + x 2 + X 3 + x 4 . The remainder resulting from dividing r(X) by q, 3 (X) has the form Then, r(a 3 ) = b(a 3 )

= bo + b1a 3 + b2a 6 + b3a 9 = bo + b1a 3 + b2(a 2 + a 3 ) + b3(a + a 3 )

(6.47)

= bo + b3a + b2a 2 + (b1 + b2 + b3)a 3 . From the preceding expression we see that r(a 3 ) can be computed by using a circuit that divides r(X) by 3 (X) = 1 + X + x 2 + X 3 + X 4 and then combining

Section 6.7

Implementation of Galois Field Arithmetic

223

the coefficients of the remainder Il:D(X) as given by (6.47). Such a circuit is shown in Figure 6.9, where the feedback connection of the shift register is based on q; 3 (X) = 1 + X + x 2 + x 3 + X4 . Because a6 is a conjugate of a3, it has the same minimal polynomial as a 3 , and therefore ir(a 6 ) can be computed from the same remainder lbJ(X) resulting from dividing !r(X) by ip 3 (X). To form rc(a 6 ). we combine

(

~ 'j

FIGURE 6.9: Another circuit for computing rr(a 3 ) in GF(2 4 ).

FIGURE 6.10: Circuit for computing rr(a 3 ) and n-(a 6) in GF(2 4 ).

224

Chapter 6

Binary BCH Codes

the coefficients of b(X) in the following manner: r(ci)

= lb(ci) = bo + b1a 6 + b2a 12 + b3a 18 = bo + b1 (a 2 + a 3 ) + b2 (1 + a + a 2 + a 3 ) + b3a 3 = (bo

+ b2) + b2a + (b1 + b2)a 2 + (b1 + b2 + b3)a 3 .

The combined circuit for computing r(a 3 ) and r(a 6 ) is shown in Figure 6.10. The arithmetic operation of division over GF(2111 ) can be performed by first forming the multiplicative inverse of the divisor f3 and then multiplying this inverse 13- 1 by the dividend, thus forming the quotient. The multiplicative inverse of f3 can be found by using the fact that {3 2,,, -l = 1. Thus,

6.8

IMPlEMIENTATION Of !ERROR CORRECTION

Each step in the decoding of a BCH code can be implemented either by digital hardware or by software. Each implementation has certain advantages. We consider these implementations next. 6.8.1

Syndrome Computations

The first step in decoding a t-error-correction BCH code is to compute the 2t syndrome components S1, S2, · · · , S2 1 . these syndrome components may be obtained by substituting the field elements a, a 2 , ... , a 21 into the received polynomial r(X). For software implementation, ai into r(X) is best substituted as follows: S;

= r(a;) = rn-1(ai)n-l + rn-2(a;)',- 2 + · · · + r1a; + ro = (· · · ((r

11

_1a;

+r

11

_2)ai

+r

11

_3)ai

+ · · · + r1)a; + ro.

This computation takes n - 1 additions and n - 1 multiplications. For binary BCH codes, we have shown that S2; = sf. With this equality, the 2t syndrome components can be computed with (n - l)t additions and nt multiplications. For hardware implementation, the syndrome components may be computed with feedback shift registers as described in Section 6.7. We may use either the type of circuits shown in Figures 6.7 and 6.8 or the type of circuit shown in Figure 6.10. The second type of circuit is simpler. From the expression of ( 6.3), we see that the generator polynomial is a product of at most t minimal polynomials. Therefore, at most t feedback shift registers, each consisting of at most m stages, are needed to form the 2t syndrome components. The computation is performed as the received polynomial r(X) enters the decoder. As soon as the entire r(X) has entered the decoder, the 2t syndrome components are formed. It takes n clock cycles to complete the computation. A syndrome computation circuit for the double-error-correcting (15, 7) BCH code is shown in Figure 6.11, where two feedback shift registers, each with four stages, are employed. The advantage of hardware implementation of syndrome computation is speed; however, software implementation is less expensive.

Section 6.8 ir(X) - - a - - - --~-------~

Input

·· >

of Error Correction

225

15-bit buffer register

FIGURE 6.11: Syndrome computation circuit for the double-error-correcting (15, 7)

BCH code. 15.~.2

1Fur11«Jlun1gi ti'ilie IEirrmir-l«:ll«:arltkm ~(Q)i)Jrrn«:llmizii

(i

(X)

For this step the software computation requires somewhat fewer than t additions and t multiplications to compute each if<µ\X) and each d1.l, and since there are t of each, the total is roughly 2t 2 additions and 2t 2 multiplications. A pure hardware implementation requires the same total, and the speed depends on how much is done in parallel. The type of circuit shown in Figure 6.2 may be used for addition, and the type of circuits shown in Figures 6.5 and 6.6 may be used for multiplications. A very fast hardware implementation of finding if(X) would probably be very expensive, whereas a simple hardware implementation would probably be organized much like a general-purpose computer, except with a wired rather than a stored program. 15.~.3

Cmt'il(QlilJlitsr\ti«:imi «:llf lEirirm·-l«:ll«:
In the worst case, this step requires substituting n field elements into an error-location polynomial if (X) of degree t to determine its roots. In software this requires nt multiplications and nt additions. This step can also be performed in hardware using Chien's

226

Chapter 6

Binary BCH Codes

searching circuit, shown in Figure 6.1. Chien's searching circuit requires t multipliers for multiplying by a, a 2 , ... , al, respectively. These multipliers may be the type of circuits shown in Figures 6.3 and 6.4. Initially, a1, a2, · · · , a 1 found in step 2 are loaded into the registers of the t multipliers. Then, these multipliers are shifted n times. At the end of the /th shift, the t registers contain a1 ot1, a2a 21 , · · · , a 1a 11 . Then, the sum

is tested. If the sum is zero, an~! is an error-location number; otherwise, a 11 ~ 1 is not an error-location number. This sum can be formed by using t m-input modulo-2 adders. An m-input OR gate is used to test whether the sum is zero. It takes n clock cycles to complete this step. If we want to correct only the message digits, only k clock cycles are needed. A Chien's searching circuit for the double-error-correcting (15, 6) BCH code is shown in Figure 6.12. For large t and m, the cost for building t wired multipliers for multiplying a. a 2 , ... , a 1 in one clock cycle becomes substantial. For more economical but Output

15-bit buffer register r(X) Multiplying by a initially load with cr 1

+ +

Multiplying by a 2 initially load with cr 2

FIGURE 6.12:

code.

Chien's searching circuit for the double-error-correcting (15, 7) BCH

Distribution and Error Detection of Binary BCH Codes

Section 6.9

22221

slower multipliers, we may use the of circuit shown in Figure 6.5 ( or shown in Figure 6.6). a; is loaded into register B, and o/ is stored in register C. After m dock cycles, the product CJ;o/ is in register A. To form a;a 2i. a;o/ is loaded into register B. After another m dock cydes, a;a 2; will be in register A. Using this of multiplier, nm dock are needed to complete the third step of BCH code. Steps 1 and 3 involve roughly the same amount of computation. Because n is generally much larger than t, 4n t is much larger than 4t 2 , and steps 1 and 3 involve most of the hardware implementation of these is essential 1 can be if high decoding speed is needed. With hardware implementation, done as the received polynomial IT'(X) is read in, and step 3 can be accomplished as r(X) is read ouL fo this case the computation time required in steps 1 and 3 is negligible. lrll.~

WIE!lGHl IDJ!:SnmillUJl!ON

t~,N[)J

!E~~o~ [DJ!Elf!EG!ON Of 18l!NAIRV 18l(H (O[ll!E§

The distributions of double-error-correcting, triple-error-correcting, and some low-rate primitive BCH codes have been completely determined; however, the weight distributions for the other BCH codes are still unknown. The weight distribution of a double-error·-correcting or a triple-error-correcting primitive BCH code can be determined by first computing the weight distribution of its dual code and then applying the Mac Williams identity of (3.32). The weight distribution of the dual of a double-error-correcting primitive BCH code of length 2111 - 1 is given in TABLE 6.9: ·weight distribution of the dual of a doubleerror-correcting primitive binary BCH code of length 2111 - 1.

([])«Jl«llm :::3

0 2111-l _ 2<m+lJ/2-l 2111-l 2111-l + 2<111+1)/2-1

1

[2111-2 111

+ 2<111-1)/2-1 ](2111 2111 - 1 + 1)(2111 -

- 1)

(2 1) [2111-2 - 2<111-1)/2-1 ](2111 -- 1)

TABLE 6.10: Weight distribution of the dual of a double-error-correcting primitive binary BCH code of length 2111 - 1.

0 2111-l - 2<111+2)/2-1 2111-l - 2111/2-1 2111-l 2111-l + 2111/2-1 2111-l + 2<111+2)/2-1

1

2<111-2)/2-1[2(111-2)/2 + 1](2/Jl 2<111+2)/2-1(2111/2 + 1)(2111 c2111-2 + l)c2111 - 1) 2<111+2)/2-1(2111/2 - 1)(2"' 2<111-2)/2-1[2<111-2)/2 - 1](2"'

- 1)/3 1)/3 1)/3 - 1)/3

228

Chapter 6

Binary BCH Codes

Weight distribution of the dual of a triple-errorcorrecting primitive binary BCH code of length 2 111 - 1.

TABLE 6.11:

Odd m 2:: 5 Weight, i 0 2111-l - 2<111+1)/2 2m-l _ 2(m-l)/2

2111-l 2m-l 2m-l

+ 2(111-1)/2 + 2(111+1)/2

Number ofVedorn with weight i, Bi 1

+ 1](2111-l -1)(2,n -1)/3 2<111-3)/2[2<111-1)/2 + 1](5. 2111-l + 4)(2117 - 1)/3 (9. 22111-4 + 3. 2111-3 + 1)(2117 - 1) 2(111-3)/2[2(111-1)/2 -1](5. 2m-l + 4)(2111 -1)/3 2(m-5)/2[2(m-3)/2

2<111-5)/2[2<111-3)/2 -1](2111-l -1)(2111 - 1)/3

Weight distribution of the dual of a triple-errorcorrecting primitive binary BCH code of length 2111 1.

TABLE 6.12:

Even m 2:: 6 Weight, i 0 2111-l - 2<111+4)/2-1 2111-l - 2<111+2)/2-1

2111-l - 2111/2-1 2111-l 2111-l + 2111/2-1 2111-l + 2(111+2)/2-l 2111-l

+ 2<111+4)/2-1

Number of vectors with weight i, Bi 1

+ 2(111+4)/2-1](2111 4)(2111 -1)/960 7[2m-1 + icm+2)/2-1]2m (2m -1)/48 2(2111 - 1 + 2111 / 2- 1)(3. 2 + 8)(2 1)/15 2111 111 (29 · 2 - 4 · 2m + 64)(2 - 1)/64 2(2111 - 1 - 2111 / 2- 1)(3. 2111 + 8)(2111 -1)/15

[2111-l

111

111

7[2111-l - 2<111+2)/2-1 ]2111 (2111 - 1)/48 [2111-l - 2<111+4)/2-1](2/11 - 4)(2117 - 1)/960

Tables 6.9 and 6.10. The weight distribution of the dual of a triple-error-correcting primitive BCH code is given in Tables 6.11 and 6.12. Results presented in Tables 6.9 to 6.11 were mainly derived by Kasami [22]. For more on the weight distribution of primitive binary BCH codes, the reader is referred to [9] and [22]. If a double-error-correcting or a triple-error-correcting primitive BCH code is used for error detection on a BSC with transition probability p, its probability of an undetected error can be computed from (3.36) and one of the weight distribution tables, Tables 6.9 to 6.12. It has been proved [23] that the probability of an undetected error, Pu (E), for a double-error-correcting primitive BCH code of length 2111 - 1 is upper bounded by 2- 2111 for p s where 2m is the number of parity-check digits in the code. The probability of an undetected error for a triple-error-correcting primitive BCH code of length 2m - 1 with odd m satisfies the upper bound 2- 3111 , where 3m is the number of parity-check digits in the code [24]. It would be interesting to know how a general t-error-correcting primitive BCH code performs when it is used for error detection on a BSC with transition probability p. It has been proved [25] that for at-error-correcting primitive BCH of

!,

Section 6.9

Weight Distribution and Error Detection of Binary BCH Codes

22§l!

length 2m - 1, if the number of parity-check digits is equal to mt, and mis greater than a certain constant mo(t), the number of codewords of weight i satisfies the following equalities:

A;~ {

;l+Ao-n 1110) (;

)2

for O < i ::=: 2t (,-kl

for i > 2t,

(6.48)

where n = 2111 - 1, and Ao is upper bounded by a constant. From (3.19) and (6.48), we obtain the following expression for the probability of an undetected error: Pu(E) = (1 +Ao. n-l/lO)r(n-k)

t (7 ) /

(1 - p)n-i.

(6.49)

i=2t+l

Let c

= (2t + 1)/n. Then the summation of (6.49) can be upper bounded as follows

it (;)

[13]:

p1 (1

provided that p <

£,

_ p)n-i :'S rnE(c,p)

(6.50)

where E(c, p) = H(p) H(x)

+ (c -

= -x log2 x

and H'(x)

p)H' (p) - H(c)

- (1 - x) log2 (1 - x),

1-x

= log2 - - . X

E(c, p) is positive for c > p. Combining (6.49) and (6.50), we obtain the following upper bound on Pu (E) for c > p: Pu(E) :S (1 +Ao. n-l/lO)r"E(c,p)2-<11-k)

For p < c and sufficient large n, Pu (E) can be made very small. For p :::: c, we can also derive a bound on Pu (E). His clear from (6.49) that Pu(E):S(l+Ao·n-1/10)2-(11-k)t(

7 )pi(l-p)n-i_

i=O

Because

we obtain the following upper bound on Pu(£): P,, (£) :S (1

+ Ao · n -l/lO)r
(6.51)

We see that for p 2:: £, Pu still decreases exponentially with the number of paritycheck digits, n - k. If we use a sufficiently large number of parity-check digits, the probability of an undetected error Pu(E) will become very small. Now, we may

230

Chapter 6

Binary BCH Codes

summarize the preceding results above as follows: For at-error-correcting primitive BCH code of length n = 2111 - 1 with number of parity-check digits n - k = mt and m :=:: mo(t), its probability of an undetected error on a BSC with transition probability p satisfies the following bounds: p (E) < u

-

( 1 + A . n-1/10)2-n[l-R+E(e,p)] o { (1 + AO . n-1/10)2-n(l-R)

for p < £ for p :=:: £

(6.52)

where£= (2t + 1)/n, R = k/n, and AO is a constant. The foregoing analysis indicates that primitive BCH codes are very effective for error detection on a BSC. 6.10

REMARKS

BCH codes form a subclass of a very special class of linear codes known as Gappa codes [21, 22]. It has been proved that the class of Goppa codes contains good codes. Goppa codes are in general noncyclic (except the BCH codes), and they can be decoded much like BCH codes. The decoding also consists of four steps: (1) compute the syndromes; (2) determine the error-location polynomial u(X); (3) find the error-location numbers; and ( 4) evaluate the error values (this step is not needed for binary Goppa codes). Berlekamp's iterative algorithm for finding the error-location polynomial for a BCH code can be modified for finding the error-location polynomial for Goppa codes [26]. Discussion of Goppa codes is beyond the scope of this introductory book. Moreover, implementation of BCH codes is simpler than that of Goppa codes, and no Goppa codes better than BCH codes have been found. For details on Goppa codes, the reader is referred to [26]-[30]. Our presentation of BCH codes and their implementation is given in the time domain. BCH codes also can be defined and implemented in the frequency domain using Fourier transforms over Galois fields. Decoding BCH codes in the frequency domain sometimes offers computational or implementation advantages. This topic will be discussed in Chapter 7.

PROBLEMS 6.1 Consider the Galois field GF(2 4 ) given by Table 2.8. The element f3 = a 7 is also a primitive element. Let go(X) be the lowest-degree polynomial over GF(2) that has as its roots. This polynomial also generates a double-error-correcting primitive BCH code of length 15. a. Determine go(X). I.Ji. Find the parity-check matrix for this code. c. Show that go(X) is the reciprocal polynomial of the polynomial g(X) that generates the (15, 7) double-error-correcting BCH code given in Example 6.1. 6.2 Determine the generator polynomials of all the primitive BCH codes of length 31. Use the Galois field GF(2 5 ) generated by p(X) = 1 + X 2 + x 5 .

Bibliography

231

63, Suppose that the double-error-correcting BCH code of length 31 constructed in Problem 6.2 is used for error correction on a BSC. Decode the received polynomials 1r1(X) = X7 + X 30 and ll'2(X) = 1 + X 17 + X 28 . 6Al Consider a t-error-correcting primitive binary BCH code of length n = 2m - 1. If 2t + 1 is a factor of n, prove that the minimum distance of the code is exactly 2t + 1. (Hint: Let n = l(2t + 1). Show that (X" + l)/(X1 + 1) is a code polynomial of weight 2t + 1.) 605 fa there a binary t-error-correcting BCH code of length 2"' + 1 for m ::,: 3 and t < 2111 - 1 ? If there is such a code, determine its generator polynomial. 606 Consider the field GF(2 4) generated by tpi(X) = 1 + X + X 4 (see Table 2.8). Let 4 Ol be a primitive element in GF(2 ) such that tpi(Ol) = 0. Devise a circuit that is capable of multiplying any element in GF(2 4 ) by Ol 7. 6/l Devise a circuit that is capable of multiplying any two elements in GF(2 5 ). Use p(X) = 1 + X 2 + x 5 to generate GF(2 5 ). 6otl Devise a syndrome computation circuit for the binary double-error-correcting (31, 21) BCH code. iliio9 Devise a Chien's searching circuit for the binary double-error-correcting (31, 21) BCHcode. ifiio]_@ Consider the Galois field GF(2 6 ) given by Table 6.2. Let /3 = Ol 3 , lo = 2, and d = 5. Determine the generator polynomial of the BCH code that has

as its roots (the general form presented at the end of Section 6.1). What is the length of this code? 6oU Let lo = -t and d = 2t + 2. Then we obtain a BCH code of designed distance 2t + 2 whose generator polynomial has

and their conjugates as all its roots. &llo Show that this code is a reversible cyclic code. lbio Show that if t is odd, the minimum distance of this code is at least 2t + 4. (Hint: Show that 13-(t+l) and 13 1+1 are also roots of the generator polynomial.)

1. A. Hocquenghem, "Codes corecteurs d'erreurs," Chiffres, 2: 147-56, 1959. '.?o R. C. Bose and D. K. Ray-Chaudhuri, "On a Class of Error Correcting Binary

Group Codes," Inform. Control, 3: 68-79, March 1960.

::t

W. W. Peterson, "Encoding and Error-Correction Procedures for the Bose-Chaudhuri Codes," IRE Trans. Inform. Theory, IT-6: 459-70, September 1960.

,;t D. Gorenstein and N. Zierler, "A Class of Cyclic Linear Error-Correcting Codes in pm Symbols," J. Soc. Ind. Appl. Math., 9: 107-214, June 1961.

5. I. S. Reed and G. Solomon, "Polynomial Codes over Certain Finite Fields," J. Soc. Ind. Appl. Math., 8: 300-304, June 1960.

232

Chapter 6

Binary BCH Codes

6. R. T. Chien, "Cyclic Decoding Procedure for the Bose-ChaudhuriHocquenghem Codes," IEEE Trans. Inform. Theory, IT-10: 357-63, October 1964. 7. G.D. Forney, "On Decoding BCH Codes," IEEE Trans. Inform. Theory, IT-11: 549-57, October 1965. 8. E. R. Berlekamp, "On Decoding Binary Bose-Chaudhuri-Hocquenghem Codes," IEEE Trans. Inform. Theory, IT-11: 577-80, October 1965. 9. E. R. Berlekamp, Algebraic Coding Theory, McGraw-Hill, New York, 1968. 10. J. L. Massey, "Step-by-Step Decoding of the Bose-Chaudhuri-Hocquenghem Codes," IEEE Trans. Inform. Theory, IT-11: 580-85, October 1965. 11. J. L. Massey, "Shift-Register Synthesis and BCH Decoding," IEEE Trans. Inform. Theory, IT-15: 122-27, January 1969.

12. H. 0. Burton, "Inversionless Decoding of Binary BCH Codes," IEEE Trans. Inform. Theory, IT-17: 464-66, July 1971. 13. W.W. Peterson and E. J. Weldon, Error-Correcting Codes, 2d ed., MIT Press, Cambridge, 1970. 14. F. J. MacWilliams and N. J. A. Sloane, The Theory of Error-Correcting Codes, North Holland, Amsterdam, 1977. 15. G. C. Clark, Jr., and J. B. Cain, Error-Correction Coding for Digital Communications, Plenum Press, New York, 1981. 16. R. E. Blahut, Theory and Practice of Error Control Codes, Addison-Wesley Reading Mass., 1983. 17. S. B. Wicker, Error Control Systems for Digital Communication and Storage, Prentice Hall, Englewood Cliffs, N.J., 1995. 18. H. B. Mann, "On the Number of Information Symbols in Bose-Chaudhuri Codes," Inform. Control, 5: 153-62, June 1962. 19. C. L. Chen, "High-Speed Decoding of BCH Codes," IEEE Trans. Inform. Theory, IT-27(2): 254-56, March 1981. 20. T. Kasami, S. Lin, and W.W. Peterson, "Some Results on Weight Distributions of BCH Codes," IEEE Trans. Inform. Theory, IT-12(2): 274, April 1966. 21. T. Kasami, S. Lin, and W. W. Peterson, "Some Results on Cyclic Codes Which are Invariant under the Affine Group," Scientific Report AFCRL-66-622, Air Force Cambridge Research Labs, Bedford, Mass., 1966. 22. T. Kasami, "Weight Distributions of Bose-Chaudhuri-Hocquenghem Codes," Proc. Conf Combinatorial Mathematics and Its Applications, R. C. Bose and T. A. Dowling, eds., University of North Carolina Press, Chapel Hill, 1968.

Bibliography

233

Z3o S. K. Leung-Yan-Cheong, E. R. Barnes, and D. U. Friedman, "On Some Properties of the Undetected Error Probability of Linear Codes," IEEE Trans. Inform. Theory, IT-25(1): 110-12, January 1979.

z,t G.

T. Ong and C. Leung, "On the Undetected Error Probability of TripleError-Correcting BCH Codes," IEEE Trans. Inform. Theory, IT-37: 673-78, 1991.

M. Sidelnikov, "Weight Spectrum of Binary Bose-ChaudhuriHocquenghem Code." Prob!. Inform. Transm., 7(1): 11-17, 1971.

Ztt V.

Z(fiio V. D. Goppa, "A New Class of Linear Codes," Prob[. Peredachi Inform., 6(3): 24-30, September 1970.

Z1o V. D. Goppa, "Rational Representation of Codes and (L, g) Codes," Prob!. Peredachi Inform., 7(3): 41-49, September 1971. Zft N. J. Patterson, "The Algebraic Decoding of Goppa Codes," IEEE Trans. Inform. Theory, IT-21: 203-7, March 1975. Zilt E. R. Berlekamp, "Goppa Codes," IEEE Trans. Inform. Theory, IT-19(5):

590-92, September 1973.

34t Y. Sugiyama, M. Kasahara, S. Hirasawa, and T. Namekawa, "A Method for Solving Key Equation for Decoding Goppa Codes," Inform. Control, 27: 87-99, January 1975. 3].o R. E. Blahut, "Transform Techniques for Error Control Codes," IBM 1. Res. Dev., 23(3): 299-315 May 1979.

CHAPTER

1

Nonbinary BCH Codes, Reed-Solomon Codes, and Decoding Algorithms This chapter presents nonbinary BCH codes with symbols from GF(q) and their decoding algorithms. The most important and popular subclass of nonbinary BCH codes is the class of Reed-Solomon codes. Even though Reed-Solomon codes form a subclass of BCH codes, they were constructed independently using a totally different approach by Reed and Solomon in 1960 [1 ], the same year as BCH codes were discovered. The relationship between Reed-Solomon codes and BCH codes was proved by Gorenstein and Zierler in 1961 [2]. The minimum distance of a Reed-Solomon code is equal to the number of its parity-check symbols plus one. Reed-Solomon codes are very effective in correcting random symbol errors and random burst errors, and they are widely used for error control in communication and data storage systems, ranging from deep-space telecommunications to compact discs. Concatenation of these codes as outer codes with simple binary codes as inner codes provides high data reliability with reduced decoding complexity. Decoding of a nonbinary BCH code or a Reed-Solomon code requires determination of both the locations and the values of the symbol errors. The first error-correction procedure for nonbinary BCH and Reed-Solomon codes was found by Gorenstein and Zierler [2], and it was later improved by Chien [3] and Forney [4]. But Berlekamp's iterative decoding algorithm [5] presented in the previous chapter was the first efficient decoding algorithm for both binary and nonbinary BCH codes. In 1975 Sugiyama, Kasahara, Hirasawa, and Namekawa showed that the Euclidean algorithm for finding the greatest common divisor of two polynomials can be used for decoding BCH and Reed-Solomon codes [6]. This Euclidean decoding algorithm is simple in concept and easy to implement. BCH and Reed-Solomon codes can also be decoded in the frequency domain. The first such frequency-domain decoding algorithm was introduced by Gore [7], and it was later much improved by Blahut [8]. All the preceding decoding algorithms can be modified for correcting both symbol errors and erasures. Reed-Solomon codes have been proved to be good error-detecting codes [9], and their weight distribution has been completely determined [10]-[12]. Good treatment of nonbinary BCH and Reed-Solomon codes and their decoding algorithms can be found in [5] and [13-20]. 7.1

q-ARY LINEAR BLOCK CODES

Consider a Galois field GF(q) with q elements. It is possible to construct codes with symbols from GF(q). These codes are called q-ary codes. The concepts and 234

Section 7.1

q-ary Linear Block Codes

;ns

properties developed for the binary codes in the previous chapters apply to q-ary codes with few modifications. Consider the vector space of all the q 11 n-tuples over GF(q ): (vo, v1, · · · , Vn-1)

with v;

E

GF(q) for O ::: i < n. The vector addition is defined as follows: L,

+ (vo, v1, · · · , Vn-d = (uo + vo, u1 + v1, · · · , Un-1 + V11-1), where the addition u; + v; is carried out in GF(q ). The multiplication of a scalar in (uo, u1, · · · , Un-1)

GF(q) and an n-tuple (vo, v1, · · ·, v11 _1) over GF(q) is given as follows: L,

a· (vo, v1, ···, v11 -1) =(a· vo,a · v1, ···,a· v11 -1)

where the product a · v; is carried out in GF(q ). The inner product of two n-tuples, (uo, u1, · · · , u 11 _1) and (vo, v1, · · · , v11 _1), is defined as follows: (uo, u1, · · · , u 11 _1) · (vo, v1, · · · , v11 _1)

~

n-1

Lu; · v; i=O

where addition and multiplication are carried out in GF(q ). An (n, k) linear block code with symbols from GF(q) is simply a k-dimensional subspace of the vector space of all then-tuples over GF(q ).

)l])JEJFKNli'lI'RN '10].

A q-ary linear block code has all the structures and properties developed for binary linear block codes. A q-ary linear block code is specified either by a generator matrix or by a parity-check matrix over GF(q ). Encoding and decoding of q-ary linear codes are the same as for binary linear codes, except that operations and computations are performed over GF(q ). A q-ary (n, k) cyclic code is generated by a polynomial of degree n - k over GF(q), g(X) =go+ g1X

+ · · · + gn-k-lxn-k-l + xn-k

where go I- 0 and g; E GF(q). The generator polynomial g(X) is a factor of X 11 - 1. A polynomial v(X) of degree n - 1 or less over GF(q) is a code polynomial if and only if v(X) is divisible by the generator polynomial g(X). In this chapter we present two important classes of cyclic codes over G F( q) whose constructions are based on an extension field of GF(q). Construction of an extension field of GF(q) is similar to the construction of an extension field of GF(2). A polynomial f (X) with coefficients from G F(q) is called manic if the coefficient of the highest power of X is 1. A polynomial p(X) of degree 111 over G F( q) is said to be irreducible if it is not divisible by any polynomial over G F( q) of degree less than 111 but greater than zero. An irreducible polynomial p(X) of degree 111 over GF(q) is called primitive if the smallest positive integer n for which p(X) divides X 11 - 1 is n = q 111 - 1. For any positive integer 111, a Galois field GF(q 111 ) with q 111 elements can be constructed from the ground field GF(q ). The construction is exactly the same as the

236

Chapter 7

Nonbinary BCH Codes

construction of G F(2111 ) from G F(2). The construction of G F( q 111 ) is based on a monic primitive polynomial p(X) of degree m over GF(q ). Let a be a root of p(X). Then, 0, 1, a, a 2 , · · · , aq

111 -

2

form all the elements of G F( q 111 ), and aq 1 = 1. The element a is called a primitive element. Every element f3 in GF(q 111 ) can be expressed as a polynomial in a, 111

-

/3 = ao + a1a + a2cx2 + ... + am-l<Xm-l where a; E GF(q) for O :s i < m. Then, (ao, a1, · · · , a 111 _1) is a vector representation of (3. Therefore, every element in GF(q 111 ) has three forms: power (0 is represented by a 00 ), polynomial, and vector forms. The elements in GF(q 111 ) form all the roots of Xq X. Let f3 be an element in GF(q 111 ). The minimal polynomial of f3 is the monic polynomial (X) of the smallest degree over GF(q) that has f3 as a root; that is, (/3) = 0. Just as in the binary case, (X) is irreducible. Let e be the smallest nonnegative integer for which f3qe = (3. The 2 e-1 integer e is called the exponent of f3 and e :S m. The elements (3, f3q, f3q , · · · , f3q are conjugates. Then, 111

n

e-l

(X) =

(X -

/3q; ),

i=O

and (X) divides Xq 1."J.

111 -

X.

PRIMITIVE BCH CODES OVER Gf{q)

The binary BCH codes defined in Section 6.1 can be generalized to nonbinary codes in a straightforward manner. Let a be a primitive element in GF( q 111 ). The generator polynomial g(X) of at-error-correcting primitive q-ary BCH code is the polynomial of the smallest degree over GF(q) that has a, a 2, ... , a 21 as roots. For 1 :s i :s 2t, let ; (X) be the minimal polynomial of ai. Then, g(X)

= LCM{1 (X), 2(X), · · · , 21 (X)}.

(7.1)

Because each; (X) divides Xq 1 -1, g(X) divides Xq 1 -1. Since a is a primitive element in GF(q 111 ), 1(X) is a primitive polynomial of degree m. Hence, the smallest positive integer n for which 1(X) divides X 11 - 1 is n = q 111 1. This result implies that q 111 - 1 is the smallest positive integer for which g(X) divides Xq -l - 1. Because the degree of each; (X) ism or less, the degree of g(X) is 2mt or less. Similar to the way we proved the BCH bound for the minimum distance of a binary BCH code, we can prove that the minimum distance of the q-ary BCH code generated by g(X) of (7.1) is lower bounded by 2t + 1. Summarizing the foregoing results, we see that the q-ary BCH code generated by the polynomial g(X) of (7.1) is a cyclic code with the following parameters: 111

111

-

-

111

Block length: n

=q

111

-

1,

Number of parity-check symbols: n - k Minimum distance:

dmin ::::

2t

+ 1.

:s 2mt,

Section 7.3

Reed-Solomon Codes

2311

This code is capable of correcting t or fewer random symbol errors over a span of qm - 1 symbol positions. For q = 2, we obtain the binary primitive BCH codes. Similar to the binary case, the matrix over GF(qm),

lHI=

(X

0/,2

(Xn-1

(X2

(a2)2

(a2)'1-l

(X3

(a3)2

(a3y1-l

1 0/,21

(a2t)2

(a21)'1-l

1 1 1

is a parity-check matrix of the t-error-correcting primitive q-ary BCH code generated by the polynomial g(X) of (7.1).

The special subclass of q-ary BCH codes for which m = 1 is the most important subclass of q-ary BCH codes. The codes of this subclass are called the Reed-Solomon (RS) codes in honor of their discoverers, Irving S. Reed and Gustave Solomon [1 ]. RS codes have been widely used for error control in both digital communication and storage systems. Let a be a primitive element in GF(q ). The generator polynomial g(X) of a t-error-correcting RS code with symbols from GF(q) has a, a 2 ,. ·. , a 21 as all its roots. Because a; is an element of GF(q ), its minimal polynomial ef»; (X) is simply X - ai. Then, it follows from (7.1) that g(X) = (X - a)(X - a 2) ... (X - a 21 )

=go+ g1X + g2X 2 + · · · + g21-1X 21 - 1 + X 21

(7.2)

with g; E GF(q) for O ::S i < 2t. Since a, a 2 , ... , a 21 are roots of xq-l - 1, g(X) divides xq-l - 1. Therefore, g(X) generates a q-ary cyclic code of length n = q - 1 with exactly 2t parity-check symbols. H follows from the BCH bound that the minimum distance of the code is at least 2t + 1; however, the generator polynomial g(X) is a code polynomial and has 2t + 1 terms. None of the coefficients in g(X) can be zero, otherwise the resulting codeword would have weight less than 2t + 1, which would contradict the BCH bound on the minimum distance. Therefore, g(X) corresponds to a codeword of weight exactly 2t + 1. This implies that the minimum distance of the RS code generated by the polynomial g(X) of (7.2) is exactly 2t + 1, and the code is capable of correcting t or fewer symbol errors. In summary, a t-error-correcting RS code with symbols from GF(q) has the following parameters: Block length: n = q - 1, Number of parity-check symbols: n - k Dimension: k

=q

- 1 - 2t,

Minimum distance:

dmin =

2t

+ 1.

= 2t,

238

Chapter 7

Nonbinary BCH Codes

Thus, we see that RS codes have two important features: (1) the length of the code is one less than the size of the code alphabet, and (2) the minimum distance is one greater than the number of parity-check symbols. Codes with minimum distance one greater than the number of parity-check symbols are called maximum distance separable (MDS) codes. RS codes form the most important class of MDS codes. EXAMPLE 7.1

Let a be a primitive element in GF(2 6 ) constructed based on the primitive polynomial p(X) = 1 + X + X 6 (see Table 6.2). Consider the triple-error-correcting RS code with symbols from GF(2 6). The generator polynomial g(X) of this code has a, a 2 , a 3 , a4, as, and a 6 as all its roots; hence, g(X)

= (X + a)(X + a 2)(X + a 3 )(X + a 4)(X + a 5 )(X + a 6 ) = a21 + alO X + ass x2 + a43 X3 + a48 x4 + aS9 XS + X6.

The code is a (63, 57) triple-error-correcting RS code over GF(26 ). Encoding of a RS code is similar to that of the binary case. Let a(X)

= ao + a1X + a2X 2 + · · · + ak-lxk-l

be the message to be encoded, where k = n - 2t. In systematic form, the 2t parity-check symbols are the coefficients of the remainder b(X) = bo + b1X + · · · + b 21 _ 1 x 21 - 1 resulting from dividing the message polynomial X 21 a(X) by the generator polynomial. In hardware implementation, this is accomplished by using a division circuit as shown in Figure 7.1. As soon as the message a(X) has entered the channel and the circuit, the parity-check symbols appear in the register. The weight distribution of Reed-Solomon codes has been completely determined [10]-[12]. For at-error-correcting RS code of length q - 1 with symbols from GF(q ), the number of codewords of weight i is given by

+ 1 _s: i _s: q - 1. Suppose a q-ary RS code is used for error detection on a discrete memoryless channel with q inputs and q outputs. Let (1 - E:) be the probability that a transmitted symbol is received correctly and E: / (q-1) be the probability that a transmitted symbol is changed into each of the q - 1 other symbols. Using the weight distribution given by (7.3), it can be shown that the probability of undetected error for a RS code is [9]

for 2t

Pu(E)

= q-21

X

(

l

1+

~

21-l (

qc 1- q _ l

q

~

q-1-i )

1)

(q21 - qi)

~

- q (l -

( ~ );

E: )q

1

q

-1

l

.

(7.4)

Section 7.3

Reed-Solomon Codes

23l~

- ~ - de,o<es ,, adde, tha< adds lwo eleme,ts from GF(q)

g;

~ ~~'i>

denotes a multiplier that multiplies a field element from GF(q) by a fixed element g; from the same field

~~ denotes a storage device that is capable of storing a field element b; from ~~GF(q)

FIGURE 7.1:

go+ g1X

Encoding circuit for a q-ary RS code with generator polynomial ~(X)

+ g2X 2 + · · · + g21-1X 21 - 1 + X 21 .

=

H also has been shown in [9] that Pu (E) < q- 21 and decreases monotonically as E decreases from (q - 1)/q to 0. Hence, RS codes are good for error detection. Let Jc be a nonnegative integer less than t. Suppose at-error-correcting q-ary RS code is used to correct all error patterns with Jc or fewer symbol errors. Let Pu (E, Jc) denote the probability of undetected error after correction. It has also been proved in [9] that Pu(E, Jc)=

t ( 1) [ q:

q-2t (q - l)h - Eh(l - E)q-l-h

h=O

+

min{2t-1,q-l-h}

L

(q _ l _ l

h) (--1

E )I (

q

1=0

qE )q-l-h-1

1- ~ q

J

R1,,1(E) ,

(7.5)

where min{2t-l-l,h}

R11,1(E)=

L

(h)

(-l)h-j j

(1-q-2t+l+j)

(1-

q~l

)j (1- q~l )h-j

;=0

(7.6)

for O ::=: l < 2t. It is easy to check that Pu(E) given by (7.4) can be obtained from P,i(E, Jc) of (7.5) by setting Jc= 0. In [9] it is shown that the probability Pu(E, Jc) of

240

Chapter 7

Nonbinary BCH Codes

undetected error after decoding decreases monotonically from (7.7) as E decreases from (q - 1)/q to 0. Therefore, we have the following upper bound on Pu (E, A) from (7.7): (7.8)

for O :': E :': (q - 1)/q. The preceding results on error probabilities indicate that RS codes are effective for pure error detection or simultaneous error correction and detection. Consider the set of codewords in a (q - 1, q - 1 - 2t) q-ary RS code whose l leading information symbols are identical to zero, with O ::: l < q - 1 - 2t. Deleting these l zero-information symbols from each codeword in the set, we obtain a shortened (q -1- l, q -1- 2t -l) RS code. The minimum distance of this shortened RS code is still 2t + 1. Hence, the shortened code is also a MDS code. Encoding and decoding of a shortened RS code are the same as those for the original RS code of natural length (see Section 5.10). Two information symbols can be added to a RS code of length q - 1 without reducing its minimum distance. The extended RS code has length q + 1 and the same number of parity-check symbols as the original code. For at-error-correcting RS code of length q - 1, the parity-check matrix takes the form

H-[;

a2

a a2

(a2)2

a2t

(a2t)2

Then, the parity-check matrix of the extended RS code is 0 1 0 0 H1=

H

(7.9)

0 0 1 0 This code is called a doubly extended RS code and is also a MDS code. The preceding result was first obtained by Kasami, Lin, and Peterson [11, 21] and later independently by Wolf [22]. If we delete the first column of H1, we obtain a singly extended RS code of length q, which is again a MDS code. In all practical applications of RS codes for error control, q is set to 2m, and code symbols are from the Galois field GF(2m).

Section 7A

1.~

Decoding of Nonbinary BCH and RS Codes

24'l

DIECOfDi!IT\IIG OIF IT\IIOIT\1118!IT\IIA~V 18(~ AIT\lllDJ ~§ COIDJiE§: l~iE 181E~l1E1ZAMI? AlGOml~M

In this and the next three sections, we present various algorithms for decoding nonbinary BCH and RS codes. Let v(X) = vo

+ v1X + · · · + Vn-1X

11 -

1

be the transmitted code polynomial, and let ir(X) = ro

+ r1X + · ·. + r

11 _

1

x

11

-

1

be the corresponding received polynomial. Then, the error pattern added by the channel is e(X) = ir(X) - v(X)

= eo + e1X + · · · + e

11 _

1

x

11 -

1

,

where e; = r; v; is a symbol in GF(q ). Suppose the error pattern e(X) contains v errors (nonzero components) at locations Xii, X h, ... , Xi" with O :s: ji < .h < · · · < jv :S: n - 1. Then, (7.10)

where e.ii, e12 , · · · , eJ" are error values. Hence, to determine e(X), we need to know the error locations X J; 's and the error values e J; 's (i.e., we need to know the v pairs (Xi;, eJ)'s). As with binary BCH codes, the syndrome is a 2t-tuple over GF(qm):

with S; = ir(a;) for 1 :S: i :s: 2t. Because ir(X) = w(X)

+ e(X), we have (7.11)

From (7.10) and (7.11), we obtain the following set of equations that relate the error locations and error values to the syndrome of the received polynomial n·(X):

2

S2 = e11· a i 1

+ e12· a 212 + · · · + eh· a 2i" (7.12)

S2t -- e Ji. a21J1

+ e12. a2t}2 + ... + e . a2tj" ]l'

For 1 :s: i :s: v, let L,

{3; =

.

aJ;

and

•

242

Chapter 7

Nonbinary BCH Codes

which are simply the error-location numbers and error values. With the preceding definitions of {3; and 8;, we can simplify the equations of (7.12) as follows: S1

= 81fJ1 + 82fJz + · · · + 8vf3v

S2

= 81f3f + 82f3i + · · · + 8vf3~ (7.13)

For decoding a q-ary BCH code or a RS code, the same three steps used for decoding a binary BCH code are required; in addition, a fourth step involving computation of error values is required. Therefore, the decoding consists of the following four steps:

1. Compute the syndrome (S1, S2, · · · , S21). 2. Determine the error-location polynomial a(X). 3. Determine the error-value evaluator. 4. Evaluate error-location numbers and error values and perform error correction. As with binary BCH codes, the error-location polynomial a (X) is defined as a(X) = (1 - f31X)(l - f32X) · · · (1 - f3vX) (7.14) where ao = 1. The error-location numbers are the reciprocals of the roots of a(X). From (7.13) and (7.14), it is possible to obtain the following set of equations that relates the coefficients a; 'sofa (X) and the syndrome components S; 's:

+ a1Sv + a2Sv-l + · · · + avS1 = 0 Sv+2 + a1Sv+l + a2Sv + · · · + avS2 = 0 Sv+l

(7.15)

(These equalities will be derived later.) These equations are known as the generalized Newton's identities. Our objective is to find the minimum-degree polynomial cr(X) whose coefficients satisfy these generalized Newton's identities. Once we have found o-(X), we can determine the error locations and error values. We can compute the error-location polynomial a(X) iteratively in 2t steps with Berlekamp's algorithm presented in Section 6.2. At the µ,th step, we determine a polynomial of minimum degree

Section 7.4

Decoding of Nonbinary BCH and RS Codes

2,ij!.3

whose coefficients satisfy the following µ - lv identities: S1,,+1 + a?) S1 1, + · · · + a 1~) S1 = 0

Sz µ +2 + a 1(v) Sz /.1. +l + · · · + a 1(v) S2 = 0 µ (7.16)

The next step is to find a new polynomial of minimum degree ,o,
= a
a
11;+!

whose coefficients satisfy the following(µ+ 1) - lv+l identities: (v+ll

+ ··· +a1(µ+l) S1 µ+l

=

(J_1,+l) (µ+l) Sz/.1 + 1+2 + a 1 S1 /.I + 1+1 + · · · + a1{J.+t S2

=

Sz µ +i+l +a1

Sz µ +i

0 O

(7.17) Sv+l + al(v+l)sv + · · · + az(v+l) Sv+l-1 +1 t..t+I /1.

= o·

We continue the foregoing process until 2t steps have been completed. At the 2tth step, we have u(X) = o-< 21 \X),

which is the true error-location polynomial provided that the number of errors in e(X) does not exceed the error-correcting capability t. fo this case, the coefficients of rDT (X) satisfy the set of generalized Newton's identities given by (7.15). Suppose we have just completed the µth step and found the solution u
to ,o,
-f 0, we need to adjust u<,_.ll(X) to satisfy the equalities of (7.17). We make the correction as follows: we go back to the steps prior to the µth step and determine a

If dµ

244

Chapter 7

Nonbinary BCH Codes TABLE 7.1: Berlekamp's iterative procedure for finding the error-location polynomial of a q-ary BCHcode.

-1

0

1 1

1

1

1- S1X

0 0

S1

-1

0

2 3 2t

polynomial
lp

has the largest value, where lp is the (7.20)

and
is an error-location number. We can do this systematically with Chien's search. EXAMPLE 7.2

Consider a triple-error-correcting RS code with symbols from GF(24 ). The generator polynomial of this code is g(X)

=

(X + a)(X + a 2)(X + a 3)(X + a 4)(X + a 5)(X + a 6)

= a6 + a9X + a6X2 + a4X3 + al4X4 + alOXS + X6.

Let the all-zero vector be the transmitted codeword, and let r = (0 0 0 a 7 0 0 a 3 0 0 0 0 0 a 4 0 0) be the received vector. Thus, r(X) = a 7 x 3 + a 3 x 6 + a 4 x12. Step 1. We compute the syndrome components as follows (using Table 2.8): S1

= r(a) = alO + a9 +a= a12,

= r(a 2) = a 13 + 1 +a 13 = 1, 3 10 6 S3 = r(a ) = a + a + a = a 14, S4 = r(a4) = a4 + al2 + a7 = alO, S2

Ss

= r(a 5) = a 7 + a 3 + a 4 = 0,

s6

= r(a 6) = a 10 + a 9 +a= a 12 .

Decoding of Nonbinary BCH and RS Codes

Section 7.4

245

TABLE 7.2: Steps for finding the error-location polynomial of

the (15,9) RS code over GF(2 4). §fl

-1 0 1 2 3 4

5 6

u(µ,)

(J[)

l µ, 1 a12 a7 1

1 1 1 + a12x l+a 3X 1 +a 3X +a 3X 2 1 + a4 X + a12 x2 1 +a 7 X +a 4X 2 +a 6X 3 1 + a 7X + a 4X 2 + a 6x 3

ot7 (XlO

0

0 0 1

1 2 2 3

§fl-ffµ,

-1 0 O(take l(take l(take 2(take 2(take

p p p p p

= -1) = 0) = 0) = 2) = 3)

§1!ie]lll Zo To find the error-location polynomial u (X), we fill out Table 7.1 and obtain Table 7.2. Thus, u (X) = 1 + a 7 x + a4 x 2 + a6 x3. §~ie]lll So By substituting 1, a, a 2 , · · · , a 14 into cr(X), we find that a 3, a 9, and a 12 are roots of u(X). The reciprocals of these roots are a 12 , a 6 , and a 3 , which are the error-location numbers of the error pattern ie(X). Thus, errors occur at positions X 3, x 6 , and x 12 .

Next, we need to determine the error values, by finding the error-value evaluator. We define the syndrome polynomial §(X) as follows: §(X) ~ S1 + S2X + · · · + S21X 21 - 1 + S21+1X 21 + · · · (7.21)

00

= Z::,sixj- 1 . j=l

Note that only the coefficients of the first 2t terms are known. For 1 :::: j < oo, we also define V

Sj='f:,81f3/.

(7.22)

i=l

The first 2t such S/s are simply the 2t equalities of (7.13). Combining (7.21) and (7.22), we can put S(X) in the following form: 00

V

00

V

=L 1=1

(7.23)

81/31 'f:,(/31X) 1- 1 . }=1

Note that 1

00

f:r

- - - - ~(P. X)i-1 (1 - /31X) JJI •

(7.24)

246

Chapter 7

Nonbinary BCH Codes

It follows from (7.23) and (7.24) that

=

S(X)

L 1-Of3f31X V

I I

(7.25)

.

l=l

Consider the product a(X)S(X), a(X)S(X)

= =

+ cr1X + · · · + O'vXv) · (S1 + S2X + S3X2 + · · ·) 2 S1 + (S2 + cr1S1)X + (S3 + cr1S2 + cr2S1)X + · · · + 21 1 (S21 + cr1S21-l + · · · + crvS21-v)X - + · · · (1

(7.26)

Using (7.25), we can also put a(X)S(X) in the following form: u(X)S(X)

(n(l -

=

/3;X)l ·

i=l

1~

f3z

n V

x.

l

f3z

Z=l

1 8zf3z X l

Z=l

O f3

V

=L

(t

(1 - /3; X)

(7.27)

i=l V

=

fl

L8zf3z l=l

(1- {3;X).

i=l,icf.l

We define V

Zo(X)

V

fl

~ L 8zf3z l=l

(1 - f3;X).

(7.28)

i=l.icf.l

Note that Zo(X) is a polynomial of degree v - 1. From (7.26), (7.27), and (7.28), we see that Zo(X) must be equal to the first v terms from x 0 to xv-l in u(X)S(X) of (7.26); that is, Zo(X)

=

+ (S2 + cr1S1)X + (S3 + cr1S2 + cr2S1)X 2 + · · · + (Sv + cr1Sv-l + · · · + O'v-1S1)Xv-l. S1

(7.29)

Because the degree of Zo(X) is v - 1, the coefficients of powers from xv to x 21 - 1 in the expansion of u (X)S(X) ( (7.26)) must be zero. Setting these coefficients to zero, we have exactly the same set of equations as (7.15). Next, we show that the error values can be determined from Zo(X) and u(X). Substituting f3-; 1 in Zo(X) (given by (7.28)), we have

n

V

1 Zo(/3,~ )

=

L

V

8zf3z

l=l

1 (1 - /3;/3-; )

i=l,icf.l

n

(7.30)

V

= 8kf3k

i=l,i#

1

(1 - {3;/3-; ).

Decoding of Nonbinary BCH and RS Codes

Section 7.4

26.!1

We take the derivative of u(X) in (7.14),

n(l =- L n d

V

u'(X) = dX

f3iX)

i=l

V

(7.31)

V

f3z

1=1

(1 - f3i X).

i=l,ic/1

Then,

n V

1 u'(/3i: )

=

-f3k

1

(1- f3if3i: ).

(7.32)

i=U#

From (7.30) and (7.32), we find that the error value ok at location f31c is given by

o k

=

(7.33)

-Zo(/31~1) u'(/3i:1) .

This expression was derived by Forney [4]. The polynomial Zo(X) is called the error-value evaluator. A slightly different error-value evaluator is defined as !::,.

Z(X) = u(X)

+ XZo(X)

= 1 + (S1 + a1)X + (S2 + a1S1 + cr2)X 2

(7.34)

Then, -Z(f3i:l)

Ok=

1 .

(7.35)

nr=u#(1 - f3i /3,~ )

The expression for evaluating 01c was derived by Berlekamp [5]. IEKAMPLIE 7/.3

Consider the triple-error-correcting RS code of length 15 given in Example 7.2, where we assumed that the all-zero codeword was transmitted, and n· = (0 0 0 a 7 0 0 a 3 0 0 0 0 0 a 4 0 0) was received. Using the Berlekamp algorithm, we find that u(X)

= 1 + a 7 X + a 4X 2 + a 6x3,

and error-location numbers are a 12 , a 6 , and a 3 . The errors occur at X 3 , x 6 , and x12 . Now, we are ready to evaluate the error values at these positions. From (7.29) we find that Zo(X)

= S1 + (S2 + a1S1)X + (S3 + a1S2 + cr2S1)X 2 = a12 + (l + a 7 ot12)X + (a14 + a 7 + a4a12)X2 = a12 + (1 + a4)X + (a14 + a7 + a)X2 = a 12

+ ax.

248

Chapter 7

Nonbinary BCH Codes

r(X)

+ -

r(X) - e(X) e(X)

Syndrome computation

o-(X)

u(X) Register --1>1 Galois field arithmetic processor Z 0 (X) Registerl4--1>1

.___ _ _ _ _ _ _ Z"""o(.,_X-'-)_ _ _ Error-magnitude 1 - - - - - - ' 1

computation

FIGURE 7.2: A general organization of a q-ary BCH decoder.

(The computations are carried out in GF(2 4 ) given by Table 2.8.) Using (7.33), we obtain the error values at locations x 3, x 6 , and x12: e3

=

e6

=

-Zo(a-3) o-'(a-3)

a12 + aa-3 a3(1 + a6a-3)(1 + a12a-3)

-Zo(a-6)

+ aa-6 a3 - - -a3 a6(1 + a3a-6)(1 + a12a-6) - 1 ' a12 + aa-12 a6 a12(1 + a3a-12)(1 + a6a-12) = a2 = a4

=

o-'(a-6)

o-'(a-12)

=

;;9 = a

7

'

a12

-Zo(a-12) e12

a

-

Thus, the error pattern is

which is exactly the difference between the received vector and the transmitted vector. The decoding is completed by taking r(X) - e(X). A general organization of a BCH decoder is shown in Figure 7.2.

1.5

DECODING WITH THE EUCLIDEAN ALGORITHM

In the expansion of o-(X)S(X) ((7.26)), only the coefficients of the first 2t terms (x 0 to x 21 - 1) are known. Let [o-(X)S(X)b

Section 7.5

Decoding with the Euclidean Algorithm

24!:l

denote the first 2t terms of OT(X)§(X). Then, OT(X)§(X) - [u(X)§(X))li is divisible by X 21 . This simply says that if OT (X)§(X) is divided by X 21 , the remainder is [u(X)§(X)fai. Mathematically, this statement is expressed as follows: OT(X)§(X)

= [OT(X)§(X)b modX 21 .

(7.36)

In fact, Zo(X) = [OT(X)§(X)fat.

(7.37)

Therefore, we have u(X)§(X)

= Zo(X) modX 21

(7.38)

which is called the key equation in decoding of BCH codes [5]. Any method of solving the key equation to find u(X) and Zo(X) is a decoding method for q-ary BCH codes. H the number of errors v during the transmission of a code polynomial w(X) is less than or equal to t, (i.e., v :::: t ), then the key equation has a unique pair of solutions, ( OT (X), Zo(X) ), with deg Zo(X) < deg OT(X) ::st,

(7.39)

where deg f (X) denotes the degree of polynomial f (X). We have already presented Berlekamp's algorithm for solving the key equation, which is a very effective method for practical implementation and has been widely used. There is another method for solving the key equation that is much easier to understand. This method is based on the Euclidean algorithm for finding the greatest common divisor (GCD) of two polynomials. Consider two polynomials, a(X) and b(X), over GF(q). Assume that deg a(X) 2: deg b(X). Let GCD[a(X), b(X)] denote the greatest common divisor of a(X) and b(X). Then, we can find GCD[a(X), b(X)] by iteratively applying the division algorithm as follows:

+ r1(X) b(X) = q2(X)r1(X) + r2(X) r1(X) = q3(X)r2(X) + r3(X) a(X) = q1(X)b(X)

r;-2(X) = q; (X)r;-1 (X)

Tn-2(X)

= qn(X)r

11

+ r; (X)

_1(X)

Tn-1(X) = q11 +1(X)r11(X),

+r

11

(X)

(7.40)

250

Chapter 7

Nonbinary BCH Codes

where q; (X) and r; (X) are the quotient and the remainder, respectively, at the ith step of the iterative division. The iteration stops when the remainder is identical to zero. Then, the last nonzero remainder rn(X) is the GCD of a(X) and b(X) (may be different by a constant scalar c ); that is, rn(X)

where c

E

= cGCD[a(X), b(X)],

GF(q). Note that for 1 :Si

:s n,

degr;-1(X) > degr;(X). From (7.40), it is possible to show that GCD[a(X), b(X)] = f (X)a(X)

+ 8(X)b(X),

(7.41)

where f(X) and 8(X) are polynomials over GF(q) [Euclid's algorithm]. In fact, we can express the remainder at each division step as follows:

r2(X)

+ 81 (X)b(X) = h(X)a(X) + 82(X)b(X)

r;(X)

= f;(X)a(X) + 8;(X)b(X)

r 11 (X)

= f,

r1 (X) = Ji (X)a(X)

1

(X)a(X)

(7.42)

+ 8n(X)b(X).

From (7.41) and (7.42), we have J(X)

= c-l fn(X)

8(X)

= c- 18 (X).

(7.43)

11

From (7.40) and (7.42), we obtain the following recursive equations for finding r; (X), f; (X), and 8; (X):

= r;-2 (X) - q; (X)r;-1 (X) f;(X) = /;-2(X) - q;(X)f;-1(X) r;

(X)

(7.44)

8;(X) = 8;-2(X) - q;(X)8;-1(X) for 1 :S i

:s n. The initial conditions for the recursion are r -l (X) ro(X)

f-1 (X)

= a(X), = b(X), = 8o(X) = 1,

fo(X) = 8-1 (X) = 0.

(7.45)

Section 7.5

Decoding with the Euclidean Algorithm

251

TABLE 7.3: Steps for finding the GCD of X 3 + 1

x2 +

1 given in Example 7.4. ri(X)

-1 0 1 2

{§i(X)

x3 + 1 x 2 +1 X+l

X

0

X+l

/i(X)

gi(X)

1 0 1 X+l

0

1 X X2 + X + 1

An important property of Euclid's algorithm is deg a(X) = deg g; (X)

+ deg r;-1 (X).

(7.46)

We see that as i increases, the degree of r;-1 (X) decreases, and the degree of g; (X) increases. This result will be used for solving the key equation. IEXAMIPILIE 71 .dil

Let a(X) = X 3 + 1 and b(X) = X 2 + 1 be two polynomials over GF(2). Euclid's algorithm for finding the GCD[X 3 + 1, X 2 + 1] is shown in Table 7.3. We see that last nonzero remainder is which is the GCD of x 3 + 1 and

71 .5.1

x 2 + l.

5ohJarrn\9) t~ie 1<:®y IEIDJMaii:aorrn [16]

We can express the key equation in the following form: u(X)§(X)

= Q(X)X 21 + Zo(X).

(7.47)

Rearranging (7.47), we have Zo(X) = -Q(X)X 21

+ u(X)§(X).

(7.48)

X 21

Setting a(X) = and b(X) = §(X), we see that (7.48) is exactly in the form given by (7.41). This suggests that u(X) and Zo(X) can be found by the Euclidean iterative division algorithm for the two polynomials: a(X)

=

X 21 ,

(7.49)

b(X) = §(X),

where Let zgl (X) = r; (X) u
= g;(X)

yU\X) = f; (X).

(7.50)

252

Chapter 7

Nonbinary BCH Codes

Then, it follows from (7.49) and (7.50) that we can put (7.42), (7.44), and (7.45) in the following forms: (7.51)

and zg\x)

= zg- 2\x)

1

- q;(X)zg- l(X),

uCil(X) = u(i-Z)(X) - q;(X)u(i-ll(X),

(7.52)

y
with ztl\x) Z60l (X) y<- 1\X) y< 0\X)

= xzi, = S(X), = u
= u<-l)(X) = 0.

To find u(X) and Zo(X), we carry out the iteration process given by (7.52) as follows: at the ith step, 2 1. We divide zg- \x) by zg- 1\x) to obtain the quotient q;(X) and the remainder zgl (X). 2. We find u
u
Iteration stops when we reach a step p for which (7.53)

Then, Zo(X) =

z~) (X),

u(X) = u
If the number of errors is t or less, there always exists a step p for which the condition given by (7.53) holds. It is easy to see that p

s

2t.

Note that u(X) = u
Section 7.5

Decoding with the Euclidean Algorithm

253

TABLE 7.4: Euclidean's iterative procedure for

finding the error-location polynomial and error-value evaluator.

i

z{O (X)

-1

x2r

-

§(X)

-

0

(X)

{JJi

I[])

Uff,{X)

0 1

1 2 i

IE){AMIPILIE 7.5

Consider the triple-error-correcting RS code of length n = 15 over GF(2 4 ) whose generator polynomial has a, a 2 , a 3, a 4, a 5, and a 6 as roots; that is,

Suppose the received polynomial is ir(X) = (J/7 x3 + (J/11 xrn.

The syndrome components are S1 = ir(a) = alO + a21 = a7,

s2 = s3 = s4 =

ir(a2) = 0/13 + 0/31 = 0/12, r(a3) = 0/16 + a41 = a6, ir(a4) = 0/19 + a51 = 0/12,

Ss = ir(as) = a7 +a= al4,

s6 =

ir(a6) = (J/10 + (J/11 = (J/14.

Hence, the syndrome polynomial is §(X) = (J/7 + a12x + a6x2 + a12x3 + a14x4 + a14xs.

Using the Euclidean algorithm, we find u(X) =all+ asx + a9x2 = all(l + a12 X +

a13 X2),

and Zo(X)

as shown in the Table 7.5.

= a 3 + a 2 X,

254

Chapter 7

Nonbinary BCH Codes

Steps for finding the error-location polynomial and error-value evaluator of the RS code given in Example 7.5.

TABLE 7.5:

i -1 0 1 2

Z(i)(X)

0 x6 S(X) a8 + a3 X + as x2 + as X3 a3 +a2x

qi(X) -

+ a6 x4

a+ax a 11

+ a 8X

u i (X) 0 1 a+ax all + a8 X + a9 x2

From u(X), we find that the roots are a 5 and a 12 . Hence, the error location numbers are a 10 and a 3. The error values at these locations are -Zo(a-3) a3 + a2a-3 1 7 - - -a 3 11 3 3 8 - u'(a- ) - a · a (1 + alOa- ) - a '

e3 -

e10

=

-Zo(a-10) a3 + a2 . a-10 - a4 - ~11 = 3 10 u'(a- ) all· alO(l + a a-10) - a 8 UC

Therefore, the error polynomial is e(X) = a 7 X 3 + a 11 X10 , and the decoded codeword v(X) = r(X) - e(X) is the all-zero codeword.

- - Uncoded BPSK ""*- RS (255,239, 17) -e- RS (255, 223, 33)

FIGURE 7.3: Error performances for (255,223, 33) and (255,239, 17) RS codes.

Section 7.6

Frequency-Domain Decoding

2S5

The two most commonly used RS codes for error control in data communication and storage systems are the (255, 223, 33) and the (255, 239, 17) RS codes over GF(2 8 ). The (255, 223, 33) RS code is also a NASA standard code for space and satellite communications. Both codes are decoded with either the Berlekamp algorithm or the Euclidean algorithm. Their error performances with BPSK transmission over the A WGN channel are shown in Figure 7.3. 7,15

!HtlEQUJIENCV~IDJOMAiN IDJIE(OIDJ!NG

So far, BCH and RS codes have been decoded in the time domain; however, these codes also can be decoded in the frequency domain. fa this section, we first give a spectral description of these codes and then present a frequency-domain decoding algorithm for them [8, 17]. Consider the Galois field GF(q) with characteristic p (see Section 2.2). Let 1 be the unit element of GF(q). Then, pis the smallest positive integer such that the sum p

2) = 1 + 1 + ... + 1 = 0. ~

i=l

p

For any nonnegative integer n, the sum 11

Ll=~=A, i=l

11

where A is the remainder resulting from dividing n by p. Mathematically, we write

A= n(modulo

p).

(7.54)

Note that A is an element in GF(q). Let w(X) = vo + v1X + ... + v11 _1xn-l be a polynomial over GF(q), where n divides q 111 - 1, and n -=I=- 1. Let a be an element of order n in GF(q 111 ). Then, a" = 1 and is a root of X 11 - 1. The Galois field Fourier transform of w(X) is defined as the polynomial (7.55)

over GF(q 111 ), where for O :S j < n, 11-l

v1 = w(ai) = L

v;aii.

(7.56)

i=O

The coefficient Vi is called the jth spectral component of V(X). Given V(X), the polynomial w(X) can be determined by taking the inverse Fourier transform of V(X), as shown in Theorem 7.1.

256

Chapter 7

Nonbinary BCH Codes

THEOREM 7.1 Let V(X) Fourier transform of v(X)

= Vo+ ViX + ... + V,1-lxn-l be the Galois field = vo + v1X + ... + Vn-lxn-l. Then, 1

v; =

.

n(modulo p) l

V(a-') n-1

= n(modulo p)

..

(7.57)

L VJa-'l · J=O

Proof. Factor xn - 1 as xn - 1 = (X - l)(xn-l

+ xn-Z + ... + X + 1).

Because a is a root of xn - 1, and a ::/= 1, a must be a root of xn-l ... + X + l; that is, 1 +a+ ... + an-2

+ xn-Z +

+ an-1 = 0.

For any integer r that is not a multiple of n (i.e., r ::/= O(modulo p)), (a't = 1, and hence a' is also a root of xn - 1. Since a' ::/= 1, a' must be a root of xn-1 + xn-Z + ... + X + 1, and hence n-1

Lari= 0. J=O

(7.58)

Now, consider V(a-i). It follows from (7.55) and (7.56) that n-1

n-1

V(a-i) =La-ii L

J=O n-1

vzali

l=O

(7.59)

n-1

= I:vz La(l-i)J_

l=O

J=O

It follows from (7.58) that for l ::/= i, the second sum on the right side of (7.59) is equal to 0. For l = i, the second sum becomes 1 + 1 + ... + 1 = n(modulo p). Consequently, (7.59) becomes V(a-i) = v; · n(modulo p).

(7.60)

Note that p divides q and does not divide qm - 1. Because n is a factor of qm -1, then n and pare relatively prime. Therefore, n(modulo p) ::/= 0. It then follows from (7.60) that v;

This completes the proof.

=

1

n(modulo p)

.

V(a-').

Q.E.D.

Section 7.6

Frequency-Domain Decoding

2:57

The polynomials v(X) and V(X) form a transform pair. V(X) is the spectrum polynomial ( or simply spectrum) oh(X). From (7.56) and (7.57), we readily see that the transform pair have the following properties: ]., The jth spectral component Vj is zero if and only if otj is a root of v(X). Z, The ith component oh(X) is zero if ifot-i is a root of V(X). Now, we are ready to characterize BCH and RS codes in the frequency domain. Consider a primitive q-ary t-error-correcting BCH code of length n = q 111 - 1 whose generator polynomial !!!;(X) has a, a 2 , ... , a 21 as roots. Recall that a polynomial v(X) of degree n 1 or less over GF(q) is a code polynomial if and only if v(X) is divisible by !!!;(X). This is equivalent to saying that v(X) is a code polynomial if and only if v(X) has a, a 2 , ... , a 21 as roots. Let v(X) = vo + v1X + ... + v11 _1x 11 - 1 be a code polynomial in a primitive q-ary t-error-correcting BCH code of length n = q 111 - 1, and let V(X) = Vo+ V1X + ... + V11 _1x 11 - 1 be its Fourier transform. It follows from the first property of the transform pair (v(X), V(X)) that the 2t consecutive spectral components of V(X) from position X to position X21 are zero; that is, V1 = V2 = ... = Vir = 0. Consequently, a primitive q-ary t-error-correcting BCH code of length n = q 111 - 1 is the set of polynomials of degree n - 1 or less over GF(q) whose Fourier transforms have 2t consecutive zero spectral components from position X to position X21 . This description is a frequency-domain characterization of a BCH code. For a q-ary RS code, both v(X) and its Fourier transform V(X) are polynomials over GF(q). fo the frequency domain, at-error-correcting RS code with symbols from GF(q) consists of all the polynomials V(X) =Vo+ V1X

+ ... + 'lli,-1X

11 -

1

of degree of n - 1 or less for which V1

=

V2

= ... =

V21

= 0.

IEXAMIPILIE 1 .6

Again, we consider the triple-error-correcting RS code of length 15 over GF(2 4) with generator polynomial g(X)

= (X + ot)(X + a 2)(X + a 3)(X + a 4)(X + a 5 )(X + a 6 ) = (){6

+ ot9X + ot6X2 + ot4X3 + 0/14x4 + OllOxS + x6.

Substituting X with of g(X):

oti

for O ::: i < 15 in g(X), we obtain the Fourier transform

G(X) =as+ all x7

+ otX8 + alOX9 + ot3Xl0 + ot7Xll + ot9X12 + ot7 X13 + ot4X14.

(Table 2.8 for GF(2 4) is used for computations.) Because g(X) has a to a 6 as roots, G(X) has zero spectral components from X1 to X 6 ; that is, G1 = G2 = ... = G6 = 0. Now, consider the code polynomial v(X) = (X 8 + X

+ l)g(X),

(7.61)

258

Chapter 7

Nonbinary BCH Codes

which has a to a 6 and a 10 as roots. The Fourier transform of v(X) is V(X) =as+ al3 X7 + all XS+ al2x9 + a8Xll + a10X12 + Xl3 + a8Xl4_

We see that V(X) has seven zero spectral components at locations X 1 to X 6 and X 10 . Before we discuss decoding of BCH and RS codes in the frequency domain, we present an important property of Galois field Fourier transforms. Let a(X) = ao + a1X + ... + an-lxn-l, b(X) = bo + b1X + ... + b11 _1xn-l

be two polynomials over GF(q ). We define the following product of a(X) and b(X): c(X)

L,.

a(X) · b(X)

co+ qX + ...

where c;

= a;

+ Cn-lxn- 1 ,

· b; for O ::: i < n. Let

and

be the Fourier transforms of a(X) and b(X), respectively. Then, the Fourier transform of the product polynomial c(X) = a(X) . b(X) is the convolution of A(X) and B(X) given in Theorem 7.2. THEOREM

7.2 The Fourier transform of c(X) = a(X). b(X) is given by C(X) =Co+ C1X

+ ... + Cn-lxn-l,

where for O::: j < n, (7.62)

Proof. Taking the Fourier transform of c(X), we have n-1

CJ

=

n-1

L c;aij = L a;b;aii. i=O

(7.63)

i=O

Expressing a; in terms of the inverse transform of A(X), we have l

n-1

'"°' A ka

a;= -n-(m~od_u_l_o_p_) L

k=O

-ik ·

(7.64)

Section 7.6

Frequency-Domain Decoding

25!\lJ

Combining (7.63) and (7.64), we have n-1 C·1

=

l

n(modulo p)

11-l

LA1cLb;o/U-kl; k=O

(7.65)

i=O

however, n-l

Lb;oti(}-lc) = Bj-lc· i=O

(7.66)

From (7.65) and (7.66), we have 1

C·1

=

11-l

~ A1cB1 -k· n(modulo p) L

(7.67)

k=O

This completes the proof. Now, we consider decoding of BCH and RS codes in the frequency domain. Let 1r(X) = ro + r1X + ... + r11 _1xn-l be the received polynomial, where n = q 111 1. Then, 1r(X) = w(X) +e(X), where w(X) and ie(X) are the transmitted code polynomial and the error polynomial, respectively. The Fourier transform of 1r(X) is JR(X) =Ro+ R1X + ... + R11 _1X 11 -1, where n-l

Rj

= 1r(otj) =

L r;otij. i=O

(7.68)

Let V(X) =Vo+ V1X + ... + V,,-1xn-l, and lE(X) = Eo + E1X + ... + E 11 _1X 11 - 1 be the Fourier transforms of w(X) and ie(X), respectively. Then, JR(X) = V(X) + lE(X), with (7.69)

for O :S j < n. Because w(X) is a code polynomial that has ot, ot 2 , ... , cx 21 as roots, then Vj = 0

(7.70)

for 1 :S j :S 2t. From (7.69) and (7.70), we find that for 1 :S j :S 2t, (7.71)

Let§= (S1, S2, ... , S21 ) be the syndrome oh(X). Then, for 1 :S j :S 2t, (7.72)

260

Chapter 7

Nonbinary BCH Codes

It follows from (7.68), (7.71), and (7.72) that Rj

= Ej = Sj = r(aj)

(7.73)

for 1 ::: j ::: 2t. This result says that the 2t spectral components R1, R2, ... , R2r of R(X) are the 2t syndrome components and are equal to the 2t spectral components £1, £2, ... , E21 of the FouriertransformE(X) of the error polynomial e(X). Ifwe can determine the spectral components Eo, £21 +1, ... , £ 11 _1, then E(X) is determined, and the inverse transform of E(X) gives the error polynomial e(X). Decoding is accomplished by subtracting e(X) from r(X). Suppose there are v ::: t errors, and (7.74) The error-location numbers are then aj1, ah, ... , aj". The error-location polynomial is a

(X) = (1 - aj1X)(l - ah X) ... (1 - aj" X).

which has a- ji, a-h, ... , a-jv as roots. Hence, for 1 ::: i ::: v, (7.75) Note that a(X) is a polynomial over GF(q 111 ). We may regard a(X) as the Fourier transform of a polynomial A(X) =AO+ A1X + ... + ).. 11 _1X 11 -l over GF(q ), where A·= 1

1 . a(a- 1 ) n(modulo p)

(7.76)

for O ::: j < n. It follows from (7.75) and (7.76) that for 1 ::: i ::: v, Aj; =

(7.77)

0.

Consider the product A(X) · e(X) = L';:6).. j section. From (7.74) and(7.77), we readily see that l(X) · e(X)

= O;

·

e j X as defined earlier in this

(7.78)

that is, ).. j · e j = 0 for O ::: j < n - 1. Taking the Fourier transform of l(X) · e(X) and using (7.62; Theorem 7.2) and (7.78), we have n-1 LOkEj-k k=O

=0

(7.79)

for O::: j < n. Since the degree of u(X) is v, uk = 0 fork > v. Then, (7.79) becomes (7.80)

Section 7.6

for O s j < n -1. Because CTo form:

Frequency-Domain Decoding

Mi

= 1, the preceding equation can be put in the following (7.81)

for Os j < n. Since £1, E2, ... , E2 1 are already known (see (7.73)), it follows from (7.81) that we obtain the following recursive equation for computing E21+1 to £ 11 _1: (7.82) fort+ 1

sls

n - 1 - t. Setting j

= v in (7.81), we obtain

From this equation, we find that 1 Eo =--(Ev+ CT1Ev-l

+ · · · + O'v-1E1).

(7.83)

O'v

From (7.82) and (7.83), we can determine the entire lE(X). Taking the inverse transform of V(X) = li(X) - lE(X), we obtain the decoded code polynomial v(X), which completes the decoding. The error-location polynomial can be computed by using the Berlekamp iterative algorithm. The decoding consists of the following steps: ]_o Take the Fourier transform li(X) of ll"(X). lo Find u(X). 2t Compute lE(X). 4lo Take the inverse transform w(X) of V(X) = li(X) - lE(X).

A transform decoder is depicted in Figure 7.4. IEXAM!rLIE 7/.7/

Again, consider the (15, 9) RS code over GF(24) given in Example 7.2. Suppose a code polynomial v(X) is transmitted, and ll"(X) = a 7 X 3 + a 3 X 6 + a 4 X 12 is received. The Fourier transform of ll"(X) is R(X)

= a12 x + x2 + a14 x3 + aw x4 + a12 x6 +x1 +a14xs +a1ox9 +a12x11 +x12 + a14xn + a10x14_

The coefficients of powers X to X 6 give the syndrome components; that is, S1 = a 12 , S2 = 1, S3 = a 14, S4 = a 10 , Ss = 0, and S6 = a 12 . They are also the spectral components E1, E2, E3, E4, Es, and E6 of the error spectral polynomial IE(X). Using the Berlekamp algorithm based on the syndrome (S1, S2, S3, S4, Ss, S6), we find the error-location polynomial

262

Chapter 7

Nonbinary BCH Codes Transform computer

f---------~----~


Buffer register

Recursive extension

f4----'

E(X)

V(X) Inverse transform computer

v(X) 1----~

FIGURE 7.4: A transform decoder for a q-ary BCH or RS code.

(see Example 7.2). From (7.82), we obtain the following recursion equation for computing spectral components £7 to £14 of E(X): Ez+3

= CJ1Ez+2 + CJ2E1+1 + CJ3E1 = a 7 Ez+2 + a 4 E1+1 + a 6E1

for 4:::: l :::: 11. We compute the spectral component Eo from 1

Eo = -(£3 + CJ1E2 + CJ2E1) CJ3 = a- 6 (£3 + a 7 E2 + a 4 E1)

= a-6(al4 +

a7 + a16)

=0. The resultant error spectral polynomial is E(X)

= a12X + +x7

x2 + a14X3 + a10X4

+ a14x8 +

+x12 + Q'.14 x13

+ al2x6

al0x9 + a12xll

+ alO x14_

We find that E(X) = R(X), and V(X) = 0. Therefore, the decoded codeword is the all-zero codeword. The inverse transform of E(X) is e(X) = a 7 x 3 + a 3x 6 + a 4x12. This is exactly the same result as given in Example 7 .2.

Section 7.7

Correction of !Errors and Erasures

263

A q-ary t-error-correcting BCH (or RS) code can be used to correct all combinations of v symbol errors and e symbol erasures provided that the inequality v

+ e/2 :St

(7.84)

holds. Each of the decoding algorithms presented in the last three sections can be modified to do the job. Suppose the received polynomial Ir(X) contains v symbol errors at positions . Because xi1, Xi 2 , ••• , Xi,,, and e symbol erasures at positions X i1, X h, · · . , the erased positions are known, decoding is to find the locations and values of the errors and the values of the erased symbols. The erasure-location numbers corresponding to the erased positions Xii, X h, ... , Xi, are aii , ah, ... , ale. We form the erasure-location polynomial: /3(X)

n e

~

(1 - ai1 X).

(7.85)

l=l

Now, we fill thee erased positions in Ir(X) with zeros (or arbitrary symbols from GF(q )). This substitution of e zeros into the erased positions in Ir(X) can introduce

up toe additional errors. Let Il'*(X) denote the modified received polynomial. Let

n V

o-(X)

~

(1 -

o/k X)

(7.86)

k=l

be the error-location polynomial for the errors in Ir(X) at positions xi1, xi2, ... , Xi,,. Then, the error-location polynomial for the modified received polynomial ir*(X) is y(X)

= u(X)f:J(X),

(7.87)

for which f:J(X) is known. Now, decoding is to find u(X) and the error-value evaluator Zo(X) fon*(X). We compute the syndrome polynomial

from the modified received polynomial li'*(X). Then, the key equation becomes u(X)P(X)§(X)

= Zo(X)mod X 21 .

(7.88)

The decoding problem is to find the solution (u(X), Zo(X)) of this equation such that u(X) has minimum degree v, and deg Zo(X) < v + e. Since P(X) and §(X) are known, we can combine them. Let 1r(X) ~ [P(X)§(X)b

=

T1

+ T2X + · · · + T21X 21 ~l

(7.89)

264

Chapter 7

Nonbinary BCH Codes

denote the polynomial that consists of the 2t terms of {l(X)S(X) from Then, we can write the key equation of (7.88) as
= Zo(X)mod X21 .

x 0 to x 21 - 1 . (7.90)

This key equation may be solved by using either Euclid's or Berlekamp's algorithm. The Euclidean algorithm for error/erasure decoding consists of the following: 1. Compute the erasure-location polynomial /l(X) using the erasure information from the received polynomial r(X). 2. Form the modified received polynomial r*(X) by replacing the erased symbols with zeros. Compute the syndrome polynomial S(X) from r*(X). 3. Compute the modified syndrome polynomial T(X) = [/l(X) S(X)]zi. 4. Set the following initial conditions:

z6- 1\x) = X (T(-l)(X)

21

z60) (X) = T(X),

'

= 0,

and (T(O)(X)

= 1.

5. Execute the Euclidean algorithm iteratively as described in Section 7.5 until a step p is reached for which deg Z 0(p) (X) <

t {

+ e/2,

t+(e-1)/2,

for even e, for odd e.

(7.91)

Then, set u(X) = uCPl(X), and Zo(X) = z~\X). 6. Find the roots of u(X) and determine the error locations in r(X). 7. Determine the values of errors and erasures from Zo(X) and y(X) u(X)/l(X). The error values are given by e;k

for 1 :s k

=

-Zo(a-ik) y'(a-ik)

(7.92)

:s v, and the values of the erased symbols are given by (7.93)

for 1 :s l :s e, where y' (X) is the derivative of the overall error/erasure-location polynomial y(X) = u(X)/l(X); that is, d dX

I

y (X) = -y(X) v+e

=

-a Laj1 l=l

n

v+e

(1 - aj; X).

(7.94)

i=l,i/l

(a is the constant# 1 that may appear in u(X) when the Euclidean algorithm is used).

Section 7.7

Correction of Errors and !Erasures

2155

Again consider the triple-error-correcting RS code of length 15 over GF(2 4 ) generated by g(X) = (X + a)(X + a 2)(X + a 3)(X + a 4)(X + a 5 )(X + a 6). This code is capable of correcting all combinations of two or fewer errors and two or fewer erasures. Suppose the all-zero codeword is transmitted, and the received vector is Ir=

where

(000

* 00 * OOa 00a 4 00),

* denotes an erasure. The received polynomial is

Because the erased positions are X3 and X6 , the erasure-location polynomial is

+ a 3 X)(l + a 6 X) = 1 + a 2X + a 9X 2 .

{j(X) = (1

Replacing the erased symbols with zeros, we obtain the following modified received polynomial: The syndrome components computed from ll'*(X) are S1 S2 S3

= rr*(a) = a 8 , = ll'*(a2) = a11, = ll'*(a3) = a9,

S4 Ss S6

= ll'*(a 4) = 0, = ll'*(a 5 ) = 1, = ll'*(a6) = as.

The syndrome polynomial is then §(X)

= a 8 + a 11 X + a 9 X 2 + X 4 + a 8 X 5 ,

and the modified syndrome polynomial is 1f(X)

= [/J (X)§(X) hr = Cl8 + Cll4x + a4x2 + a3x3 + al4x4 + xs.

Using the Euclidean decoding algorithm, we set the initial conditions as follows:

1t \x) = x 1

6

,

z6°)(X)

= T(X),

u<- 1\x) = 0, and u< 0l(X) = 1. Since t = 3 and e = 2, the algorithm terminates when deg Zo(X) < 4. Executing the algorithm, we obtain Table 7.6. The error-location polynomial is

+ a 8X + a 6 X 2) a(l + a 9 X)(l + a 12 X).

u (X) = a(l =

266

Chapter 7

Nonbinary BCH Codes

The two roots of u(X) are a- 9 and a- 12 . The reciprocals of these two roots give the error locations, a 9 and a 12 . The error-value evaluator is Zo(X)

= a9+

2 3 8 a X + aX + aX .

The overall error/erasure-location polynomial is y(X)

= u(X)/3(X) = a(l + a 3 X)(l +

a 6 X)(l + a 9 X)(l + a 12 X),

and its derivative is y'(X) = o: 4 (1 + a 6 X)(l + a 9X)(l + a 12 X) + a 7 (1 + a 3 X)(l + a 9X)(l + o: 12 X) + o: 10 (1 + a 3 X)(l + a 6 X)(l + a 12 X) + o: 13 (1 + a 3 X)(l + a 6 X)(l + a 9X).

It follows from (7.92) and (7.93) that the error values at positions e9

=

-Zo(a-9) y'(a-9)

e12

=

-Zo(a- 12 ) _ y'(a 12 )

x 9 and X12 are

o:13

= -al2 o: 3

=a,

= 14 = a a

-11

=a

4

,

and the values of the erased symbols at positions X 3 and X 6 are

Then, the estimated error polynomial is e(X)

= aX 9 +a 4 X 12 .

Subtracting e(X) from ir*(X), we obtain the decoded code polynomial v(X) = 0, which is the transmitted code polynomial.

EXAMPLE 7.9

Consider the (63, 55, 9) RS code generated by g(X)

= (X + a)(X + a 2 )(X + a 3 )(X + a 4 )(X + a 5 )(X + a 6 )(X + a 7 )(X + a 8 ) = xs + a43X7 + a59X6 + 0:31 xs + alOX4 + a40X3 + al4x2 + a7 X + o:36.

Section 7,7

Correction of Errors and Erasures

2671

TABLE 7 Ji: Steps for finding the error-location polynomial and error-value

evaluator of the RS code given in Example 7,6,

i

Z(li) (X)

-1

x6

-

0

0

1I'(X)

-

1

1

a7 + a3 X + x2 + alOX3 + a8X4

a14 + X

a14 + X

2 3 a 9 + a 8 X + otX + otX

ots + ot7 X

a +a 9 X +a 7 X 2

u(X)

%1(X)

I)])

This code is capable of correcting all combinations of three or fewer errors and two or fewer erasures, Suppose the all-zero codeword is transmitted, and the received vector is Il'

=

(0 0 0 0 0 Oa 15 0 0 00 0 0 0 0 0 0 0 0 0 a 37 0 0 00 0 0 0 * 0 0 0

ooa 4 oooooooooooooooooo * 000000000), The received polynomial is Jr=

0/15 x6 + 0/37 x20 + (*)X28 + ot4X34 + (*)X53'

Because the erased positions are X28 and X53 , the erasure-location polynomial is /J(X)

= (1 + a 28 X)(l + a 53 X) = 1 +a39x +ot18x2,

Replacing the erased symbols with zeros, we obtain the following modified received polynomial: Il'*(X) = a15 x6 + ot37 x20 + a4 X34,

The syndrome components computed from Il'*(X) are S1

S2 S3 S4

= Il'*(a) = ir*(a 2) = ir*(a 3) = ir* (a4)

= al9, = a, = 1, = a22'

Ss = S6 = S7 = Ss =

Il'*(as) = n'*(a 6) = Il'*(a 7) = ir* (a8) =

a43,

a4, a 58 , ot28,

The syndrome polynomial is then §(X) = al9 + aX + x2 + a22 X3 + a43 x4 + a4 xs + a58 x6 + a28 x7,

and the modified syndrome polynomial is 1I'(X)

= [fi (X)§(X) ht = al9 + a59 X +

aX2 + a41 X3 + a32 x4 + ot62 xs + ot60 x6 + a48 x7,

Using the Euclidean decoding algorithm, we set the initial condition as follows: 1 z6- 1(x) = X 8 ,
z6°)(X)

= 0, and


= 1f(X),

\x) =

L

268

Chapter 7

Nonbinary BCH Codes

TABLE 7.7: Steps finding the error-location polynomial and error-value evaluator

of the (63,55,9) RS code over GF(26 ) given in Example 7.9.

i

zO\x) 0

-1

xs

-

0

T(X)

-

1

0

1 Q'.46 + a48x +


qi(X)

a58x2 + a30x3

a27 + alSX

a27 + alSX

a22 + a36x

Q'.38 + a44x + a51 x2

a48 + a56X

Q'.47 + a22x + a42x2 + a44x3

+a25 x4 + as xs + al2 x6

2

a57 + a31 X + a56 x2 + a44 X3 +al7 x4 + al9 xs

3

a3 + a53 X + a30 x2

+a24 x3 + al3 x4

Since t = 4 and e = 2, the algorithm terminates when deg Zo(X) < 5. Executing the algorithm, we obtain Table 7.7. The error-location polynomial is
= a47 (l +

a38 X + ass x2 + a60 X3)

= a 47 (1 +

6 20 34 a X)(l + a X)(l + a X).

The three roots of
The overall error/erasure-location polynomial is y(X)

= u(X)/J(X) = a 47 (1 + a 6 X)(l + a 20 X)(l + a 28 X)(l + a 34 X)(l + = Q'.47 + a28x + a18x2 + a48x3 + a12x4 + a62x5,

and its derivative is

r' (X)

= a2s + a4s x2 + a62 x4.

The error values at positions X 6 , x 20 , and

x 34 are

a 53 X)

Problems

and the values of the erased symbols at positions

-Z0 (cc 28 ) h8 = y I ( a -28) f53

=

-Zo(a-53) y' ca-53)

2t!i'91

x 28 and X53 are 0

= 29 a = Q, = ~ = 0. ()/,13

Then, the estimated error polynomial is e(X)

= a1s x6 + ot37 x20 + ot4 X34.

Subtracting
=

@,

!P'IROIB:llEM§ 7Jl Consider the triple-error-correcting RS code given in Example 7.2. Find the code polynomial for the message 1I1(X)

= 1 +a 5 X +aX 4 +a 7 X 8 .

72, Using the Galois field GF(25 ) given in Appendix A, find the generator polynomials of the double-error-correcting and triple-error-correcting RS codes of length 31. 73 Using the Galois field GF(26 ) given in Table 6.2, find the generator polynomials of double-error-correcting and triple-error-correcting RS codes of length 63. 7o4l Consider the triple-error-correcting RS code of length 15 given in Example 7.2. Decode the received polynomial

using the Berlekamp algorithm. 7/5 Continue Problem 7.4. Decode the received polynomial with the Euclidean algorithm. 7Jf; Consider the triple-error-correcting RS code of length 31 constructed in Problem 7.2. Decode the received polynomial rr(X)

= a2 + a21 x12 + a7 x20

using the Euclidean algorithm. 7o7 Continue Problem 7.6. Decode the received polynomial in the frequency domain using transform decoding. 7Jj For the same RS code of Problem 7.6, decode the following received polynomial with two erasures:

with the Euclidean algorithm. 7!~ Prove that the dual code of a RS code is also a RS code. '7ol@ Prove that the (2111 -1, k) RS code with minimum distanced contains the primitive binary BCH code of length 2111 - 1 with designed distance d as a subcode. This subcode is called a subfield subcode.

270

Chapter 7

Nonbinary BCH Codes

7.11 Let a be a primitive element in GF(2m). Consider the (2m - 1, k) RS code of length of 2m - 1 and minimum distanced generated by a 2) ... (X - ad- 1).

g(X) = (X - a)(X

Prove that extending each codeword v parity-check symbol

= (vo, v1 , · · ·

, v2,,,_ 2 )

by adding an overall

2111 -2 Voo

=-

L

v;

i=O

produces a (2m, k) code with a minimum distance of d + l. 7.12 Consider at-symbol error-correcting RS code over GF(2m) with the following parity-check matrix:

n-['.

a2

a a2

(a2)2

a" I ] (a2r-l

a2t

(a2t)2

(a2t\n-l

'

where n = 2m -1, and a is a primitive element in GF(2m ). Consider the extended Reed-Solomon code with the following parity-check matrix: 0 0

1 0

0 1

0 0

H1=

H

Prove that the extended code also has a minimum distance of 2t + l. 7.13 Let a(X) = ao + a1X + · · · + ak-lxk-l be a polynomial of degree k - 1 or less over GF(2m). There are (2ml such polynomials. Let a be a primitive element in GF(2m). For each polynomial a(X), form the following polynomial of degree 2m - 2 or less over GF(2m): v(X)

= a(l) + a(a)X + a(a 2)X 2 + ... + a(a 2"'- 2)x 2111 - 2.

Prove that the set {v(X)} forms the (2m -1, k) RS code over GF(2m). (Hint: Show that v(X) has a, a 2 , ... , a 2"' -k-l as roots). This original definition of a RS code is given by Reed and Solomon [1 ].

BIBLIOGRAPHY

1. I. S. Reed and G. Solomon, "Polynomial Codes over Certain Fields," J. Soc. Ind. Appl. Math., 8: 300-304, June 1960. 2. D. Gorenstein and N. Zierler, "A Class of Cyclic Linear Error-Correcting Codes in pm Symbols," J. Soc. Ind. Appl. Math., 9: 107-214, June 1961. 3. R. T. Chien, "Cyclic Decoding Procedure for the Bose-ChaudhuriHocquenghem Codes," IEEE Trans. Inf Theory, IT-10: 357-63, October 1964.

Bibliography

zn

4lo G.D. Forney, "On Decoding BCH Codes," IEEE Trans. Inf Theory, IT-11: 549-57, October 1965.

5o E. R. Berlekamp, Algebraic Coding Theory, McGraw-Hill, New York, 1968. lo>o

Y. Sugiyama, M. Kasahara, S. Hirasawa, and T. Namekawa, "A Method for

Solving Key Equation for Decoding Goppa Codes," Inf Control, 27: 87-99, January 1975. 7o W. C. Gore, "Transmitting Binary Symbols with Reed-Solomon Codes," Proc. Conf Infor. Sci. and Syst., Princeton, N.J., 495-97, 1973.

~t R. E. Blahut, "Transform Techniques for Error-Control Codes," IBM J. Res. Dev., 23(3), 299-315 May 1979. i\llo T. Kasami and S. Lin, "On the Probability of Undetected Error for the Maxi-

mum Distance Separable Codes," IEEE Trans. Commun., COM-32: 998-1006, September 1984. ll!bo E. F. Assmus, Jr., H.F. Mattson, Jr., and R. J. Turyn, "Cyclic Codes," Scientific Report No. AFCRL-65-332, Air Force Cambridge Research Labs, Bedford, Mass., April 1965. Ho T. Kasami, S. Lin, and W.W. Peterson, "Some Results on Weight Distributions

of BCH Codes," IEEE Trans. Inf Theory, IT-12(2): 274, April 1966. Uo G.D. Forney, Jr., Concatenated Codes, MIT Press, Cambridge, 1966. Ho J. L. Massey, "Shift-Register Synthesis and BCH Decoding," IEEE Trans. Inf

Theory, IT-15: 122-27, January 1969. 14L W. W. Peterson and E. J. Weldon, Jr., Error-Correcting Codes, 2d ed., MIT Press, Cambridge, 1970.

150 F. J. MacWilliams and N. J. A. Sloane, The Theory of Error-Correcting Codes, North Holland, Amsterdam, 1977. 160 G. C. Clark, Jr., and J. B. Cain, Error-Correcting Coding for Digital Communications, Plenum Press, New York, 1981. 170 R. E. Blahut, Theory and Practice of Error Control Codes, Addison-Wesley, Reading, Mass., 1983. lllt A. M. Michelson and A. H. Levesque, Error-Control Techniques for Digital Communication, John Wiley, New York, 1985.

190 S. B. Wicker, Error Control Systems for Digital Communication and Storage, Prentice Hall, Englewood Cliffs, N.J., 1995. Wo S. B. Wicker and V. K. Bhargava, Reed-Solomon Codes and Their Applications, IEEE Press, New York, 1994.

272

Chapter 7

Nonbinary BCH Codes

21. T. Kasami, S. Lin, and W. W. Peterson, "Some Results on Cyclic Codes Which Are Invariant under the Affine Group," Scientific Report AFCRL-66-622, Air Force Cambridge Research Labs, Bedford, Mass., 1966. 22. J. K. Wolf, "Adding Two Information Symbols to Certain Nonbinary BCH Codes and Some Applications," Bell Syst. Tech. J., 48: 2405-24, 1969.

23. M. Morii and M. Kasahara, "Generalized Key-Equation of Remainder Decoding Algorithm for Reed-Solomon Codes," IEEE Trans. Inf Theory, IT-38 (6): 1801-7, November 1992.

M@]@rg'ttf=l(Q)g~~ []®[@(cl@~~® @u1d lrBfTI~'[~ (G~CQJITTfTI®1Cff (@d~~ Majority-logic decoding is a simple and effective scheme for decoding certain classes of block codes, especially for decoding certain classes of cyclic codes. The first majority-logic decoding algorithm was devised in 1954 by Reed [1] for the class of RM codes presented in Chapter 4. Reed's algorithm was later extended and generalized by many coding theorists. The first unified formulation of majority-logic decoding was due to Massey [2]. Most majority-logic decodable codes found so far are cyclic codes. Important cyclic codes of this category are codes constructed based on finite geometries, namely, Euclidean and projective geometries. These codes are called finite geometry codes and they contain punctured RM codes in cyclic form as a subclass. A special subclass of finite geometry codes forms a special subclass of low-density parity-check codes that will be discussed in Chapter 17. Finite geometry codes were first investigated by Rudolph [3] in 1967. Rudolph's work was later extended and generalized by many coding researchers, from the late 1960s to the late 1970s. fa this chapter we first introduce majority-logic decoding based on orthogonal parity-check sums formed from the parity-check matrix or the dual space of a code. Then, we present several classes of cyclic majority-logic decodable codes.

Consider an (n, k) cyclic code C with parity-check matrix lHI. The row space of lHI is an (n, n - k) cyclic code, denoted by Cd, which is the dual code of C, or the null space of C. For any codeword win C and any codeword win Cd, the inner product of w and w is zero; that is, W · 'o/

= wovo + w1v1 + · · · + W

11

-lVn-1

= 0.

(8.1)

fa fact, an 11-tuple wis a codeword in C if and only if for any vector win Cd, w · w = 0. The equality of (8.1) is called a parity-check equation. Clearly, there are z(n-k) such parity-check equations. Now, suppose that a codeword win C is transmitted. Let ie = (eo, ei, · · · , e11 _1) and ir = (ro, r1, · · · , r 11 _1) be the error vector and the received vector, respectively. Then, (8.2) ir = w+ ie.

For any vector win the dual code Cd, we can form the following linear sum of the received digits: (8.3) A= 'W. Jr= woro + w1r1 + ... + Wn-lrn-1, which is called a parity-check sum or simply check-sum. If the received vector ir is a codeword in C, this parity-check sum, A, must be zero; however, if iris not a

274

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

codeword in C, then A may not be zero. Combining (8.2) and (8.3) and using the fact that w · v = 0, we obtain the following relationship between the check-sum A and error digits in e: (8.4)

An error digit ez is said to be checked by the check-sum A if the coefficient wz = 1. In the following, we show that certain properly formed check-sums can be used for estimating the error digits in e. Suppose that there exist J vectors in the dual code Cd,

Wj = (w10, WJl, · · · , WJ,n-l),

that have the following properties: 1. The (n - l)th component of each vector is a 1; that is, Wl,n-l = W2,n-l = · · · = WJ,n-l = l.

2. For i I- n - 1, there is at most one vector whose ith component is a 1; for example, if w1,; = 1, then w2,; = w3,; = · · · = w1,; = 0. These J vectors are said to be orthogonal on the (n - l)th digit position. We call them orthogonal vectors. Now, we form J parity-check sums from these J orthogonal vectors: A1

= W1

A2

= w2 · r

·r

= w10ro + wnri + · · · + w1,n-lrn-l = w20ro + w21r1 + · · · + w2,n-lrn-l (8.5)

A1

= Wj

·r

= W1oro + wnri + · · · + WJ,n-lrn-l·

Because w1,n-l = w2,n-l = · · · = WJ,n-l = 1, these J check-sums are related to the error digits in the following manner: A1

= w10eo + w11e1 + · · · + w1,n-2en-2 + en-l

A2

=

w20eo

+ w21e1 + · · · + W2,n-2en-2 + en-l (8.6)

We see that the error digit en-l is checked by all the preceding check-sums. Because of the second property of the orthogonal vectors w1, w2, · · · , w 1 any error digit

Section 8.1

One-Step Majority-Logic Decoding

275

other than e11 _1 is checked by at most one check-sum. These J check-sums are said to be orthogonal on the error digit e11 _1. Since w;,j = 0, or 1, each of the foregoing check-sums orthogonal on e11 _1 is of the form Aj

= en-1 +

L

e;.

i,tcn-1

If all the error digits in the sum of A j are zero for i f. n - l, the value of e11 _1 is equal to A j (i.e., e11 _1 = A j ). Based on this fact, the parity-check sums orthogonal on e11 _1 can be used to estimate e11 _1 or to decode the received digit r11 _1. Suppose that there are LJ /2J or fewer errors in the error vector s:: = (eo, ei, · · · , e11 _1) (i.e., LJ /2J or fewer components of e are 1). If e11 _1 = 1, the other nonzero error digits can distribute among at most LJ /2J - 1 check-sums orthogonal on e11 _1. Hence, at least J - LJ /2J + 1, or more than half of the checksums orthogonal on e 11 _1, are equal to e,,_1 = 1; however, if e 11 _1 = 0, the nonzero error digits can distribute among at most LJ /2J check-sums. Hence, at least J - LJ /2J or at least half of the check sums orthogonal on e11 _1 are equal to e11 _ 1 = 0. Thus, the value of e11 _1 is equal to the value assumed by a clear majority of the parity-check sums orthogonal on e11 _1; if no value is assumed by a clear majority of the paritycheck sums (i.e., there is a tie), the error digit e11 _1 is zero. Based on the preceding facts, an algorithm for decoding e11 _1 can be formulated as follows: The error digit e11 _ 1 is decoded as 1 if a clear majority of the parity-check sums orthogonal on e11 _1 isl; otherwise, e11 _1 is decoded as 0.

Correct decoding of e11 _1 is guaranteed if there are LJ /2J or fewer errors in the error vector le. Hit is possible to form J parity-check sums orthogonal on e11 _1, it is possible to form J parity-check sums orthogonal on any error digit because of the cyclic symmetry of the code. The decoding of other error digits is identical to the decoding of e11 _1. The decoding algorithm just described is called one-step majority-logic decoding [2]. H J is the maximum number of parity-check sums orthogonal on en-1 ( or any error digit) that can be formed, then, by one-step majority-logic decoding, any error pattern of LJ /2J or fewer errors can be corrected. The parameter fML = LJ /2J is called the majority-logic error-correcting capability of the code. Let dmin be the minimum distance of the code. Clearly, the one-step majority-logic decoding is effective for this code only if fML = LI /2J is equal to or close to the error-correcting capability t = l(dmin - l)/2J of the code; in other words, J should be equal to or close to dmin - 1. DJEJFrnnmN ft]_ A cyclic code with minimum distance dmin is said to be completely orthogonalizable in one step if and only if it is possible to form J = dmin - 1 parity-check sums orthogonal on an error digit. At this point, a clarifying example will be helpful. IEXAMIPllE K 1

Consider a (15, 7) cyclic code generated by the polynomial g(X)

= 1 + x 4 + x 6 + x7 + x 8 .

Chapter 8

276

Majority-Logic Decodable and Finite Geometry Codes

The parity-check matrix of this code (in systematic form) is found as follows: 1 0 0 0 0 0 0 0

ho h1 h2 h3 h4 hs h6 h7

H=

0 1 0 0 0 0 0 0

0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0

0 0 0 0 0 0 1 0

0 0 0 0 0 0 0 1

1 0 0 0 1 0 1 1

1 1 0 0 1 1 1 0

0 1 1 0 1 1 0 1 0 1 1 0 1 0 1 0

0 1 0 1 1 1 0 0

0 0 1 0 1 1 1 0

0 0 0 1 0 1 1 1

Consider the following linear combinations of the rows of H: Digit positions: h.3 w1= h1 + h.5 w2 = W3 = ho + h2 + h.6 h7 W4 =

= = = =

0 (0 (0 (1 (0

1 0 1 0 0

2 3 0 1 0 0 1 0 0 0

4 5 0 0 0 1 0 0 0 0

6 7 8 0 0 0 0 0 0 1 0 0 0 1 1

9 0 0 0 0

10

0 0 0 1

11 1 0 0 0

12 1 0 0 0

13 0 1 0 0

14 1), 1), 1), 1).

We see that all four vectors have a 1 at digit position 14 (or X14 ), and at any other digit position, no more than one vector has a 1. Therefore, these four vectors are orthogonal on the digit position 14. Let r be the received vector. The four parity-check sums formed from these orthogonal vectors are related to the error digits as follows: A1 A2 A3 A4

= = = =

w1 w2 W3 W4

·r ·r ·r ·r

= = = =

e3 e1 eo

+en+ e12 +es

+e2

+e6 e7 +es+ ew

+e14 +e13 + e14 +e14 +e14.

We see that e14 is checked by all four check-sums, and no other error digit is checked by more than one check-sum. If e14 = 1, and if there is one or no error occurring among the other 14 digit positions, then at least three (majority) of the four sums, A1, A2, A3, and A4, are equal to e14 = 1. If q4 = 0 and if there are two or fewer errors occurring among the other 14 digit positions, then at least two of the four check-sums are equal to q4 = 0. Hence, if there are two or fewer errors in e, the one-step majority-logic decoding always results in correct decoding of e14. Because the code is cyclic, four parity-check sums orthogonal on any error digit can be formed. It can be checked that four is the maximum number of parity-check sums orthogonal on any error digit that can be formed. Thus, by one-step majority-logic decoding, the code is capable of correcting any error pattern with two or fewer errors. It can be shown that there exists at least one error pattern with three errors that cannot be corrected. Consider an error pattern e with three errors, eo, e3, and es (i.e., eo = e3 = es = 1). From the four parity-check sums orthogonal on e14, we have A1 = 1, A2 = 0, A3 = 1, and A4 = 1. Because the majority of the four sums is 1, according to the decoding rule, e14 is decoded as 1. This results in an incorrect decoding. The code given in this example is actually a BCH code with a minimum distance of exactly 5. Therefore, it is completely orthogonalizable.

I ll

Section 8.1

One-Step Majority-Logic Decoding

2717/

Given an (n. k) cyclic code C for which J parity-check sums orthogonal on an error digit can be formed, the one-step majority-logic decoding of the code can easily be implemented. from the null space Cc1 of the we determine a set of J vectors w1, w2, · · · , w 1 that are orthogonal on the highest-order digit x 11 - 1 . Then, J parity-check sums Ai, A2, · · ·, A1 orthogonal on the error digit e11 _ 1 are formed from these J orthogonal vectors and the received vector 1r. From (8.5), we see that the vector wJ tells what received digits should be summed up to from the check-sum A 1 . The J check-sums can be formed using J multi-input modulo-2 are used as to a I-input adders. Once these J check-sums are majority-logic gate. The output of a majority-logic gate is 1 if and if more than half its inputs are 1; otherwise, the output is 0. The output is the estimated value of e11 _1. A general one-step majority-logic decoder is shown in Figure 8.1. This decoder is called the type-H one-step decoder [2]. The error correction procedure is as follows: With gate 1 turned on and gate 2 turned off, the received vector re is read into the buffer register. §tre]lll '.t The J parity-check sums orthogonal on e11 _1 are formed summing the appropriate received digits. §tlejpi :t The J orthogonal check sums are fed into a majority-logic gate. The first received digit r11 ~1 is read out of the buffer and is corrected by the output of the majority-logic gate. §ftie]PJ 4lo At the end of step 3, the buffer register has been shifted one place to the right with gate 2 on. Now, the second received digit is in the rightmost stage of the buffer register and is corrected in exactly the §tlie[p> ]_"

,,--

:,,

!~'.,·*.•.·.·.·.·

j:-

A.1

I-input majority gate

FIGURE 8.1:

General type-Hone-step majority-logic decoder.

278

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes ~!-+-~~~~~~~~~~~---,

~¥M~~~~~~ Input r(X)

Output

4-input majority gate

FIGURE 8.2: Type-II one-step majority-logic decoder for the (15, 7) BCH code.

same manner as was the first received digit. The decoder repeats step 2 and 3. Step 5. The received vector is decoded digit by digit in the same manner until a total of n shifts. If the received vector r contains LJ /2J or fewer errors, the buffer register should contain the transmitted code vector, and the inputs to the majority-logic gate should all be zero at the completion of the decoding operation. If not all the inputs to the majority gate are zero, an uncorrectable error pattern has been detected. The type-II one-step majority-logic decoder for the (15, 7) BCH code considered in Example 8.1 is shown in Figure 8.2. The parity-check sums orthogonal on an error digit also can be formed from the syndrome digits. Let

ho H=

h1 h2

hn-k-1

1 0 0 0 0 1 0 0 0 0 1 0

0 0 0

POO

POl

PO.k-1

PlO P20

Pll P21

P2.k-1

0

1

Pn-k-1.0

Pn-k-1.1

Pn-k-1,k-l

0

0

0

Pl.k- l

be the parity-check matrix for an (n, k) cyclic code C in systematic form. Because the orthogonal vectors w1, w2, .... w1 are vectors in the row space of H, they are linear combinations of rows of lHL Let Wj = (WjO, Wjl, · · · , Wj.11-l)

= aJolhio

+ aJ1lln1 + · · · + aj,11-k-1lln11-k-l·

Section 8.1

Because of the systematic structure of WjO

Let n·

= CljO,

= (ro, r1. · · · . r 11 _1)

where the ith

Wjl

Majority-logic Decoding

21:9

we see that (8.7)

= Cljl• · · · , U}j,n-k-1 = Clj.n-k-1·

the

be the received vector.

of IC is

digit is (8.8)

s; =rr-Iln;

for O :s i < n - k. Now. consider the Aj

sum

= Wj · u

+ (lj1Iln1 + · · · + (/i-11-k-1G1111-k-1) · li' = oion· · Ilno + aj1IC · Iln1 + · · · + aJ,11-k-1ir · Iln11-k-l· = (ojollno

(8.9)

From (8.7), (8.8), and (8.9), we obtain Aj

= WjOSO

+ Wj]SJ + · · · + Wj.11-k-lS11-k-l·

(8.10)

Thus, the check-sum A j is simply a linear sum of the syndrome digits whose coefficients are the first n - k digits of the orthogonal vector Wj. Based on (8.10), we obtain a different implementation of the one-step majority-logic decoding, as shown in Figure 8.3 received vector can be shifted into the register from the right end). This decoder is called the type-I one-step majority-logic decoder [2]. The error correction procedure is as follows: §tlej]l) JL The syndrome is computed as usual by shifting the received polynomial IC(X) into the syndrome register. §tle]ll) Zo The J parity-check sums orthogonal on e11 _1 are formed taking proper sums of the syndrome digits. These J check-sums are fed into a I-input majority-logic gate. §tlie[JJ ?lo The first received digit is read out of the buffer register and is corrected by the output of the majority gate. At the same time the syndrome register is also shifted once (with gate 2 on), and the effect e11 _1 on the syndrome is removed (with gate 3 on). The new contents in the syndrome register form the syndrome of the altered received vector cyclically shifted one place to the right. §tlieJP) 4L The new syndrome formed in step 3 is used to decode the next received digit r11 _2. The decoder repeats steps 2 and 3. The received digit r11 _2 is corrected in exactly the same manner as the first received digit r 11 _1 was corrected. §tl<el]ll §0 The decoder decodes the received vector re digit digit in the same manner until a total of n shifts of the buffer and the syndrome registers. At the completion of the decoding operation, the syndrome register should contain only zeros if the decoder output is a codeword. H the syndrome register

280

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes Feedback connection Gate 2 o

r(X) - - Gatel Input~--~

ei

e

Syndrome register

O[J

·E ~


h
a0 1-<

"O

~

]-input majority gate

VJ

Gate 3

Gate 4

n-stage buffer register

FIGURE 8.3: General type-I one-step majority-logic decoder.

does not contain all zeros at the end of the decoding, an uncorrectable error pattern has been detected. If we are interested in decoding only the received message digits but not the received parity digits, the buffer register needs store only the k received message digits, and it consists of only k stages. In this case, both type-I and type-II decoders require roughly the same amount of complexity. IEXAMPllE 8.2

Consider the (15, 7) BCH code given in Example 8.1. From the vectors w1, w2, w3, and w4 that are orthogonal on the digit position 14, we find that the parity-check sums orthogonal on e14 are equal to the following sums of syndrome digits:

Based on these sums we construct the type-I one-step majority-logic decoder for the (15, 7) BCH code as shown in Figure 8.4. Suppose that the all-zero codeword (0, 0, · · · , 0) is transmitted, and r(X) = X13 + X 14 is received. Clearly, there are two errors at locations X 13 and X 14 . After the entire received polynomial has entered the syndrome register, the syndrome register contains (0 0 1 1 1 0 0 1). The four parity-check sums orthogonal on q4 are

Section 8.1

Majority-Logic Decoding

S3

A1

l

FIGURE 8.4:

S;_

Se,

A

II

A1

31

'1

-

S7

~j V

V

Majority gate

I Gate I-< -

vi

l j 8 I A2

y

L

So

S5

S1

2~1

j

-- --

j

>

1------

15-stage buffer register

Type-I one-step majority-logic decoder for (15, 7) BCH code.

Because the majority of these four sums is 1, the output of the majority-logic gate is 1, which is the value of e14 . Simultaneously, the buffer and syndrome registers are shifted once; the highest-order received digit 1'14 = 1 is then corrected the output of the majority-logic gate, and the new contents in the syndrome register are (0 0 0 10111). The new parity-check sums are now (ll - 1 A1 -,

2 -1 ,

A(ll -

A3 (lJ - 1 -,

A <,lJ 4

= 1.

Again, the output of the majority-logic gate is 1, which is the value of e13. Both the buffer and syndrome registers are shifted once more; the received digit r13 will be corrected, and the syndrome register will contain only zeros. At this point, both errors have been corrected, and the next 13 received digits are error-free. One-step majority-logic decoding is most efficient for codes that are completely orthogonalizable, or for codes with large J compared with dmin - 1. When J is small compared with dmin - 1, one-step majority-logic decoding becomes very inefficient, and much of the error-correcting capability of the code is sacrificed. Given a code one would like to know the maximum number of parity-check sums orthogonal on an error digit that can be formed. This question is answered by Theorem 8.1. It! Let C be an (n, k) cyclic code whose dual code Cc1 has minimum distance 8. Then, the number of parity-check sums orthogonal on an

TJH[lE!FtlEM

282

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

error digit that can be formed, J, is upper bounded by ]<

-

l

lJ

n --

8-l

.

(8.11)

Proof. Suppose that there exist J vectors w1, w2, · · · , w 1 in the dual code of C that are orthogonal on the highest-order digit position, x 11 - 1 . Because each of these J vectors has a weight of at least 8, the total number of l's in these J vectors is at least J 8; however, because of the orthogonal structure of these J vectors, the total number of l's in them cannot exceed J + (n 1). Therefore, we have Jo:::: J + (n -1). This implies that J:::: (n - 1)/(8 - 1). Because J is an integer, we must have J:::: L(n -1)/(8 - l)J. Q.KD. The dual code of the (15, 7) BCH code has a minimum distance of 4. Therefore, the maximum number of parity-check sums orthogonal on an error digit is upper bounded by Ll4/3J = 4. This proves our claim in Example 8.1 that J = 4 is the maximum number of parity-check sums orthogonal on an error digit that can be formed for the (15, 7) BCH code. If it is possible to form J parity-check sums orthogonal on an error digit for a cyclic code, then the code has a minimum distance of at least J + 1 (Massey bound [2]). The proof of this statement is left as a problem. As we pointed out earlier in this section, one-step majority-logic decoding is most effective for cyclic codes that are completely orthogonalizable. Unfortunately, there exist very few good cyclic codes in this category. The double-error-correcting (15, 7) code considered in Example 8.1 is the only known BCH code that is completely orthogonalizable in one step. Several small classes of one-step majoritylogic decodable cyclic codes are presented in the next two sections. Two of the classes are proved to be completely orthogonalizable. 8.2

A CLASS OF ONIE-STEP MAJORITY-LOGIC DECODABLE CODIES

In this section we present a class of one-step majority-logic decodable cyclic codes whose construction is based on a certain symmetry property. Let C be an (n, k) cyclic code generated by g(X), where 11 = 2111 - 1. We may extend each vector v = (vo, v1, · · · , v11 _1) in C by adding an overall parity-check digit, denoted by v00 , to its left. The overall parity-check digit v00 is defined as the modulo-2 sum of all the digits of v (i.e., v00 = vo + v1 + · · · + v11 _1). Adding v00 to v results in the following vector of n + 1 = 2111 components: Ve= (v 00 , VO,

v1, · · · , V11 -J).

The overall parity-check digit is 1 if the weight of v is odd, and it is O if the weight of v is even. The 2k extended vectors form an (n + 1, k) linear code, denoted by Ce, which is called an extension of C. Clearly, the codewords of Ce have even weight. Let cx be a primitive element in the Galois field GF(2 111 ). We may number the components of a vector Ve = (v 00 , vo, v1, · · · , v2,11-2) in Ce by the elements of GF(2111 ) as follows: the component v00 is numbered cx 00 = 0, the component vo is numbered cx = 1, and for 1 :::: i < 2111 - 1, the component Vi is numbered cx;. We

A Class o"f One-Step 1\/lajority-Logic Decodable Codes

Section 8.2

2:133l

call these numbers the location numbers. Let Y denote the location of a component of "'le, Consider a permutation that carries the component of "'le at the location Y to the location Z = aY + b, where a and bare elements from the field of GF(2111 ), and a -1- 0. This permutation is called an affine permutation [14]. of an affine permutation to a vector of 2111 components results in another vector of 2111 components. IEXAMPLIE la.3

Consider the following vector of 16 elements of GF(2 4 ) (using Table 2.8): otoo

ao

Oll

0(2

0/3

(1

1

0

0

1

0

which are numbered with the

Ol5

0/6

0/ 7

0/8

0/9

OllO

0/11

0[12

0/13

0[14

0

1

0

0

1

0

0

1

0

0).

Now, we apply the affine permutation Z = aY

+ a 14

to the components of the preceding vector. The resultant vector is Oloo

aO

al

Ol2

0/3

(X4

0/5

(X6

Ol 7

0/8

0/9

0/10

all

(¥12

0/13

(¥14

(0

0

1

1

0

0

0

0

1

0

1

0

1

0

0

1).

For example, the component at the location Y = a 8 is carried to the location

z = ot . Ol8 + 0/14 = 0/9 + (¥14 = ()/4. An extended cyclic code Ce of length 2111 is said to be invariant under the group of affine permutations if every affine permutation carries every codeword in Ce into another codeword in Ce. In the discussion we state a necessary and sufficient condition for an extended cyclic code of length 2111 to be invariant the affine permutations. Leth be a nonnegative integer less than 2111 • The radix-2 (binary) expansion of h is where 8; = 0 or 1 for O :S i < m. Let h' be another nonnegative integer less than 2111 whose radix-2 expansion is 2 h'=8'+8'2+8'2 +···+8'111-l 2111 0 1 2

1 •

The integer h' is said to be a descendant of h if 8; :S 8; for O :S i < m.

Let m

= 5. The integer 21 has the following radix-2 expansion: 21 = 1 + 0 · 2 + 1 · 22

+ 0 · 23 + 1 . 2 4 .

284

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

The following integers are proper descendants of 21: 20 = 0 + 0 · 2 + 1 · 22 + 0 · 23 + 1 . 24 , 17 = 1 + 0 · 2 + 0 · 22 + 0 · 23 + 1 . 24 , 16 = 0 + 0 · 2 + 0 · 22 + 0 · 23 + 1 . 24, 5 = 1 + 0 · 2 + 1 · 22 + 0 · 23 + 0 . 24 , 4 = 0 + 0 · 2 + 1 · 22 + 0 · 23 + 0 . 24 , 1 = 1 + 0 · 2 + 0 · 22 + 0 · 23 + 0 · 24, 0 = 0 + 0 · 2 + 0 · 22 + 0 · 23 + 0 . 24 .

Let f..(h) denote the set of all nonzero proper descendants of h. The following theorem characterizes a necessary and sufficient condition for the extension Ce of a cyclic code C of length 2117 - 1 to be invariant under the affine group of permutations.

8.2 [4, 5] Let C be a cyclic code of length n = 2111 - 1 generated by g(X). Let Ce be the extended code obtained from C by appending an overall parity-check digit. Let a be a primitive element of the Galois field GF(2111 ). Then, the extended code Ce is invariant under the affine permutations if and only if for every that is a root of the generator polynomial g(X) of C and for every h' in f..(h), ah' is also a root of g(X), and a 0 = 1 is not a root of g(X). Tlll!lEORJEM

a"

The proof of this theorem is omitted here. For a proof, the reader is referred to [4] and [5]. A cyclic code of length 2111 - 1 whose generator polynomial satisfies the conditions given in Theorem 8.2 is said to have the doubly transitive invariant (DTI) property. RM codes and extended primitive BCH codes are invariant under the affine permutations [4, 5]. Given a code Ce oflengthn = 2 111 that is invariant under the affine permutations, the code C obtained by deleting the first digit from each vector of Ce is cyclic. To see this, we apply the permutation Z = aY to a vector (v 00 , vo, v1, · · · , v2111-2) in Ce, This permutation keeps the component v00 at the same location a 00 but cyclically shifts the other 2111 - 1 components one place to the right. The resultant vector is (vex,,

V2'"-2,

vo, v1, ... 'v211,_3),

which is also in Ce. Clearly, if we delete v 00 from each vector of Ce, we obtain a cyclic code of length 2111 1. Now, we are ready to present a class of one-step majority-logic decodable codes whose dual codes have the DTI property. Let J and L be two factors of 2 111 - 1 2111 such that J. L = 2 111 - 1. Clearly, both J and Lare odd. The polynomial X - 1 + 1 can be factored as follows: x

2111 1 -

+ 1 =Cl+ X 1 )(1 +

Let 1r(X) = 1 +

x 1 + x 21 +

x 1 + x 21 +

... + x
... + x(L- 1l 1 .

(8.12)

Section 8.2

A Class of

From Theorem 2.12 we know that the 2111 - 1 nonzero elements of ) form the 2111 - 1 roots of -l + 1. Let a be a element of GF(2 111 ). Because 2111 (aL)1 = a - 1 = 1, the polynomial X 1 + 1 has a 0 = 1, aL, an,.··, aU-llL as all the n (X) has a 17 as a root if and only if h is not a its roots. 111 multiple of and O < h < 2 - 1. No,N, we form a over as foHm,vs: H (X) has a 17 as a is a root of n(X) and for every h' in t:,.(h). o, 17 ' is also a root if and if (1) root of Ji[ (X). Let Let !I/Ji, (X) be the minimal of a'.

(8.13)

divides ;{(X) and is a factor of -l + L Let C' be the cyclic H follows from Theorem 8.2 that C' has the extended code C~ of C' is invariant under the group of affine permutations. Let C be the dual code of C'. C is also Since divides x 2'" - 1 + 1, we have

H is clear that

X2111-1

+1=

G()()

Let k be the degree of H(X). Then, the degree of G(X) is 2"' -1 - k. The of C is ~(X)

= x2'"-1c-1ccx-1),

which is the reciprocal of G(X). Next, we will show that the code C is one-step majority-logic decodable and is capable of correcting tM L = LJ /2J or fewer errors where J = (2m - 1)/ L. First, we need to determine J vectors from C' (the dual of C) that are orthogonal on the digit at location a 2111 - 2 . Because;, (X) is a multiple of H (X) and has degree less than 2m -1, it is a code polynomial in C' generated by H (X). Clearly, the polynomials Xr,(X), X 2 n(X) .... , x 1 - 1Ji[(X) are also code polynomials in C'. From (8.12) we see that, for i -f. j, Xi JiI(X) and Ii!(X) do not have any common term. Let Yo, 1Y1 •. · · , v 1 _ 1 be the J corresponding codewords of Ji[ (X). XJi[ (X) .... , x 1 - 1n (X). The weight of each of these vectors is L. Adding an overall parity .. check digit to each of these vectors, we obtain J vectors illlO, llllJ, · · · , illl;-1 of length 2111 that are codewords in the extension c; of C'. Since L is odd, the overall parity-check digit of each 1m, is 1. Thus, the J vectors llllo. l!.Il1. · · · , ll!l;-1 have the following properties:

JL They all have 1 at location a 00 (the overall parity-check digit position). ;t One and only one vector has a 1 at the location o) for O :::: j < 2111 - 1. Therefore, they form J vectors orthogonal on the digit at location a 00 • Now, we apply the affine permutation Z = aY + a 2"'- 2 to

into J vectors is invariant under the group of affine permutations). Note that the permutation Z = aY + a 2111- 2 carries the com2111 ponent of l!Il; at location a 00 to location a - 2 . Thus, the vectors :zo, 21, · · · , z 1 _ 1 are illlO, l!ll1, · · · , llllJ-1·

This permutation carries

:zo, :z 1 . · · · , :z 1 _ 1, which are also in C~

c;

u.no. @1, · · · • iillJ-1

286

Majority-Logic Decodable and Finite Geometry Codes

Chapter 8

orthogonal on the digit at location a 2 2. Deleting the digit at location a 00 from a:o, :t:1, · · ·, :t:;-1, we obtain J vectors wo, w1, · · ·, w;-1 of length 2111 -1, which are vectors in C' and are orthogonal on the digit at location a 2 2. From these J vectors we can form J parity-check sums orthogonal on the error digit e2m-2, Therefore, the cyclic code C generated by 11/ -

11/ -

g(X)

= x2"'-k-1G(X-1)

is one-step majority-logic decodable and is capable of correcting tML = LJ /2J or fewer errors. For convenience, we call this code a type-0 one-step majority-logic decodable DTI code. !EXAMPLE 8.5

Let m

= 5. We can factor the polynomial X24 - 1 + 1 = x 1s + 1 as x 1s + 1 = (1 + xs)(l +

xs +

X 10 ).

Thus, J = 5, L = 3, andn(X) = l+Xs +x 10 . Leta be a primitive element in GF(2 4) (use Table 2.8) whose minimal polynomial is (/) 1(X) = 1 + X + X 4 . Because a 1s = 1, the polynomial 1 + xs has 1, a 3 , a 6 , a 9 , and a 12 as all its roots. The polynomial n ( X) has a, a 2 , a4, a5, a 7 , a 8 , a 10 • a 11 , a 13 and a 14 as roots. Next, we determine the polynomial H(X). From the conditions on the roots of H(X), we find that H(X) has a, a 2 , a4, as, a 8 , and a 10 as its roots. The roots a, a 2 , a4, and a 8 are conjugates, and they have the same minimal polynomial, 4> 1(X) = 1 + X + X 4 . The roots as and a 10 are conjugates, and they have s(X) = 1 + X + X2 as their minimal polynomial. Hence, H (X) = 1 (X)
= 1 + x 3 + x 4 + xs + x 6 . We can easily check that H(X) divides n(X), and, in fact, n(X) = (1 + X 3 + X4)H(X). Also, H(X) divides x 1s + 1, and x 1s + 1 = (1 + x 3 + x 4 + xs + x8 + X9 )H(X). Thus, G(X) = 1 + x 3 + x 4 + xs + x 8 + x 9 . The polynomial H(X) generates a (15, 9) cyclic code C', which has the DTI property. The polynomials n(X), Xn(X), X2n(X), X3 n(X), and X4n(X) are code polynomials in C'. The dual code of C', C, is generated by g(X)

= x 9 acx- 1) = 1 + x + x 4 + xs + x 6 + x 9 .

Thus, C is a (15, 6) cyclic code. To decode C, we need to determine parity-check sums orthogonal on q4. The vectors corresponding to 11:(X), Xn(X), X2n(X), X3 n(X), and X4n(X) are Locatfollll Numbers

'VO

'¥1 v2 '¥3 '¥4

= = = = =

ao

al

a2

Ol3

a4

as

a6

a'

as

()(,9

Olli)

all

0{12

aB

ot:14

(1 (0 (0 (0 (0

0 1 0 0 0

0 0 1 0 0

0 0 0 1 0

0 0 0 0

1 0 0 0

0 1 0 0

0

0

0 0 0 1 0

0 0 0 0

1

0 0 1 0 0

1 0 0 0 0

0 1 0 0 0

0 0 1 0 0

0 0 0 1 0

0) 0) 0) 0) 1),

1

A Class of

Section 8.2

which are codewords in C'. Adding an overall obtain the vectors:

to these vectors, we

ll,l[J)ll:anii\l[D]m Nlllln1l1llilieir§ l)Zeo

lll!Q =

11Ill = llll2 =

= lll!4 =

1!Il3

Z

(1 (1 (1 (1 (1

1 0 0 0 0

0 1 0 0 0

0 0 0

0 0

1 0 0

1 0 0 0 0

0

0

1

0 0

0

1

0 1

0 0

0

1

0 0

0

0 0 0 0 1

0 0 0 1 0

0

tVIJW

tV1u

1

0

0 0 0 0

1 0 0 0

tVZB

0 0

1 0 0

0 0 0

1

0)

0) 0) 0)

0

extension of C'). Now, we the affine the components of Mo, 11111, w2, 11n3, and 11114. The permutation vectors:

= otY +a 14 to

results in the

fL,l[J)cciiilifoilll N1111mmlilie1r§ iOl'll

1

7lQ =

(0 7l2 = (0 7l3 = (1 7l4 = (0

7l1 =

1 0 0

1 0 0 0

0 1 0 0

0

0 0 0

0 0 0 0

1 0

Deleting the overall vectors in C':


1 0 0 0 0

tVIU

0 0 0

0 1

0 0 0 0

1

0 1 0 0 0

1) 1) 1) 1) 1),

digits from these

we

]Ll[J)ll:anifoilll NTITITIIlTill»em

(0 (0 '1"12 = (1 'Y)'3 = (0 W4 = (0 wo =

W1 =

0 1 0 0 0

0

0

0 1

0

0 0

0 0 0

0 0

1

1

0

0 1 0 0 0

l)Zi(j

1)17

0

1 0 0

1 0 0

0 0

0

0 1 0 0

0

0 0

0 1 0

iOtlll

1)£11

1 0 0 0 0

0

0

0 0 0

0 0 0

1

1

1)113

1)114!

0 1

1) 1) 1) 1)

0 0 0

1)

We see that these vectors are orthogonal on the at location a 14 . Let ir = (ro, ri, r2, r3, r4, rs, r6, r7, rs, r9, r10, r11, r12, r13, q4) be the received vector. Then, the sums orthogonal on e14 are

+ rs + r10 + r14, + rs + r13 + r14. ro + r2 + r6 + r14. r4 + r9 + r14, r3 + r11 + r12 + r14.

Ao = ir · wo = r7 A1 = ir · w1 = r1

A3

= re · w2 =Jr· W3

A4

=

A2

= = rr · W4 =

Therefore, the (15, 6) cyclic code C generated by g(X) = 1 + X + x4 + xs + x6 + x9 is one-step majority-logic decodable and is capable of correcting fML = L5/2J = 2 or fewer errors. H also corrects many error patterns of three errors. The code has a minimum distance of at least J + 1 = 5 + 1 = 6; however, the minimum distance of the code is exactly 6. Hence, the code is completely orthogonalizable.

288

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

Recall that .n:(X) has a 11 as a root if and only if h is not a multiple of L, and O < h < 2111 - 1. Therefore, 11:(X) has the following consecutive powers of a as roots: a, a 2 , · · · , aL-l. Because any descendant h' of an integer h is less than h, if h < Land h' in t,.(h), both c.Ji and a"' are roots of n(X). Consequently, the polynomial H(X) also has a, a 2 , · · · , aL-l as roots. Using the argument that proves the minimum distance of a BCH code, we can show that the minimum distance of C' generated by H (X) is at least L; however, since n (X) is a code polynomial of weight Lin C', the minimum distance of C' is exactly L. It follows from Theorem 8.1 that the number of parity-check sums orthogonal on an error digit that can be formed for C is upper bounded by

2m - 2J.

l

L - l

(8.15)

'

however, J = (2111 - 1)/L. Therefore, for large L, J is either equal to or close to the upper bound of (8.15). In general, it is not known whether the type-0 DTI codes are completely orthogonalizable. There are a number of special cases for which we can prove that the codes are completely orthogonalizable. The type-0 DTI codes may be modified so that the resultant codes are also one-step majority-logic decodable, and J - 1 parity-check sums orthogonal on an error digit can be formed. Recall that the polynomial H (X) does not have (X + 1) as a factor (i.e., it does not have a 0 = 1 as a root). Let H1 (X)

=

(X

+ l)H(X).

(8.16)

The cyclic code Ci generated by H1 (X) is a subcode of C' generated by H (X). In fact, Ci consists of the even-weight vectors of C' as codewords. Recall that the J orthogonal vectors wo, w1, · · ·, w 1 _1 in C' are obtained from the vectors zo, z1, · · · , z 1 _1 by deleting the digit at location a 00 . Because zo, z1, · · · , ZJ-l are orthogonal on the digit at location a 2'" - 2 , there is one and only one vector z; that has a 1 at location a 00 . Since zo, z 1 , · · · , z 1 _ 1 all have weight L + 1 which is even, all but one of the orthogonal vectors wo, w1, · · · , w 1 _ 1 have weight L + 1. These J - 1 Therefore, the dual code of denoted even-weight orthogonal vectors are in by Ci, is one-step majority-logic decodable, and J - 1 parity-check sums orthogonal on an error digit can be formed. Let

Ci.

G 1 (X)

Ci,

=

G(X).

X

(8.17)

+l

Then, the generator polynomial for C1 is

g1

( x)

=

x2"'-1c-2c l

cx-1) =

g(X) X + 1'

(8.18)

where g(X) is given by (8.14). C1 is called a type-l DTI code, and its dimension is one greater than that of its corresponding type-0 DTI code.

Section 8.2

A Class of One-Step Majority-Logic Decodable Codes

Form = 4 and J = 5, the type-1 DTI code C1 that corresponds to the code given in Example 8.5 is generated

2~'fll

DTI

5 6 9 ~1 (X) = - - - - --+-)_{_ _ __

+x +x +x

1+x+

1

= 1 + x 4 + x 6 + x7 + x 8. His interesting to note that this code is the (15. 7) BCH code. From Example 8.5 we see that w3 has odd weight. and therefore it is not a vector in C~ (the dual of Ci). Hence, the four orthogonal vectors in q are

wo W1

w2 W4

= (0 = (0 = (1 = (0

0 1 0 0

0 0

1 0

0 0 0 1

0 0 0 0

0 1 0 0

1

0 0 1 0

0 0 0

1 0 0 0

0 0 0 0

1 0 0 0

0 0 0 1

0 0 0 1

0 1 0 0

1). 1). 1), 1).

which are the same four orthogonal vectors given in Example 8.1. Because the dual code of type-1 DTI code C1 has a minimum distance of L + 1. the number of parity-check sums orthogonal on an error digit that can be formed is upper bounded

l

J-- l2 -1- 1 J1 J _J-1. _ - l] - L L L 111

2"' -2 L

-

Therefore, the number of parity-check sums orthogonal on an error digit that can be formed for a type-1 DTI code is equal to its upper bound. Since J is odd, LJ /2J = L(J - 1) /2J. Thus, both type-0 and type-1 DTI codes have the same majority-logic error-correcting capability. In general, there is no simple formula for enumerating the number of paritycheck digits of the one-step majority-logic decodable DTI codes (type-0 or type-1): however, for two special cases, exact formulas for n - k can be obtained [6]: Case 1. Form = 2sl and J = l1 + 1. the number of parity-check digits of the type-1 DTI code of length 2111 - 1 is n - k = c2s+l - 1i - 1.

Case II. Form = )I./ and J = l1 - 1, the number of parity-check digits of the type-1 DTI code of length 2111 - 1 is 11 -

k

= 2111

-

(2A - 1) 1 - 1.

A list of one-step majority-logic decodable type-1 DTI codes is given in Table 8.1. Short-length DTI codes are comparable with BCH codes in efficiency. For example, there exists a (63, 37) one-step majority-logic decodable type-1 DTI code that is capable of correcting four or fewer errors. The corresponding four-errorcorrecting BCH code of the same length is a (63. 39) code that has two information

290

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

Some one-step majoritylogic decodable type-1 DTI codes.

TABLE 8.1:

n

k

15

9 7 49 37 13 225 207 175 37 21 343 139 961 833 781 151 30

63

255

511 1023

tML

1 2 1 4 10 1 2 8 25 42 3 36 1 5 16 46 170

n

k

2047

1211 573 3969 3871 3753 3611 3367 2707 2262 2074 1649 1393 1377 406 101 43

4095

iML

11 44 1 2 4

6 32 17 19 22 45 52 136 292 409 682

digits more than the (63, 37) type-1 DTI code; however, the decoding circuit for the (63, 39) BCH code is much more complex than for the (63, 37) DTI code. For large block lengths, the DTI codes are much less efficient than the BCH codes of the same length and the same error-correcting capability. 8.3

OTHER ONE-STIEF' MAJORITY-LOGIC DIE(ODABU: (ODIES

There are two other small classes of one-step majority-logic decodable cyclic codes: the maximum-length codes and the difference-set codes. Both classes have been proved to be completely orthogonalizable. 8.3.1

Maxoml!.lm-length Codes

For any integer m ::: 3, there exists a nontrivial maximum-length code with the following parameters: Block length: Number of information digits: Minimum distance:

n = 2111 k=m, d

-

l,

= 2111-1.

The generator polynomial of this code is g(X)

=

X 11 +1 p(X) '

(8.19)

where p(X) is a primitive polynomial of degree m. This code consists of the all-zero codeword and 2111 -1 codewords of weight 2111 - 1 (see Problem 8.11). Maximum-length

Section 8.3

Other One-Step

Decodable Codes

2£1

codes were first shown to be majority-logic decodable by Yale [7] and Zierler [8] independently. The dual code of the maximum-length code is a - 1, 2111 - m - 1) cydic code by the of the Jlll(X), (X)

=X

111

j;l])(X- 1 ).

Because Jlll*(X) is also a polynomial of degree m, the dual code is thus a Hamming code. Therefore, the null space of the ,u,uuuuuH.-, ...,ul",'H vectors of 3 is the minimum Now, consider the

Q = {w(X) =

+

+

xn-l :

0 ::: i < j < n -- 1)

(8.20)

in the Hamming code (X). No two in Q can have any common terms except the term the sum of these two would be a code of two terms in the Hamming code. ~fhis is impossible, since the minimum weight of a Hamming code is 3. Therefore, the set Q contains polynomials orthogonal on the x11 - 1 . To find we start with a polynomial xn-l + for O ::: j < 11 - 1 and then determine such that x 11 - 1 + x.i + is divisible by Jlll*(X), as follows: Divide x 11 - 1 + x.i (X) step-by-step with long division until a single term Xi appears at the end of a certain step. w(X) = xn-l + + Xi is a polynomial orthogonal on digit position x 11 - 1 . Clearly, if we started with xn-l + , we would obtain the same polynomial w(X). Thus, we can find (n - 1)/2 = 2111 - 1 - 1 polynomials orthogonal on digit position . That is, J = 2m·- 1 - 1 parity-check sums orthogonal on e11 _ 1 can be formed. Because the maximum-length code generated by [l;(X) of (8.19) has a minimum distance of exactly 2m-l, it is orthogonalizable. The code is capable of correcting tML = 2111 - 2 - 1 or fewer errors with one-step majority-logic decoding.

Consider the maximum-length code with m = 4 and polynomial JID(X) = + X + X 4 . This code has block length n = 15 and minimum distance cl = 8. The generator polynomial of this code is 1

xis+ 1 g(X)

=

JID(X)

+xs+x7+xs+x11_

+

=l+X+

His a (15, 4) code. The null space of this code is generated by 1

Jlll*(X) = X 4 ]lll(X- ) =

We divide X 14

+ X 13

+ X 3 + 1.

+ X 3 + 1 with long division as follows:

JID*(X) =

xtO

x4 + x3 +

1

\x14 + xB x14 + xB + xrn x 10 (stop).

292

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

A single term, X 10 , appears at the end of the first step of the long division. Then, w1 (X) = X 14 + X 13 + x 10 is a polynomial orthogonal on X 14 . Now, we divide xi4 + x12 by p*CX): xrn + x9 + x6 x4

+

x3

+

+

1 1x14 x14

+

X13 xB X13

+ +

x12

x12 x12

+ +

xrn xlO

xlO xrn

+ + +

x9 x9 x9

+

x6 x6 (stop).

Then, w2(X) = X14 + X 12 + X 6 is another polynomial orthogonal on x 14 . The rest of the polynomials orthogonal on X 14 can be found in the same manner; they are w3(X)

= 1 + x 11 + x 14,

+ + X 4, xz + x3 + xi4.

ws(X) = X w1(X) =

1

X8

w4(X) w6 (X)

= x 4 + x 9 + x 14, = x 5 + x 7 + x 14,

From the set of polynomials orthogonal on X 14, we obtain the following seven parity-check sums orthogonal on e14: A1 = e10

A2 A3 A4 As A6 A7

= e6 = eo = e4 = e1 = es = e2

+ e13 + e12 + e11 + e9 + es + e7 + e3

+ e14, + e14, + e14, + e14, + e14, + e14, + e14.

In terms of syndrome bits, we have A1 = s10, A2 = S6, A3 = so, A4 = s4 + s9, As = + ss, A6 = ss + s7, and A7 = s2 + s3. The code is capable of correcting three or fewer errors by one-step majority-logic decoding.

s1

8.3.2

Difference-Set Ciodles

The formulation of difference-set codes is based on the construction of a perfect difference set. Let P = {lo, l 1, l2, · · · , lq} be a set of q + 1 nonnegative integers such that O::::: lo< 11 < '2 < · · · < lq ::::: q(q + 1). From this set of integers it is possible to form q (q D

=

+ 1) ordered differences as follows:

{/J - I; : j -::fa i).

Obviously, half the differences in D are positive, and the other half are negative. The set P is said to be a perfect simple differe1Zce set of order q if and only if it has the following properties:

Section 8.3

Other One-Step Majority-logic Decodable Codes

]_, AH the differences in D are distinct. AH the differences in D are distinct. 2lo If lj - !; is a negative difference in then q (q to any difference in D.

2!9l3

1,,

+ 1) + 1 + (Ii

- l;) is not equal

it follows from the definition that P' = {O, 11 - lo. !2 - lo. · · · . lq - lo} is also a perfect simple difference set.

Consider the set P = {O. 2, 7, 8, 11} with q = 4. The 4 · 5 = 20 ordered differences are D

= {2, 7, 8, 11, 5, 6. 9, 1, 4. 3, -2. -7, -8, -11, -5. -6. -9, -1, -4. -3}.

lt can be checked difference set

that P satisfies all three properties of a perfect

[9] has constructed perfect difference sets for order q = ps, where p is a prime ands is any integer (see also [10]). lln the discussion we shall be concerned with q = 2'. Let P = {lo = 0, 11. !2, · · · , /i,} be a difference set of order 2'. We define the 7l(X) Let n = 2' (2' that is,

= 1+

+

+···+

(8.21)

+ 1) + 1 and Ilu(X) be the greatest common divisor of 7l(X) and X" + 1; lln(X)

= GCD{7l(X). = 1 + h1X

X"

+ 1}

+ h2X 2 + · · · + h1c-1

(8.22)

+

Then a difference-set code of length 11 is defined as the cyclic code generated ~(X)

= =

This code has the

X" +1 lln(X)

1 + g1X

(8.23)

+ g2X2 + ... + xn-k.

parameters: Code length:

11

= 225 + 25 + 1

Number of parity-check digits: n - k Minimum distance: d

= 3' + 1

= 2s + 2.

Difference-set codes were discovered Rudolph [11] and Weldon [12] independently. The formula for the number of parity-check digits was derived Graham and Mac Williams [13].

294

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

!EXAMPLE 8.9

In Example 8.8 we showed that the set P = {O, 2, 7, 8, 11} is a perfect simple difference set of order q = 22. Let z(X) = 1 + x 2 + X 7 + x 8 + x 11 . Then, h(X)

= GCD{l + x 2 + x 7 + x 8 + x 11 , 1 + x 21 } = 1 + x 2 + x 7 + x 8 + x 11 .

The generator polynomial of the difference-set code of length n = 21 is g(X)

=

x21 + 1 h.(X)

= 1 + x2 + x4 + x6 + x1 + xio_ Thus, the code is a (21, 11) cyclic code. Let Jhl* (X) = X"h(X- 1) be the reciprocal polynomial of lh!(X). Then, the (n, n - k) cyclic code generated by h.*(X) is the null space of the difference-set code generated by g(X) of (8.23). Let z*(X)

= X 12 sz(X- 1) (8.24)

Because z(X) is divisible by h(X), z*(X) is divisible by h*(X). Thus, ,z*(X) is in the null space of the difference-set code generated by g(X) of (8.23). Let wo(X)

= x11-l-l2Sz*(X) = xn-l-l2s + ... + xn-l-/2 + xn-l-/1 + xn-1_

Obviously, wo(X) is divisible by h* (X) and is also in the null space of the difference-set code generated by g(X) of (8.23). Now, let w;(X) = xi;-1;-1-1

+ xi;-1,-2-1 + ... + xi;-11-1 + x1;-1 (8.25)

be the vector obtained by shifting wo(X) cyclically to the right l; times. Because {lo = 0, /1, l2, · · · , !2,} is a perfect difference set, no two polynomials w; (X) and Wj (X) for i I- j can have any common term except X"- 1. Thus, w 0 (X), w 1 (X) .. · · . w2, (X) form a set of J = 2s + 1 polynomials orthogonal on the digit at position X"- 1 . Since the code generated by g(X) of (8.23) is proved to have a minimum distance of 2" + 2, it is completely orthogonalizable and is capable of correcting tM L = 2 1 - 1 or fewer errors. !EXAMPLE 8.10

Consider the code given in Example 8.9, which is specified by the perfect difference set P = {O, 2, 7, 8, 11} of order 22 . Thus, we have z*(X)

= x 11 z(X- 1) = 1 + x 3 + x 4 + x 9 + x 11

Section 8.3

Other One-Step Majority-Logic Decodable Codes

295

Input~-~

- - > Gatel

FIGURE 8.5: Type-H majority-logic decoder for the (21, 11) difference-set code.

and wo(X)

= x9;z*(X) = x9 +

x12 + xB +xis+ x20.

By shifting wo(X) cyclically to the right 2 times, 7 times, 8 times, and 11 times, we obtain wi(X) = x + xn w2 cx) = x4 + xG w3(X) = 1 + xs w4(X) = x 2 + x 3

+ + + +

x14 x16 x7 x8

+xis+ + x19 + + xi7 + + x 10 +

x20, x20. x20, x 20 .

Clearly, wo(X), w1(X), w2(X), w3(X), and W¥4(X) are five polynomials orthogonal on x20 . From these five orthogonal polynomials, we can form the following five parity-check sums orthogonal on e20:

=

A1 s9 A2 = s1 A3 = s4 A4 = so As = s2

=

e9 = e1 + S6 = e4 + ss + s7 = eo + s3 + ss = e2

+ + + + +

e12 en e6 es e3

+ + + + +

+ e1s + e14 + eis + e16 + q9 + q + e17 + es + e10 + e13

e20, e20, e20, e20, e20.

A type-II majority-logic decoder for this code is shown in Figure 8.5. The construction of a type-I decoder for this code is left as an exercise. Difference-set codes are nearly as powerful as the best known cyclic codes in the range of practical interest. Unfortunately, there are relatively few codes with useful parameters in this class. A list of the first few codes with their generator

296

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

TABLE 8.2:

s

n

1 2 3

A list of binary difference-set cyclic codes.

k

d

t

Gel!Jleirafoir polynomial, g(X)*

7 21 73

3 11 45

4 6 10

1 2 4

0,2,3,4 0,2,4,6, 7,10 0,2,4,6,8,12, 16,22,25,28

4

273

191

18

8

0,4, 10, 18,22,24,34,36,40,48, 52,56,66,67, 71, 76, 77,82

5

1057

813

34

16

0, l,3,4,5,11, 14, 17, 18,22,23, 26,27,28,32,33,35,37,39,41, 43,45,47,48,51,52,55,59,62, 68, 70, 71, 72, 74, 75, 76,79,81, 83,88,95,98,101,103, 105, 106,108,111,114,115,116,120, 121,122,123,124,126,129,131, 132,135,137,138,141,142,146, 147,149,150,151,153,154,155, 158,160,161,164,165,166,167, 169,174,175,176,177,178,179, 180,181,182,183,184,186,188, 189,191,193,194,195,198,199, 200,201,202,203,208,209,210, 211,212,214,216,222,224,226, 228,232,234,236,242,244

Associated diffeirem::e set

0,2,3 0,2, 7,8,11 0,2, 10,24,25,29, 36,42,45 0,18,24,46,50,67, 103, 112, 115, 126, 128,159,166,167, 186,196,201 0, 1,3, 7, 15,31,54, 63, 109, 127, 138, 219,255,277,298, 338,348,439,452, 511,528,555,597, 677,697,702,792, 897,905,924,990, 1023

*Each generator polynomial is represented by the exponents of its nonzero terms. For example, {O, 2, 3, 4) represents g(X) = 1 + x2 + x 3+ x4.

polynomials and their corresponding perfect simple difference sets is given in Table 8.2. Other one-step majority-logic decodable cyclic codes will be presented in Section 8.5. 8.4

MUlTi?llE-STIEf' MAJJO~!TY-lOGK [))IE(OID!NG

The one-step majority-logic decoding for a cyclic code is based on the condition that a set of J parity-check sums orthogonal on a single error digit can be formed. This decoding method is effective for codes that are completely orthogonalizable or for codes with large J compared with their minimum distance dmin· Unfortunately, only several small classes of cyclic codes are known to be in this category; however, the concept of parity-check sums orthogonal on a single error digit can be generalized in such a way that many cyclic codes can be decoded by employing several levels of majority-logic gates.

Section 8.4

Multiple-Step Majority-Logic

Let E = {e; 1 , e; 2 , · · · , e;M} be a set of 11/i error digits, where O :': i1 < i2 < .. · < i M < n. The integer 11/i is called the size of E.

ttz A set of J parity-check sums A1, A2, · · · , A1 is said to be orthogonal on the set E if and if (1) every error digit e;, in Eis checked every check-sum A 1 for 1 :': j :': J, and (2) no other error is checked more than one check-sum.

IDJEJFKNTI1I'WN

For E

=

v~<-HU ..n v

the following four

sums are

{e6, eg}:

A1

A2 A3 A4

= eo = = =

+e2

+e6 +e6 +e6 + +e6

e3 + e4 ei

es

e7

+es, +es, +es, +es.

Following the same argument employed for one-step majority-logic decoding, we can correctly determine the sum of error digits in E, e; 1 + e; 2 + · · · + e;M from the check-sums A1, · · · , A1 orthogonal on E that there are LJ /2J or fewer errors in the error pattern e. This sum of error digits in E may be as an additional check-sum and so can be used for decoding. Consider an (n. k) code C that is used for error control in a communication (or storage) system. Let re= (eo, ei. · · · , e11 _1) denote the error vector that occurs during the transmission of a codeword win C. Let Ef. EJ. · · ·, be some properly selected sets of error digits of e. Let S(E/) denote the modulo-2 sum of the error digits in Suppose that for each set it is possible to form at least J parity-check sums orthogonal on it. Then, the sum S(E,1) can be estimated from these J orthogonal check-sums. The estimation can be done by a majority-logic gate with the J orthogonal check-sums as The estimated value of S(E,1) is the of a majority-logic gate, which is 1 if and only if more than half of the are 1; otherwise, it is 0. The estimation is correct provided that there are LJ /2J or fewer errors in the error vector e. The sums S(Ef), S(EJ), · · · , S(E,1), · .. together with other check-sums) are then used to estimate the sums of error in the second selected sets, whose size is smaller than that of the first selected sets. Suppose that for each set it is possible to form J or more check-sums orthogonal on it. Then, the sum S(E;) can be determined correctly from the check-sums orthogonal on provided that there are no more than LJ /2J errors in e. Once the sums, S(Ef), S(Ei), · · · , S(E;), · · ·, are determined, they (maybe together with other check-sums) are used to estimate the sums of error digits in the third selected sets, Ef, E~. · · · , Ef. · · · , whose size is smaller than that of the second selected sets. The process of estimating check-sums from known check-sums is called orthogonalization [2]. The orthogonalization process continues until a set of J or more check-sums orthogonal on only a single error digit, say e11 _1, is obtained. Then, the value of e11 _1 can be estimated from these orthogonal check-sums. Because of the cyclic structure of the code, other error digits can be estimated in the same manner and by the same circuitry. A code is said to be L-step orthogonalizable ( or L-step majority-logic decodable) if L steps of orthogonalization are required to

Ef, ··.

Ef.

E7

Ef. Ei, ··· . E;, ···,

E;

E;

298

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

make a decoding decision on an error digit. The decoding process is called L-step majority-logic decoding. A code is said to be completely L-step orthogonalizable if J is 1 less than the minimum distance of the code (i.e., J = dmin - 1). Because majority-logic gates are used to estimate selected sums of error digits at each step of orthogonalization, a total of L levels of majority-logic gates are required for decoding. The number of gates required at each level depends on the structure of the code. The following two examples are used to illustrate the notions of multiple-step majority-logic decoding. EXAMPLE 8.11

Consider the (7, 4) cyclic code generated by g(X) = 1 + X + X 3 . This is a Hamming code. The parity-check matrix (in systematic form) is found as follows: H =

ho] h1

[ h2

=

[1001011] 0 1 0 1 1 1 0 . 0 0 1 0 1 1 1

We see that the vectors ho and h2 are orthogonal on digit positions 5 and 6 (or xs and X6 ). We also see that the vectors lhio + Jru 1 and lhl2 are orthogonal on digit = {es, e6} and El = {e4, e6} be two selected sets. Let positions 4 and 6. Let r = (ro, r1, r2, r3, r4, rs, r6) be the received vector. Then, the parity-check sums formed from ho and h2 are

Ei

A1 A2

= r - lll.o = eo = r · lll.2 =

+e3 e2

and the parity-check sums formed from ho B1 = r . (ho B2 = r · 1h12

+e4

+es+ e6 +es + e6

+ h1 and h2 are

+ lll.1) = eo + e1 e2

+e4 +e4

+es

+e6 +e6.

Ei

The parity-check sums A1 and A2 are orthogonal on the set = {es, e6}, and the parity-check sums B1 and B2 are orthogonal on the set El = {e4, e6}. Therefore, the sum S(Ei) =es+ e6 can be estimated from A1 and A2, and the sum S(El) = e4 + e6 can be estimated from B1 and B2. The sums S(Ei) and S(El) will be correctly estimated provided that there is no more than one error in the error vector e. Now, = {e6}. We see that S(Ei) and S(El) are orthogonal on e6. Hence, e6 can be let estimated from S(Ei) and S(El). The value of e6 will be estimated correctly provided that there is no more than one error in e. Therefore, the (7, 4) Hamming code can be decoded with two steps of orthogonalization, and it is two-step majority-logic decodable. Because its minimum distance is 3 and J = 2, it is two-step completely orthogonalizable. A type-II decoder for this code is shown in Figure 8.6. Lets = (so, s1, s2) = r · H 7 be the syndrome of the received vector r. Then, we can form the parity-check sums A1, A2, B1, and B2 from the syndrome digits as follows:

Ef

A1 = so, B1 = so +s1,

Section 8.4

Multiple-Step

Output

M

FIGURE 8.6: Type-H two-step majority-logic decoder for the (7, 4) Hamming code. we may construct a type- I majority-logic decoder for

Based on these the (7. 4) Hamming code.

IE:l{AM!fllLIE 2U:?2

Consider the triple-error-correcting (15, 5) BCH code whose generator polynomial is ~(X) = 1

The parity-check matrix Ilno Iln1 Iln2 1h13

lHI=

1h14 1h15

illl6 1h17

lhls 1h19

+x 4 +x 5 +x 8 +x 10 •

+X +

systematic form) is

1 0 0 0 0 0 0 0 0 0 0 0

1 0 0 0 0 0 0 0

0

0

1 0 0 0 0

0 0 0

0 0 0

1 0 0 0 0 0 0

0 0 0 0

1 0 0 0 0 0

0 0 0 0 0

0 0 0 0 0 1 0 0 1 0 0 0 0 0 0

0 0 0 0 0 0 0

1

0 0 0 0 0 0 0 0

0 0

0

1

0 0 0 0 0 0 0 0 0 1

1 0 1 1 1 1 0 1 1 1 0 0 1 0

1 0

1 1 0 1 1 0 0 0 1 1 1 0 1 1 1 0 1 1 0 1 1 1 0 1

1 1 0

1 1 0

0 1 0 1

300

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

Let

Ef = Ej

{e13, e14},

= {en, e14},

EJ =

{es, e14},

EJ = El = EJ =

{e12, e14}, {e10, e14}, {e2, e14}

be six selected sets of error digits. For each of the preceding sets it is possible to find six parity-check sums orthogonal on it. Let r = (ro, r1, r2, r3, r4, rs, r6, r7, rs, r9, r10, rn, r12, r13, r14) be the received vector. By taking proper combinations of the rows lHI, we find the following parity-check sums orthogonal on and EJ:

Ef, EJ, Ej, El, EJ,

lo Check-sums orthogonal on

Ef =

{e13, e14}:

A11=r-h4 =e4+e1o+e13+e14 A12 =Jr· h7 = e7 + e12 + e13 + e14 A13=1r-h9 =e9+e11+e13+e14 A14 = r ·(ho+ hs) = eo + es + e13 + e14 A1s = r · (h1 + hs) = e1 + es + e13 + e14 A16=ll'·(h3+h6)=e3+e6 +e13+e14.

2. Check-sums orthogonal on A21 A22 A23 A24 A2s A26

= = = = = =

r r r r r r

Ei = {e12, e14}:

· ho = · h3 = · h7 = · (h1 + h2) = · (h4 + hs) = · (h6 + h9) =

3. Check-sums orthogonal on

Ej

eo e3 e7 e1 e4 e6

+ + + + + +

e10 en e13 e2 es e9

A41 A42 A43 A44 A4s A46

El =

= r · ho =Jr· h4 = r · (h1 = r · (h3 = r · (h7 = r · (h2

e12 e12 e12 e12 e12 e12

+ + + + + +

e14 e14 e14 €14 e14 e14.

= {en, e14}:

A31 = r · h.3 = e3 + e12 A32 = r · h9 = e9 + e13 A33 = r · (ho + hs) = eo + es A34 = r · (h1 + hs) = e1 + es A3s = r · (h.2 + h4) = e2 + e4 A36=r-(h6+h7)=e6+€7

4. Check-sums orthogonal on

+ + + + + +

+ + + +

+ en + e14 + en + e14 + en + e14 + en + e14 + en + €14 +e11+e14.

{e10, e14}:

b.6) hs) hs) h9)

= = = = = =

eo e4 e1 e3 e7 e2

+ + + + + +

e12 e13 e6 es es e9

+ + + + + +

e10 e10 e10 e10 e10 e10

+ + + + + +

e14 e14 e14 e14 €14 e14.

Section 8.4 §0

Check-sums orthogonal on EJ

Multiple-Step Majority-Logic

=

{es, e14}:

As1 = rr · (Ilno + Ilns) = eo + en As2 = rr · (Iln 1 + Ilns) = ei + e13 A53 = rr · (Iln3 + Ilns) = e3 + ew = rr · (Iln4 + Ilus + Iln6) = e4 + e6 Ass=li'·(Iln2+Ilns+Iln1)=e2+e7 As6 = r · (J:ns + Ilns + Iln9) = es + e9

es + e14 es + e14 es + e14 es + e14 +es+e14 + es + e14. + + + +

60 Check-sums A61 = A62 = A63 = A64 = A6s = A66 =

(Iln1 n· · (Iln2 li' · (Ilno E' • (Iln2 rr · (Iln2 rr · (Iln2

iI ·

+ + + + + +

Iln2) = e1 + Iln4) = e4 + Iln2 + Iln6) = eo + Iln3 + Ilns) = e3 + Ilns + Iln7) = es + Iln9) = e9 +

e12 en e6 es

+ + + + €7 + e10 +

e2 e2 e2 e2 e2 e2

+ + + + + +

e14 e14 e14 e14 e14 e14.

From the foregoing orthogonal check-sums, the sums S(E}) = e13 + e14, S(EJ) = e12 + e14, S(Ej) = en + e14, S(EJ) = e10 + e14, S(EJ) = es + e14, and S(EJ) = e2 + e14 can be correctly estimated that there are no more than three errors in the error vector I!;. Let = {q4}. We see that the error sums S(E}), S(EJ), S(Ej), S(EJ), S(EJ), and S(EJ) are orthogonal on q4. Hence, e14 can be estimated from these sums. Therefore, the (15, 5) BCH code is orthogonalizable. Because J = 6, it is of correcting three or fewer errors with two-step majority-logic decoding. H is known that the code has a minimum distance of 7. Hence, it is two-step completely orthogonalizable. The type-H decoder for the 5) BCH code is shown in Figure 8.7, where majority-logic gates in a tree form) are used. Construction of a majority-logic decoder for the 5) BCH code is left as an exercise Problem 8.12).

Ef

A general type-H L-step majority-logic decoder is shown in Figure 8.8. The error correction procedure is as follows: §tlel[Jl ]_, The received vector rr(X) is read into the buffer register. §tlerpi Zo Parity-check sums (no more than of them) orthogonal on certain

Ul

selected sets of error digits are formed by appropriate sets of received digits. These check-sums are then fed into the first-level majority-logic gates (there are at most (J)L-l of them). The outputs of the first-level majority-logic gates are used to form inputs to the second-level majority-logic gates (there are at most (J)L- 2 of them). The outputs of the second-level majority-logic gates are then used to form inputs to third-level majority-logic gates (there are at most (Jl- 3 of them). This process continues until the last level is reached; there is only one gate at the last level. The J inputs to this

302

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

FIGURE 8.7: Type II two-step majority-logic decoder for the (15, 5) BCH code.

Gate 2

r(X) --

n-stage buffer register

Gatel

1st level majority gates

2nd level

Lth level

FIGURE 8.8: General type-II L-step majority-logic decoder.

Section 8.4

gate are check-sums on the highest-order error digit e11 _1, The output of this gate is used to correct the received digit r 11 _1, §tleJPl 2lo The received r 11 _1 is read out of the buffer and is corrected by the last-level gate. §tleJPl 4), At the end of 3, the buffer register has been shifted one to the right. Now, the second-highest-order received digit r11 _2 is in the rightmost stage of the buffer register, and it will be corrected in r11 _1. exactly the same manner as was the highest-order received The decoder repeats steps 2 and 3. §tleJPl §, The received vector is decoded digit digit in the manner described until a total of n shifts. A general type-I decoder for an L-step majority-logic decoder code is shown in Figure 8.9. Hs decoding operation is identical to that of the type-I decoder for

1st level majority gates

2nd level

Lth level

-->I Input

FIGURE 8.9: General type-IL-step majority-logic decoder.

304

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

a one-step majority-logic decodable code except that L levels of orthogonalization are required. An L-step majority-logic decoder requires L levels of majority-logic gates. At the ith level, no more than ul-i gates are required. Thus, the total number of majority-logic gates needed is upper bounded by 1 + l + 1 2 + ... + 1L-l. In fact, Massey [2] has proved that for an (n, k) L-step majority-logic decodable code no more than k majority-logic gates are ever required. Unfortunately, for a given L-step majority-logic decodable cyclic code, there is no known systematic method for minimizing the number of majority-logic gates except the trial-and-error method. For almost all the known classes of L-step majority-logic decodable codes, the rules for forming orthogonal parity-check sums require a total of 1 + l + 1 2 + ... + 1L-l majority-logic gates. Thus, the complexity is an exponential function of L. For large L, the decoder is likely to be impractical. Fortunately, there are many cyclic codes with useful parameters that can be decoded with a reasonably small L. Several large classes of cyclic codes have been found to be L-step majoritylogic decodable. The construction and the rules for orthogonalization of these codes are based on the properties of finite geometries, which are the subject of the next four sections. 8.5

EUCUDEAN GEOMETRY

Consider all them-tuples (ao, a1, · · ·, am-1), with components a;'s from the Galois field GF(2s). There are (2") 111 = 2111 s such m-tuples. These 2111 s m-tuples form a vector space over GF(2·1 ). The vector addition and scalar multiplication are defined in the usual way: (ao, al,··· , Gm-1)

+ (bo, b1, · · · , bm-l) = (ao + bo, al+ b1, · · · , Gm-l + bm-l),

f3 · (ao, al,··· , Gm-1)

= (/3 · GQ, f3

·al,··· , f3 · Clm-l),

where additions a; + b; and multiplication f3 . a; are carried out in GF(2 5 ). In combinatorial mathematics, the 2111 s m-tuples over GF(2') are also known to form an m-dimensional Euclidean geometry over GF(2 5 ), denoted by EG(m, 25 ) [14-16]. Each m-tuple a = (ao, a1, · · · , a111 _1) is called a point in EG(m, 2·1 ). The all-zero m-tuple, 0 = (0, 0, · · · , 0), is called the origin of the geometry EG(m, 25 ). Let a be a nonorigin point in EG(m, 25 ) (i.e., a -:/= 0). Then, the 25 points {f3a: f3 E GF(2s)} constitute a line (or 1-flat) in EG(m, 2s). For convenience, we use the notation {f3a} to represent this line. Because this line contains the origin (with f3 = 0), we say that {f3a} passes through the origin. Let ao and a be two linearly independent points in EG(m, 2·1 ) (i.e., f3oao + f3a-:/= 0 unless f3o = f3 = 0). Then, the collection of the following 2s points, {ao

+ f3a},

with f3 E GF(2s), constitutes a line in EG(m, 2·1 ) that passes through the point ao. Line {f3a} and the line {ao + /fa} do not have any point in common. Suppose that they have a common point. Then, for some /3 1 and /3 11 in GF(2 1 ), f3'a = ao

+ f3"a.

Section 8.5

Euclidean Geometry

3HOl5

:mo + ({3 11 - f3')1l! = 0. This that :mo and mare dependent, which is a contradiction to our assumption that mo and mare two linearly independent points in EG(m, 25 ). Therefore, {,Bm} and{:mo + (3:m} do not have any common We say that {,8:ID} and {mo + /3ru) are parallel lines. Note that {/321} is simply a onedimensional subspace of the vector space of an the 2111 s m-tuples over GF(2 5 ), and {:mo + /32:1} is a coset of {/32:1}. Let lt»o be a point not on line {/32:1} or on line {2:10 + /3:m}. The line (lt»o + /32:1} passes through the point lt»o and is parallel to both {/32:1} and {2:10 + {32:1}. Kn EG(m, 25 ), for every line passing the origin, there are 2Cm-l)s - 1 lines to it. A line ((3ru} and the 2Cm-l)s - 1 lines to it are said to form a bundle. The 2Cm-l)s lines in a parallel bundle are parallel to each other. Basically, the 2<m-l)s lines in a parallel bundle simply correspond to a one-dimensional of the vector space of all them-tuples over GF(2) and its 2Cm-l)s - 1 cosets. Let :lll1 and 1in2 be two independent points in EG(m, 25 ). The lines {210 + (3mi} and {2:10 + /3:ill2) have one point, 1ino, in common. Suppose that have another besides 1ino in common. Then, for some {3' -=ft O and (3 11 -=ft 0, we have 1 11 1ino + /3 211 =mo+ /3 a2. As a

This equality that /3' 2:11 - (3 11 2:12 = 0 and that m1 and a2 are linearly dependent. This is a contradiction to the hypothesis that 2:11 and 212 are linearly independent points in EG(m, 2'). Therefore, {2:10 + /3:m1) and {2:10 + {32:12} have only one point in common, and both pass through the point 2:10. We say that {:illo + /3ai} and {210 + /32:12) intersect at the point 1ino. Given a point 2:10 in EG(m, 25 ), there are 2111s -1 2 1 -1

(8.26)

hnes in EG(m, 2s) that intersect at 2:10 (including the line {fJao} that passes through the origin). This is an property that will be used to form orthogonal sums for the codes presented in the next section. Another important of lines is that any two points are connected by a line. Let :ill1 and :ill2 be two in EG(m, Suppose that 2:11 and 2:12 are linearly dependent. Then, ID2 = /3i2:11 for some element /3i in GF(2s). fa this case 2:11 and a2 are connected by the line {/31ini}. Suppose that :m1 and 1in2 are linearly independent. Let 2:13 = m1 + m2. Then, 1in2 = 1in1 + :m3, and 2:11 and a2 are connected by the line {:m1 + /3:m3}. The total number of lines in EG(m, 2 5 ) is 2(111-l)s (2"'s _ 1) 2 5 -1

Let m = 3 and s = 1. Consider the Euclidean geometry EG(3, 2) over GF(2). There are eight points and 28 lines. Each point 2:li is a 3-tuple over GF(2). Each line consists of two points {:mi, 21 1 ). The points and the lines are given in Table 8.3. Lines {1ino, 2:11}, {2:12, 2:13}, {a4, 2:15}, and {2:16, 2:17} are parallel and they form a parallel bundle. The lines that intersect at the point 1ill2 are {2:10, a2}, {a1, 1i112}, {1i112, ID3}, {a2, 2:14}, (1ii12, :ms), {1in2, ID6}, and {a2, 2:17 }.

306

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes TABLE 8.3: Points and lines in EG(3, 2).

(a) Points in EG(3, 2) ao = (000), a1 = (001), a4 = (100), as = (101), (b) Lines in EG(3, 2) {ao, ai} {a1, a2} {ao,a2} {a1,a3} {ao, a3} {a1, a4} {ao,a4} {a1,as} {ao,as} {a1,a6} {ao, a6} {a1, a7} {ao, a7} {a2, a3}

a2 a6

= =

(010), (110),

{a2, a4} {a2, as} {a2, a6}

{a2, a7} {a3,a4} {a3, as} {a3, a6}

a3 a7

= =

(011), (111).

{a3, a7} {a4, as} {a4, a6} {a4, a7} {as, a6} {as, a7} {a6, a7}

Now, we extend the concept of lines to planes in EG(m, 2s). Let ao, a1, · · · , a/1. beµ+ 1 linearly independent points in EG(m, 2s), whereµ < m. The 211.s points of the form

with f3i E GF(2s) for 1::: i ::: µ, constitute a µ-fl.at (or a µ-dimensional hyperplane) in EG(m, 2s) that passes through the point ao. We denote this µ-flat by {ao + f3ia1 + · · · + /3 11.afl.}. The µ-flat that consists of the 2f1.s points

passes through the origin. We can readily prove that the µ-flats {/3i a1 + f32a2 + /33a3 + · · · + /3/1.a/J.} and {ao + /31311 + f32a2 + · · · + /3/1.a/J.} do not have any point in common. We say that these two µ-flats are parallel. For any µ-flat passing through the origin, there are 2(m-fJ.)S -1 µ-flats in EG(m, 2s) parallel to it. These 2(m-fJ.)S parallel µ-flats form a parallel bundle. Note that a µ-flat in EG(m, 2s) is either a µ-dimensional subspace of the vector space of all the 2111 s 111-tuples over GF(2s) or a coset of a µ-dimensional subspace. The 2(m-fl.)s parallel µ-flats in a parallel bundle simply correspond to a µ-dimensional subspace of the vector space of all the m-tuples over GF(2) and its 2(m-l)s - 1 cosets. If afl.+l is not a point in the µ-flat {ao + /31311 + · · · + f3/J.a 11.}, then the(µ+ 1)-flat {ao + f3ia1 + · · · + (3/1.a/J. + f3 11.+1a1,,+d contains the µ-flat {ao + f3ia1 + · · · + f3 11.a11.}, Let bfl.+l be a point not in {ao + f3ia1 + · · · + /3/J.+1a11.+1}. Then, the two(µ+ 1)-flats {ao+f31a1 +· · +f3/J.a11. +f3 11.+1a11.+d and {ao+f31a1 +· · +/3/1.a/l. +/3fl.+1b/l.+1} intersect on the µ-flat {ao + /31 a1 + · · · + f3/l.a 11.} (i.e., they have the points in {ao + /31 a1 + · · · + /3 11.afl.} as all their common points). Given a µ-flat Fin EG(m, 2s), the number of(µ+ 1)-flats in EG(m, 2s) that intersect on Fis 2(111-fl.)S -1

2s

1

(8.27)

Section 85

Euclidean Geometry

31[))7

TABLE 8.4: 2-Flats in EG(3, 2). {ii!!o, 2l1, m2, 2l3}

{214, 215, 216, !ll7}

(mo, 211,214. as}

{:m2, !ll3, 226,217}

{2lo. 2l2, m4, 2l6} {2lo, 2l2, 2l5, m1}

{211, !ll3, 2l5, 2l7}

{ao. 211. 216,217}

{:m2, 813,214,215}

{211,213, &114, 216} {211 , 212, 214, 217}

{aio, ai4. ai3. 217}

{811,212.815, 216)

outside the F is contained in one and that intersect on F. The number of in EG(1n, 2·') is

one of the (1oi

{2lo, 2l3, 2l5. 216}

+

Tr 2(111-i+l)s - 1 /,L

i/11-f,L)S

Jl R 2c1,l-;+11s

_

1·

i=l

Consider the EG(3, 2) over GF(2) given in Example 8,13. There are fourteen -which are given in Table 8.4. The 2-flats that intersect on the line {21 1 , 213) are {00. 211, ru2, 03}, {01, 2l3, 215, :m7}, and {211, :m3, 214. 816}, The 2-flats {210. 211,212.213} and {2l4. 215. 2l6. 817} are parallel. we show that the elements in the Galois field GF(2 11 " 1') form an m-dimensional Euclidean EG(m, 2s). Let O! be a element of the 2ms elements in GF(2 1115 ) can be expressed as powers of a as follows: o/'° = 0. a 0 = 1. a 1 . a 2 , ... , a 2'"'- 2 . His known that GF(2 111c1 ) contains GF(25 ) as a subfield. Every element in GF(21115 ) can be as

where a;J E GF(2 5 ) for O :S j < m. There is a one-to-one correspondence between the element o/ and them-tuple (a;o, c1;1, · · · , a;. 111 -1) over GF(2·1·). the 21111 elements in GF(2 111 s) may be regarded as the 2111 s in EG(m, and GF(2 111 •1 ) as the geometry EG(m, 2s). fo this case, a µ-flat passing through the point a 10 consists of the following 21,Ls points:

where a 1o. 0/1, · · · , a 11 are µ 1

+ 1 linearly

independent elements in GF(21111 ), and

{3; E

Consider the Galois field GF(24 ) given by Table 2.8. Let m = 2. Let a be a element whose minimal polynomial is ¢,(X) = 1 + X + X 4 . Let f3 = 0t 5 . We see that {Jo= l, {31 = as. {32 = 0t 10 , and f3 3 = a 15 = 1. Therefore, the order of f3 is 3. We can readily check that the elements 0, 1, {3, {3 2

308

Majority-Logic Decodable and Finite Geometry Codes

Chapter 8

TABLE 8.5:

(X = (X2 = a3 = (X4 = as = a6 = a7 = (X8 = a9 = (XlO =

Elements in GF(2 4 )*.

(X f3 f3

+ (X + f32a 1 + (X

f3 f3a (32 + f3a (32 + a f3 + f3a (32 all = f32a a12 = 1 + f32a a 13 = 1 + f3a al4 = (32 + f32a

(0, 0) (1, 0) (0, 1) (/3' 1) (/3, /32) (1, 1) ((3, 0) (0, /3) (/32,/3) (/32' 1) ((3, /3) (/32, 0) (0, /32) (1, /32) (1, /3) (/32,/32)

*Elements in GF(2 4 ) are expressed in the form aiO + ail a, where a is a primitive element in GF(2 4), and aij is an element in GF(2 2) = (0, 1, /3, {3 2 ) with f3 =as.

form a field of four elements, GF(2 2 ). Therefore, GF(2 2 ) is a subfield of GF(2 4 ). Table 8.5 shows that every element a; in GF(2 4 ) is expressed in the form

with a;o and ail in GF(2 2 ) = {O, 1, (3, (3 2 }. We may regard GF(24 ) as the Euclidean geometry EG(2, 22 ) over GF(2 2 ). Then, the points (Xl4 (Xl4

+ 0. (X = (Xl4, + f3. a= (X8,

(Xl4 (Xl4

+ 1 . (X = (X 7' + (32. a= (XlO,

form a line passing through the point a 14 . The other four lines in EG(2, 22 ) passing through a 14 are {a 14, a 13 , a, as}, {a 14, a9 , a4, 0},

{a14, ao, a6, a2}, {a14, a12, a11, a3}.

The field GF(2111 s) may be regarded either as an extension field of GF(2s) or as an extension field of GF(2111 ). Therefore, GF(2"is) may be regarded either as the m-dimensional Euclidean geometry EG(m, 2s) over GF(2s) or as the s-dimensional Euclidean geometry EG(s, 2111 ) over GF(2 117 ).

Section 8.6

:SUS

Codes

Euclidean

3l(Q)9l

fElU!(UDIEAN GrEOMHIR\V (llJJIDJIES Let v

=

(vo. v1, · · · . v2""~2),

be a (2 s - 1)-tuple over the field element of 1N e may number the the Galois field of v with the nonzero elements of GF(2 11 " ) as follows: the is nurnbered for O ::: i ::: 2m -2. o/ is the location number of v;. we as the 111-dimensional Euclidean geometry over , 2s). Let F be a µ-flat in EG(m, 2s) that does not pass through the ac0 = 0. Based on this µ-flat F, we may form a vector over as follows: 117

v,

whose ith v; is 1 if its location number is a in F: otherwise, v; is 0. fo other the location numbers for the nonzero of VF form the of the µ-flat F. The vector VF is called the incidence vector of the µ-flat F. The incidence vector v F for the µ-flat F displays the contained in F. A very structural of the incidence vectors of the µ-flats in EG(m, 2s) not passing through the origin is their structure: a cyclic shift of the incidence vector of a µ-flat not the origin is the incidence vector of another µ-flat not through the origin Problem 8.33).

iE?fAMIPLrE :!UI!» Let m = 2 ands = 2. Consider the field GF(2 4 ). which is regarded as the Euclidean c,~.~~'D"''" over GF(2 2 ), EG(2. 22 ). From Example 8.15, the four 1-ftats (or lines) the point a 14 but not the are

=

= {al4. a7, 0/8, 0/10},

=

{a14. aB.

a. as).

= {Ol 1-\ 0/12, 0/11, 0/3}.

{al4, ao, a6, a2).

The incidence vectors for these four 1-fiats are ]L([J)
VL1

'YL 2 VL, VL,1

= (0

= = =

(0 (1

(0

0 1 0 0

0 0 1 0

0 0 0 1

0 0 0 0

0 1 0 0

0 0 1 0

1 0 0 0

1 0 0 0

1)/9

1)/]_I)

0 0 0 0

1 0 0 0

0 0 0 1

Suppose we cyclically shift the incidence vector VL 2 of line following vector: (1 0 1 0 0 0 1 0 0 0 0 0 0 0 1),

0 0 0 1

/JIB

IJl]A

0 1 0 0

1) 1) 1) 1)

We obtain the

which is the incidence vector of line {a 0 , a 2 , a 6 , a 14 }. H we cyclically shift the incidence vector VL 1 of line , we obtain the following vector: (1 0 0 0 0 0 0 0 1 1 0 1 0 0 0),

which is the incidence vector of line {ao, 018,

a9,

all}.

310

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

8.3 A (µ, s)th-order binary Euclidean geometry (EG) code of length 2 111 s - 1 is the largest cyclic code whose null space contains the incidence vectors of all the(µ+ 1)-flats of EG(m, 2s) that do not pass through the origin.

DEFINITION

Basically, the(µ, s )th-order EG code of length 2111 s - 1 is the dual code (or null space) of the code (or space) spanned by the incidence vectors of the (µ + 1)-flats of EG(m, 25 ) that do not pass through the origin. Due to the cyclic structural property of the incidence vectors of the flats in EG(m, 25 ) not passing through the origin, the code spanned by the incidence vectors of(µ+ 1)-flats not passing through the origin is cyclic and hence its dual code, the (µ, s )th-order EG code, is also cyclic. The generator polynomial of a (µ, s)th-order EG code is given in terms of its roots in GF(2"" ). Leth be a nonnegative integer less than 2 111 s. Then, we can express h in radix-2s form as follows: h

= 80 + 812 + 8222s + · · · + 8 _1im-l)s, 5

111

where O :S 8; < 2s for O :Si < m. The 2s-weight of h, denoted by W2., (h), is defined as the real sum of the coefficients in the radix-2 5 expansion of h; that is, m-1 W2,(h)

=

L

(8.28)

8;.

i=O

As an example, let m = 3 and s radix-2 2 form as follows: 45 with 80

= 2. Then, we can expand the integer

h

= 45 in

= l + 3 · 22 + 2 · 22x 2,

= 1, 81 = 3, and 82 = 2. The 22 -weight of 45 is then W22 ( 45)

= 1 + 3 + 2 = 6.

Consider the difference h - W2, (h), which we can express as follows: h - W2., (h) = 81 (2s - l) + 82(2 2s

-

1) + · · · + 8111 _1 (2(m-l)s - 1).

It is clear from this difference that h is divisible by 2s - 1 if and only if its 2s -weight, W2, (h), is divisible by 2s - 1. Let h UJ be the remainder resulting from dividing i h by 211,s - 1; that is, ih=q(2 111 s l)+h(I),

with O :s 11U) < 2"is - 1. Clearly, h(l) is divisible by 2s - 1 if and only if h is divisible by 2s - 1. Note that 1z(OJ = h. Now, we state a theorem (without proof) that characterizes the roots of the generator polynomial of a (µ, s)th-order EG code. The proof of this theorem can be found in [26, 27], and [33]. 8.3 Let a be a primitive element of the Galois field GF(2"75 ). Let h be a nonnegative integer less than 2111 s - 1. The generator polynomial g(X) of the (µ, s)th-order EG code of length 2111 s - 1 has a 17 as a root if and only if THEOREM

0 < max W2, (h(l)) :S (m - µ - 1)(2s - 1). 0:':cl<s

(8.29)

Section 8.6

Euclidean Geometry Codes

3li 1

Let m = 2, s = 2, and µ., = 0. Then, the Galois field GF(24 ) may be regarded as the Eudidean EG(2, 22 ) over GF(2 2 ). Let oc be a element in (use Table 2.8). Leth be a integer less than 15. It follows from Theorem 8.3 that the generator polynomial 1§(X) of the (0, 2)th-order EG code of 15 has as a root if and only if

a"

0 < max W22 (h(I)) :::: 3. 0:::1<2

-

The less than 15 that this condition are 1, 2, 3, 4, 6, 8, 9, and 12. Therefore, !§(X) has a, oc 2 . oc3, oc4, a 6 , . oc 9 , and a 12 as all its roots. The elements a, a 2 , , and a 8 have the same minimal polynomial, (X) = 1 + X + and the elements , a 6 , a 9 , and a 12 have the same minimal polynomial, (X) = 1+X + + + the polynomial of the (0, EG code of length 15 is g(X)

+ X + X 4 )(1 + X + + = 1 + x 4 + x 6 + x 7 + x8 . =

(1

H is interesting to note that the (0, 2)th-order EG code is the (15, 7) BCH code considered in Example 8.1. H is one-step majority-logic decodable.

Let 111 = 3, s = 2, and µ., = 1. Then, the Galois field GF(2 6 ) may be regarded as the Euclidean geometry EG(3, 22 ) over GF(2 2 ). Let oc be a primitive element in GF(2 6 ) (use Table 62). Leth be a nonnegative integer less than 63. H follows from Theorem 8.3 that the generator polynomial !);(X) of the (1, 2)th-order EG code of length 63 has oc 17 as a root if and only if 0 < max W22 (h(I)) :::: 3. 0:::1<2

The nonnegative integers less than 63 that satisfy this condition are

1. 2, 3, 4, 6, 8, 9, 12, 16, 18, 24, 32, 33, 48. Thus,

has the following roots:

From Table 6.3 we find that ]., oc, a 2 , a 4 , a 8 , oc 16 , and oc 32 have (X) = 1 + X + X 6 as their minimal polynomial. Z, a 3 , a 6 , oc 12 , a 24 , oc 33 , and a 48 have (X) = 1 + X + X 2 + X 4 + X 6 as their minimal polynomial. 31, a 9 • oc 18 , and oc 36 have the same minimal polynomial, ¢, 9 (X) = 1 + X 2 +

312

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

Therefore, the generator polynomial of the (1, 2)th-order EG code of length 63 is g(X)

= (1 + X + X6 )(1 + X + x 2 + X 4 + X 6 )(1 + x 2 + x 3 ) = 1 + x2 + x4 + xn + xB + x14 + xis.

Hence, the (1, 2)th-order EG code of length 63 is a (63, 48) cyclic code. Later we will show that this code is two-step orthogonalizable and is capable of correcting any combination of two or fewer errors. Decoding of the (µ,, s)th-order EG code of length 2ms - 1 is based on the structural properties of the Euclidean geometry EG(m, 25 ). From Definition 8.3 we know that the null space of the code contains the incidence vectors of all the (µ, + 1)-flats of EG(m, 25 ) that do not pass through the origin. Let F(µ,l be a µ,-flat passing through the point a 2"'' - 2 . From (8.27) we know that there are J

=

2<m-p.,)s - 1

-1

25 -1

(8.30)

(µ, + 1)-flats not passing through the origin that intersect on F(µ,). The incidence vectors of these J (µ, + 1)-flats are orthogonal on the digits at the locations that correspond to the points in F(tL). Therefore, the parity-check sums formed from these J incidence vectors are orthogonal on the error digits at the locations corresponding to the points in F(µ,). If there are LJ /2J or fewer errors in the received vector, the sum of errors at the locations corresponding to the points in p(11,) can be determined correctly. Let us denote this error sum with S(F(/Ll). In this manner the error sum S(FC/Ll) can be determined for every µ,-flat FCµ,l passing through the point a 2"'' - 2 . This forms the first step of orthogonalization. We then use the error sums S(FCtLl)'s corresponding to all the µ,-flats F(µ,) that pass through the point a 2111 '- 2 for the second step of orthogonalization. Let p(µ,-ll be a(µ, -1)-flat passing through the point a 2"''- 2 . From (8.27) we see that there are 2Cm-11,+l)s _

Ji=

1 -l>l

25 -1

µ,-flats not passing through the origin that intersect on F(11,-l). The error sums corresponding to these Ji µ,-flats are orthogonal on the error digits at the locations corresponding to the points in F(µ,-l). Let S(F(11,-l)) denote the sum of error digits at the locations corresponding to the points in F(µ,-ll. Then, S(FC11 - 1l) can be determined from the Ji error sums S(F(µ,J)'s that are orthogonal on S(F(µ,-ll). Since Ii > J, if there are no more than LJ /2J errors in the received vector, the error sum S(FCµ,-ll) can be determined correctly. In this manner the error sum S(FCµ,-1)) can be determined for every(µ, -1)-flat p(µ,-l) passing through the point a 2"''- 2 but not the origin. This completes the second step of orthogonalization. The error sums S(F(µ,-ll)'s now are used for the third step of orthogonalization. Let F(11,- 2 ) be a (µ, - 2)-flat passing through the point a 2111 '- 2 but not the origin. From (8.27) we see that there are 2Cm-tL+2)s

1

h=------l>Ji>l 2s

-1

Section 8.6

Euclidean

error sums S(F(µ,-ll)'s orthogonal on the error sum Hence, can be determined The error sums S(F(µ,- 2l)'s are then used for the next of This process continues until the error sums to all the 1-flats (lines) the a 2"'' - 2 but not the determined. There are 2111s - 1 25 -1

>···>Ji>]

>

such error sums on the error digit e2'"'-2 at the location e2"" -2 can be determined from these error sums that there are no more than LI /2J errors in the received vector. Because the code is other error digits can successively be decoded in the same manner. Because the of each error digit requires µ, + 1 steps of orthogothe (µ, s)th-order EG code of length 2111 " - 1 is therefore (µ, + 1)-step decodable. The code is capable of correcting HUH~'-"'"JH,

= l2(111-1.1)s - 1 -

t

2(2'-1)

ML

!J

(8.31)

2

or fewer errors. Therefore, its minimum distance is at least 2tML

+1=

2"(im-JJ.- 2 )s

+ · · · + 2· + 1). 1

we need only J orthogonal error sums For /..t = 0, a (0, EG code is

Note that at each of to determine an error sum for the next majority-logic decodable.

Let m = 2, s = 2 andµ, = 0. Consider the (0, 2)th-order EG code of length 15. From 8.17 we know that this code is the (15, 7) BCH code (also a type-1 DTI code). The nun space of this code contains the incidence vectors of all the 1-flats in EG(2, 22 ) that do not pass through the origin. To decode e14 , we need to determine the incidence vectors of the 1-flat passing through the point a 14 , where a is a primitive element in GF(2 4 ). There are 22-2 -

1=

1

22 -1

-1=4

such incidence vectors, which are given in Example 8.16. These four vectors are orthogonal on the digit position a 14 . In fact, these are exactly the four orthogonal vectors w1, 'i'W2, w3, and w4 given in Example 8.1.

IE){AMl?llE fLZ((J)

Let m = 4, s = 1, and µ, = 1. Consider the (1, l)th-order EG code of 24 - 1 = 15. Let a be a primitive element of GF(2 4) given by Table 2.80 Let h be

314

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

a nonnegative integer less than 15. It follows from Theorem 8.3 that the generator polynomial g(X) of this code has c/1 as a root if and only if

Note that h(O) = h. From the preceding condition we find that g(X) has the following roots: a, a 2 , a 3 , a4, a 5 , a 6 , a 8 , a 9 , a 10 , and a 12 . From Table 2.9 we find that g(X)

= (1 + X + X 4)(1 + X + x 2 + x 3 + X4)(1 + X + X2) = 1 + x + x 2 + x 4 + x 5 + x 8 + x 10 .

It is interesting to note that this EG code is actually the (15, 5) BCH code studied in

Example 8.12. The null space of this code contains the incidence vectors of all the 2-flats of the EG( 4, 2) that do not pass through the origin. Now, we will show how to form orthogonal check-sums based on the structure of EG( 4, 2). First, we treat GF(24) as the geometry EG( 4, 2). A 1-flat passing through the point a 14 consists of the points of the form a 14 + aai with a E GF(2). There are thirteen 1-flats passing through a 14 but not the origin, a 00 = O; they are {a13, a14), {a12, a14), {all, a14}{a10, a14}, {a9, a14}, {as, a14), {a 7 al4}, {a6, a14), {as, a14), {a4, a14), {a3, a14), {a2, a14}. {a, al4}. 0

For each of these 1-flats, there are 2<4-1)-1 _ 1

l=

21 -1

-1=6

2-flats not passing through the origin that intersect on it. Each of these 2-flats consists of the points of the form a 14 + aai + bai, with a and bin GF(2). The six 2-flats that intersect on the 1-flat {a 13, a 14 } are {a4,al0,a13,a14},

{a 7, al2, a13, a14},

{ao,as,a13,a14},

{a1, as, a13, a14},

{a9, all, a13, al4}, {a3, a6, al3, a14}.

The incidence vectors of these six 2-flats are Wll w12

W13 W14 W15 W16

= = = = = =

ao (0 (0 (0 (1 (0 (0

a.J.

a2

0 0

0 0 0 0 0 0

0

0 1 0

a3 a" as 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0

a6

0 0 0 0 0 1

a7 as a9 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 0

alO

1 0 0 0 0 0

an 0 0 1 0 0 0

a12

al3

a14

0 1 0 0 0 0

1 1 1 1 1 1

1) 1) 1) 1) 1) 1).

Clearly, these six vectors are orthogonal on digits at locations a 13 and a14. Let r be the received vector. The parity-check sums formed from these six

Section 8.6

A11

A12

A1s

ft16

Euclidean

Codes

3l1S

= w11 , r = e4 + e10 + e13 + e14 = w12 · r = e7 + e12 + e13 + e14 = W13 · r = e9 + en + e13 + e14 = ~'"14 · = eo + es + e13 + e14 = ~"':'15 · r = ei + es + e13 + e14 = W16 , r = e3 + e6 + e13 + e14.

Vl/e see that these six check-sums on {ee 13 , e14} are the same check sums given in Example 8.12. we can determine the error sum e13 + e14 corresponding to the 1-flat {OL 13 , OL 14 } from these six check-sums. In the same manner we can determine the error surns to the other twelve 1-flats passing a 14 . Because J = 6, we need to determine only six error sums to any six 1-flats passing through a 14 . We then use these error sums to determine €14· Thus, the l)th-order EG code of length 15 is a two-step majority-logic decodable code. Except for certain special cases, there is no formula for enumerating the number of parity-check digits of EG codes. Complicated combinatorial expressions for the number of parity-check digits of EG codes can be found in [17] and [18]. One special case isµ = m 2. The number of digits for a (m - 2, s)th-order EG code of length 2111 ,. - 1 is

-(m+l\S 1 n- k m } - .

(8.33)

This result was obtained independently by Smith [19] and by MacWilliams and Mann [20]. Fors = L vve obtain another special subclass of EG which happens to be the class of RM codes of length 2111 - 1 in form [11, 21-29]. A µth-order RM is a(µ, EG code. Hvve add an overall bit to each codeword of this code, we obtain the µ-th order RM code of in Section 4.3. Let a be a element of the Galois field nonnegative integer less than 2m. H follows from Theorem 8.3 that the generator polynomial g(X) of the µth-order RM code of length 2111 - 1 has 011, as a root if and if (8.34) The µth-order cyclic RM code of length 2111

-

dmin = 2m-i-1 -

1 has the following parameters:

L

Because J = d111 ;11 - 1, cyclic RM codes are completely orthogonalizable. The cyclic structure of RM codes was proved independently by Kasami et al. [21, 22] and Kolesnik and Mironchikov [23].

316

Chapter 8

Majority-logic Decodable and Finite Geometry Codes

Except for RM codes and other special cases, EG codes in general are not completely orthogonalizable. For moderate-length n, the error-correcting capability of an EG code is slightly inferior to that of a comparable BCH code; however, the majority-logic decoding for EG codes is more simply implemented than the decoding for BCH codes. Thus, for moderate n, EG codes provide rather effective error control. For large-length n, EG codes become much inferior to the comparable BCH codes, and the number of majority-logic gates required for decoding becomes prohibitively large. In this case, BCH codes are definitely superior to the EG codes in error-correcting capability and decoding complexity. A list of EG codes with n ::: 1023 is given in Table 8.6. See [24] for a more extensive list. TABLE 8.6: A list of EG codes.

m

s

µ

n

k

J

3 4 4 2 5 5 5 6 6 6 6 3 3 2 7 7 7 7 7 8 8 8 8 8 8 4 4 4 2 9 9 9

1 1 1 2 1 1 1 1 1 1 1 2 2 3 1 1 1 1 1 1 1 1 1 1 1 2 2 2 4 1 1 1

1 2 1 0 3 2 1 4 3 2 1 1 0 0 5 4 3 2 1 6 5 4 3 2 1 2 1 0 0 7 6 5

7 15 15 15 31 31 31 63 63 63 63 63 63 63 127 127 127 127 127 255 255 255 255 255 255 255 255 255 255 511 511 511

4 11 5 7 26 16 6 57 42 22 7 48 13 37 120 99 64 29 8 247 219 163 93 37 9 231 127 21 175 502 466 382

2 2 6 4 2 6 14 2 6 14 31 4 20 8 2 6 14 30 62 2 6 14 30 62 126 4 20 84 16 2 6 14

tML

1 1 3 2 1 3 7 1 3 7 15 2 10 4 1 3 7 15 31 1 3 7 15 31 63 2 10 42 8 1 3 7

Section 8.6

Euclidean Geometry Codes

3:H

TABLE 8.6: ( continued)

m

s

µ

9 9 9 9 3 3

1 1 1 1 3 3 1 1 1 1 1 1 1 1 2 2 2 2 5

4 3 2 1 1 0 8 7 6 5 4 3 2 1

10

10 10

10 10

10 10 10 5 5 5 5 2

ill,

511 511 511 511 511 511 1023 1023 1023 1023 1023 1023 1023

1023

,,.)

1023 1023 1023 1023 1023

2 1 0

0

k

J

ffMJl

256 130 46 10 448 139 1013 968 848 638 386 176 56 11 988 748 288 31 781

30 62 126 254 8 72 2 6 14 30 62 126 254 510 4 20 84 340 32

15 31 63 127 4 36 1 3 7 15 31 63 127 255 2 10 42 170

16

subclass of EG codes is the subclass of codes with 111 = 2 and EG code of length 11 = 22' - 1. The null space of this code contains the incidence vectors of all the lines in EG(2. not passing the H follows from (8.30) that 2s check-sums orthogonal on any code can be formed. Therefore, the minimum di.stance of the code is at least 2s + 1. H follows from (8.29) that the generator polynomial i;;(X) of this code has a 1 • , ...• a 2' and their conjugates as roots. The polynomial -l + 1 can be factored as follows: A very

/Jo

= 0. A code in this subclass is a (0,

-1

+1=

-1

+ l)(x2'c2'-1) + ... + x2'-1 +

The first factor, -l + 1, has a 0 = 1, a 2 '+ 1 . a 2C2 '+ll, ... , a< 2 '- 2l( 2 '+l) as all its roots. the second factor, ,(X) = 1 + x 2'-l + x 2 C2'-ll + ... + x 2'( 2'-ll, has 1 2 ,x . ot • .... a 2' as roots. Therefore, v(X) must be a multiple of i;;(x) ( or divisible by g(X)) and hence it is a code polynomial of the (0, s )th-order EG code of length n = 22s - 1. This code polynomial ,(x) has a weight of exactly 2s + 1. This weight together with the bound that the minimum distance of the code is at least 2s + 1 that the minimum distance of the (0, s )th-order EG code of length n = 22 s - 1 is exactly 2s + 1. Therefore, the code is one-step completely orthogonalizable. It follows from (8.33) that the number of parity-check digits of the (0, s )th-order EG code constructed based on the two-dimensional Euclidean geometry EG(2, 25 ) is n - k

= 3s -1.

(8.35)

318

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

It is interesting to note that a (0, s )th-order EG code of length n = 22s

-

1 is also a

type-I DTI code given in Section 8.2. EXAMPLE 8.21

Consider the (0, 6)th-order EG code of length n = 22 x 6 - 1 = 4095 constructed based on the lines in EG(2, 26 ) not passing through the origin. This code is a (4095, 3367) cyclic code with a minimum distance of 65 and a rate of 0.83. Let a be a primitive element of GF(2 12 ). The generator polynomial of this code has al, a2, ... , a 64 as roots. It is one-step completely orthogonalizable and can correct up to 32 errors with one-step majority-logic decoding, either type I or type II. The error performance of this code on an A WGN channel with BPSK signaling and one-step majority-logic decoding is shown in Figure 8.10. At the BER of 10-5 , it achieves a 4-dB coding gain over the uncoded BPSK. The majority-logic decoder for this code can easily be implemented in hardware with a feedback shift register of 728 flip-flops and a majority-logic circuit with 64 inputs. This hardware implementation can achieve very high decoding speed and is quite suitable for high-speed optical networks operating at 10 Gbits or for high-speed satellite communications. Consider the NASA standard (255, 223, 33) RS code over GF(28 ). For binary transmission, 10()=-----,-----,-----,-----,----,-------,------,-----= - - · BPSK uncoded ~ BER for (4095, 3367) (0, 6)th order EG code

--- ---

-', ',,,',,,',,,',,,

'' 10-.j -

''

''

''

''

''

''

''

''

''

10- 0 ~ - - ~ - - - ~ - - ~ - - - ~ - - - ~ - - ~ - - - ~ - - ~ 2 5 8 9 10 7 3 4 6 E1,IN0 (dB)

FIGURE 8.10: Bit-error performance of the (4095, 3367) (0, 6)th-order EG code with

majority-logic decoding.

Section 8.7

Twofold EG Codes

3l19

each code is expanded into an 8-bit results in a 1784) binary code. At the the received digits are back into in GF(28 ) for decoding. From Figures 7.3 and 8.10 we see that the 3367) (0, 6)th-order EG code the 223, 33) RS code more than 0.5 dB at the BER of 10- 5 . Even though the (4095, EG code is twice as iong as the (255, 223, 33) RS code in its decoding 0u,,,_wv,, because majority-logic decoding requires only simple whereas decoding of the (255, 223, 33) RS code with algebraic '-''-'''-'V'--'""'"' in GF(28 ) to find the error-location n r n " m ~ = and the error-value enumerator. The ( 4095, 3367) EG code can also be decoded vvith several other hard- or soft-decision methods to achieve better error at the expense of decoding This will be 17. ,~,HHAHS,V

EG codes were first studied by Rudolph [3]. Rudolph's work was later extended and by other coding theorists [24-30]. for decoding EG codes were suggested Weldon [31] and by Chen [32]. Chen proved that any EG code can be decoded in no more than three Chen's decoding is based on further structure of the Euclidean geometry, which is not covered in this introductory book. There are several classes of generalized EG codes [24, 28-30, 34] that all contain EG codes as subclasses. We win not cover these generalizations however, we present a simple generalization using parallel flats next. ta.7

11\ff!fOfOU:ll IEIG

((Q)[))[E§

Let F and F 1 be any two parallel µ-flats in EG(m, 2s). We say that F and form a(µ, in EG(m, 2·'), denoted by {F, Fi}. Because F and F1 do not have any in common, the(µ, 2)-frame {F, } consists of 2f-ls+l points. Let be another /J-flat parallel to F and F1. Then, the two(µ, 2)-frames {F, Fi} and {F, F2} intersect on F. Let L be a(µ+ 1)-flat that contains the µ-flat F. Then, L contains 2·' -1 other µ-flats that are parallel to F. Each of these 2s - 1 µ-flats together with F forms a (µ, 2)-frame. There are 2·' - 1 such (1_,1,, 2)-frames that intersect on F. Clearly, these 2s - 1 (µ, 2)-frames are all contained in the (µ + 1)-flat L Any point in L but outside Fis in on one and only one of these 2s - 1 (µ, 2)-frames. Because there are 2(111-f-l)S - 1 2s -1 (1_,1, + 1)-flats that intersect on F, there are 2(1J1-fj_)S -

(2S - 1) ·

2s -1

1 =

i/11-f-l)S -

1

(8.36)

(µ, that intersect on F. Any point outside Fis in on one and one of these(µ, 2)-frames. We say that these(µ, 2)-frames are orthogonal on the µ-flat F. If F does not pass through the origin, there are

(8.37) (µ, 2)-frames that are orthogonal on F and do not pass through the origin.

320

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

Again, we regard the Galois field GF(2ms) as the geometry EG(m, 2s). Let a be a primitive element of GF(211is). For any (2ms - 1)-tuple v = (vo, v1, · · · , v2111.,_2)

over GF(2), we again number its components with the nonzero elements of GF(2ms) as usual (i.e., v; is numbered with a; for O :S i < 2ms - 1). For each (µ,, 2)-frame Q in EG(m, 2s), we define its incidence vector as follows: VQ

=

={

~

(vo, v1, · · · , v2,11.,_2),

where the ith component is v;

if a; is a point in Q, otherwise.

8.4 A (µ,, s)th-order twofold EG code of length 2111 s - 1 is the largest cyclic code whose null space contains the incidence vectors of all the (µ,, 2)-frames in EG(m, 2s) that do not pass through the origin.

DEFINITION

We now state a theorem (without proof) [34] that characterizes the roots of the generator polynomial of a(µ,, s)th-order twofold EG code. 8.4 Let a be a primitive element of the Galois field GF(2111 s). Let h be a nonnegative integer less than 2ms - 1. The generator polynomial g(X) of the (µ,, s )th-order twofold EG code of length 2ms - 1 has ct" as a root if and only if 0 < max W2, (h(I)) < (m - µ,)(2s - 1). (8.38) THEOREM

0:5cl<s

!EXAMPLE 8.22

Let m = 2, s = 3, andµ, = 1. Consider the (1, 3)th-order twofold EG code of length 63. Let a be a primitive element of GF(2 6 ) given by Table 6.2. Leth be a nonnegative integer less than 63. It follows from (8.38) that the generator polynomial g(X) of the (1, 3)th-order twofold EG code of length 63 has as a root if and only if

a"

0 < max W23 (hell) < 7. 0:5c/<3

The nonnegative integers less than 63 that satisfy this condition are 1, 2, 3, 4, 5, 6, 8, 9, 10, 12, 16, 17, 20, 24, 32, 33, 34, 40, 48. Thus, the generator polynomial g(X) has the following roots: ct

1, al6

a2 1

a3 a4 cts a6 as a9 alO al2 ' ' ' ' ' ' ' ' ' al7, a20 a24, a32 a33 1 ct34, a40 1 a4S 1

1

From Table 6.3 we find that: ct, a 2 , a4, as, a 16 , 1 X X 6.

1. The roots

'1>1 (X) = +

+

and

a 32

have the same minimal polynomial,

Section 8.7

Twofold EG Codes

3l21

'.?, The roots , ot 6 , a 12 , a 24 a 48 and + + (X) = 1 + X + 3, The roots a 5 , a 10 , a 20 , a 40 , a 17 and a 34 have the same minimal polynomial, {/»5 (X) = 1 + X + + + Therefore, ~(X) = 4P1 (X) ·

(X) · {/»5(X)

= 1 + X + X2 +

+

+

+xis+ x16 + x11

+

+ xis_

Therefore, the (1, twofold EG code of 63 with m = 2 is a (63, 45) code. In it is the (63, BCH code with a minimum distance equal to 7. To decode the(µ,, s)th-order twofold EG code of 21715 -1, we first form the sums from the incidence vectors of aH the (µ,, 2)-frames in EG (m, 25 ) that do not pass through the origin (note that these incidence vectors are in the null space of the code). Let F(1,1,) be a µ,-flat that passes through the point a 2"''- 2 . From (8.37) we see that there are J

= i<m-µ,)s

_

2

(8.39)

(µ,, 2)-frames not passing through the origin that are orthogonal on F(/L). The incidence vectors of these (f.l, 2)-frames are orthogonal on the digits at the locations that correspond to the points in F(1,1,). Therefore, the parity-check sums formed from these J incidence vectors are orthogonal on the error digits at the locations that correspond to the points in F(/J.l. Let S(F(µ,)) denote the sum of error digits at the locations corresponding to the in F<1d. Then, we can correctly determine this error sum, S(F(1,1,l), from the J check-sums orthogonal on it provided that there are no more than ~ = 2<m-1<)s-l _ 1

lJ

errors in the received vector. In this manner we can determine the error sums, S(F(1.1l)'s that correspond to all the µ-flats passing through the point a 2"''- 2 . This completes the first step of orthogonalization. After this step, the rest of orthogonalization steps are the same as those for a (µ,, s )th-order EG code. Therefore, a total ofµ,+ 1 steps of orthogonalization are needed to decode a (µ,. s)th-order twofold EG code. We can easily check that at each decoding step there are at least J = 2(m-1,1,Js -2 error sums orthogonal on an error sum for the next step. Thus, the (/.l, s)th-order twofold EG code of length 2111 ' - 1 is capable of correcting. fML

=

lfJ

= 2(m-1.1)s-l -

1

(8.40)

or fewer errors with majority-logic decoding. It has been proved [34] that the minimum distance of the (f.l, s )th-order twofold EG code of length 2111 " - 1 is exactly 2(m-1.1ls -1. Therefore, the class of twofold EG codes is completely orthogonalizable.

322

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

IEXAMl?llE 8.23

Consider the decoding of the (1, 3)th-order twofold EG of length 63 with m = 2 ands = 3. In Example 8.22 we showed that this code is a (63, 45) cyclic code (also a BCH code). The null space of this code contains the incidence vectors of all the (1, 2)-frames in EG(2, 23 ) that do not pass through the origin. Regard GF(2 6 ) as the geometry EG(2, 23 ). Let a be a primitive element of GF(2 6 ) (use Table 6.2). From (8.26) we see that there are nine lines in EG(2, 23 ) that intersect at the point a 62 . Eight of these lines do not pass through the origin. From (8.37) we see that for each of these eight lines there are six (1, 2)-frames intersecting on it. The incidence vectors of these six (1, 2)-frames are in the null space of the code, and they will be used to form parity-check sums for decoding the error digit e62 at location a 62 . Because J = 6, we need to find only six lines in EG(2, 23 ) that intersect at the point a 62 and do not pass through the origin. Let f3 = a 9 . Then, 0, 1, f3, {3 2 , {3 3 , {3 4 , f3 5 , and f3 6 (J3 7 = 1) form a subfield GF(2 3 ) of the field GF(2 6 ) (use Table 6.2). Then, each line in EG(2. 23 ) that passes through a 62 consists of the following points:

where 17 follows:

E

{O, l, {3, {3 2 ,133,134,135, {3 6}. Six lines passing through the point a 62 are as L

1

= {all

'

al6 al8 a24 a48 a58 a59 a62} ' ' ' ' ' ' '

= {al, a 7, a31, 0'41 0'42, 0'45, 0'57, a62, } L 3 = {a23, a33, a34, a37, a49, a54, a56, a62}, L 4 = {a2, a12, al9 a21, an, aSl a6l a62}, L2

0

O

O

0

Ls= L6

=

{ao, a3, a1s, a20, a22, a28, as2 0 a62}, {N9 NlO ,a 13 ,a 25 , u N30 , u N32 , u N38 , u N62} . u ,u

For each of these lines we form six (1, 2)-frames intersecting on it. A (1, 2)-frame that contains the line {a 62 + 17ai} is of the form ( {0'62

+ ryai}, {a62 + a; + ,7ai}),

where a; is not in {a 62 + 17ai}. Line L1 consists of the points {a 62 + 17a}. The point 62 62 + 17a}. Thus, a 9 is not in L1. Then, the line {a + a 9 + 17a} is parallel to {a ( {a62

+ 1]0'}, {a62 + a 9 + ,7a})

forms a (1, 2)-frame containing the line L 1. This (1, 2)-frame consists of the following points:

In this manner, for each line Li, we can form six (1, 2)-frames orthogonal on it. The incidence vectors of these 36 (1, 2)-frames are given in Tables 8.7 A through 8.7F.

Section 8.7

orthogonal on

TABLE 8.7A:

Twofold EG Codes

3l2:3l

{e11, e16, e13, e24, e4g, ess,

e59, e62}.

= (ll,16,18,21,24,31,32,35,47,48,52,54,58,59,60,62) 16,18,22,23, 26,38, 45,48, 58,59,62) 6, 11,16,18,24,30,40,41,44,48,56,58,59,61,62) = (4,5,8,11,16,18,20, 25,27.33,48,57,58,59,62) = (3, 13,14,16,17,18,24,29,34,36,42,48,58, 62) 7,9,11,15.16,18, 48, 5~53,58,59,62)

W13 (X)

W14(X) W15 (X) W16(X)

*In Tables 8.7 A through 8.7F, the integers inside the parentheses are powers of X. TABLE 8.7B: e57,e62}.

W21 = o/W22(X) = W23(X) = W24(X) = W25(X) = W26(X) =

TABLE 8.7C:

(1,7, 31,39,41, 44,45,46,52,57,62) (1, 7, 31,32,33,36,41,42,45,48,53,55,57,61,62) (0, 1,7, 12, 25, 41,42,45,49,57,59,60,62) (1. 5, 6, 7, 9, 21, 26, 28, 31, 34, 42, 45, 57, 58, 62) (1, 3, 7, 8, 10, 16, 31, 40, 42, 45, 50, 54, 57, 62) (1,4,7, 14,15,18,30,31,35, 41, 43,45,57,62) Polynomials orthogonal on {e23,

e33, e34, e37. e49, e54,

es6, e62}.

=

Vi31(X) W32(X) = W33(X) = WW34()() W35(X) = W36 =

=

9, 11, 17,23,33,34,37,41,49,51,52,54,55,56,62) (5, 15, 16, 19,23,31,33,34,36,37,38,44,49,54,56,62) (1, 13, 20, 23, 26, 33, 34, 37, 42, 43, 46, 49, 54, 56, 58, 62) (0,2,8,23,32,33,34,37,42,43,46,49,54,56,58,62) (14, 23, 24, 25, 28, 33, 34, 37, 40, 45, 47. 49, 53. 54. 56, 62) (6, 7, 10,22,23,27,29,33,34,35,37,49,54,56,59,62)

TABLE 8.7D:

Polynomials orthogonal on

{e2, e14, e19, e21, e27. e51,

e61, e62}. o/W41(X) = W42(X) = W43(X) = W44(X) = W45(X) = W46 =

(2,7,8, 11, 14,19,21,23,27,28,30,36,51,60,61,62) (0, 2, 14, 19, 21, 24, 27, 34, 35, 38, 50, 51, 55, 57, 61, 62) (2,14,15,19,21.25,26,27,29,41,46, 51,54,61,62) (1,2,3,9,14,19,21,27,33,43,44,47,51,59,61,62) (2,6, 14, 16, 17, 19,20,21,27,32,37,39,45,51,61,62) (2,5, 10, 12, 14, 18, 19,21,27,42,51,52,53,56,61,62)

To decode the code, the incidence vectors of the 36 (1, 2)-frames given in Tables 8.7 A through 8.7F are used to form parity-check sums. Let S(L;) denote the sum of error digits at the locations corresponding to the points on line L; for 1 ::: i ::: 6. Then, for each error sum S(L; ), there are six parity-check sums orthogonal

324

Majority-Logic Decodable and Finite Geometry Codes

Chapter 8

TABLE 8.7E: Polynomials orthogonal on e52, e62}. W51(X) W52(X) W53(X) W54(X) W55(X) Ws6(X)

= = = = = =

= = = = = =

e1s, e20, e22, e2s,

(0,3,6,11,13, 15, 19,20,22,28,43,52,53,54,57,62) (0,3,8,9,12,15,20,22,24,28,29,31,37,52,61,62) (0, 1,3, 15,20,22,25,28,35,36,39,51,52,56,58,62) (0,3,15, 16,20,22,26,27,28,30,42,47,49,52,55,62) (0,2,3,4, 10, 15,20,22,28,34,44,45,48,52,60,62) (0,3, 7, 15, 17, 18,20,21,22,28,33,38,40,46,52,62)

TABLE 8.7F: Polynomials orthogonal on e33, e62}. W6J (X) W62(X) W63(X) W64(X) W6s(X) W66(X)

{eo, e3,

{e9, e10, e13, e25, e30, e32,

(3, 5, 9, 10, 11, 13, 25, 30, 32, 35, 38, 45, 46, 49, 61, 62) (9, 10, 13,17,25,27,28,30,31,32,38,43,48,50,56,62) (O,l,4,9,10,13, 16,21,23,25,29,30,32,38,53,62) (8,9, 10, 13, 18, 19,22,25,30,32,34,38,39,41,47,62) (7,9, 10, 12,13,14,20,25,30,32,38,44,54,55,58,62) (2,9, 10,13,25,26,30,32,36,37,38,40,52,57,59,62)

on it. Thus, S(L;) can be determined correctly provided that there are three or fewer errors in the error vector. The error sums S(L1), S(L2), S(L3), S(L4), S(Ls), and S(L6) are orthogonal on Consequently, e62 can be determined from these error sums. Thus, the (1, 3)thorder twofold (63, 45) EG code is two-step majority-logic decodable. Because its minimum distance dmin = 7 and J = 6, it is completely orthogonalizable.

e62-

There is no simple formula for enumerating the number of parity-check digits for a general twofold EG code; however, forµ, = m - 1, the number of parity-check digits for the (m - 1, s)th-order twofold EG code of length 2ms - 1 is [34] 11 _

k= (

111

+ l)s

m

_(m-l )s 111

(8.41)

A list of twofold EG codes is given in Table 8.8. We see that the twofold EG codes are more efficient than their corresponding RM codes and are comparable to their corresponding BCH codes. For example, for error-correcting capability t = 7, there is a two-step majority-logic decodable (255, 191) twofold EG code; the corresponding RM code is a (255, 163) code that is five-step majority-logic decodable (using Chen's decoding algorithm [32], it may be decoded in two steps); the corresponding BCH code is a (255, 199) code. Twofold EG codes form a special subclass of multifold EG codes presented in [30] and [34].

Section 8.8

Projective Geometry and Projective Geometry Codes TABLE 8.8:

Twofold EG codes*.

m

s

ffll

PE

1k

J

iML

3 2

2 3 2 2 4 3 3 2 2 2 5

1 1 1 2 1 1 2 1 2 3 1

63 63 255 255 255 511 511 1023 1023 1023 1023

24 45 45 171 191 184 475 76 438 868 813

14 6 62 14 14 62 6 254 62 14 30

7 3 31 7 7 31 3 127 31 7 15

4 4

2 3 3 5 5 5 2

3l25

*The (63, 24) and (63, 45) codes are BCH codes.

Like Euclidean geometry, a projective geometry may be constructed from the elements of a Galois field. Consider the Galois field GF(2<111 +lls) that contains GF(2s) as a subfield. Let ot be a primitive element in GF(2Cm+l), ). Then, the powers )l,n+lis 2 ( +1) of a. a 0 , a 1 , · · · . a- form all the nonzero elements of GF(2 111 ' ) . Let 11

=

2(111+l)s _ 1

2s -1

= 2ms + irm-l)s + ... + 2s + 1.

(8.42)

Then, the order of f3 = a 11 is 2s - 1. The 2' elements 0, 1, {3, /3 2 ....• 13 2'- 2 form the Galois field GF(2·''). Consider the first n powers of a:

]

iJ

No element a; in r can be a product of an element in GF(2 5 ) and another element a.i in r [i.e., a; I- 17 · a.i for 17 E GF(2.,)]. Suppose that a; = 17a.i. Then, ai-.f = 17. Because 1l'- 1 = 1, we obtain aU-.ilC 2 '-l) = 1. This is impossible, since (i - j)(2s -1) < i 111 +l)s -1, and the order of a is 2cm+l)s -1. Therefore, we conclude that for oJ and a.i in r, a; I- 17a.i for any 17 E GF(2'} Now, we partition the nonzero elements of GF(2Cm+lls) into n disjoint subsets as follows: {ao, f3ao. f32ao, ... , t32'-2ao),

j3a1, f32a1, ... , 132' -2al }. {a2, /3a2. 132a2, ... . t32'-2a2},

{al'

326

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

where fJ = a 11 , a primitive element in GF(2s). Each set consists of 2s - 1 elements, and each element is a multiple of the first element in the set. No element in one set can be a product of an element of GF(2s) and an element from a different set. Now, we represent each set by its first element as follows:

with O _s: i < n. For any ai in GF(2Cm+lls), if ai = {3 1 . a; with O _s: i < n, then ai is represented by (a;). If each element in GF(2Cm+l)s) is represented as an (m + 1)tuple over GF(2s), then (a;) consists of 2s - 1 (m + 1)-tuples over GF(2s). The (m + 1)-tuple for a; represents the 2s -1 (m + 1)-tuples in (a;). All the (m + 1)-tuples representing the elements in (a;) are multiples of the (m + 1)-tuple representing a;. The (m + 1)-tuple over GF(2") that represents (a;) may be regarded as a point in a finite geometry over GF(2s). Then, the points (ao), (al), (a2), ... , (an-1)

are said to form an m-dimensional projective geometry over GF(2s), denoted by PG(m, 2s) [15, 16]. In this geometry, the 2" - 1 elements in {a;, {Jc/, ... , [JC 2 '- 2lai} are considered to be the same point in PG(m, 2s ). This is a major difference between

a projective geometry and a Euclidean geometry. A projective geometry does not have an origin. Let (a;) and (ai) be any two distinct points in PG(m, 2s). Then, the line (1-flat) passing through (or connecting) (a;) and (ai) consists of points of the following form: (8.43)

where Y/1 and 172 are from GF(2s) and are not both equal to zero. There are (2s) 2 - 1 possible choices of Y/1 and 172 from G F(2s) (excluding 171 = Y/2 = O); however, there are always 2s - 1 choices of Y/1 and Y/2 that result in the same point. For example, . . · · 2' 2 · YJl a' + 172a 1 , fJ 171 a' + fJ 172a 1 , · · · , fJ · - YJl a'

+ fJ 2'· - 2172a 1·

represent the same point in PG(m, 28 ). Therefore, a line in PG(m, 2s) consists of

points. To generate the 2" + 1 distinct points on the line {(17 1a; + 172ai)}, we simply choose 171 and 172 such that no choice (Y/1, 172) is a multiple of another choice (17~, 17~) [i.e., (171,172) i= (817~, 817~) for any 8 E GF(2")]. EXAMPLE 8.24

Let m = 2 ands = 2. Consider the projective geometry PG(2, 22 ). This geometry can be constructed from the field GF(2 6 ), which contains GF(2 2 ) as a subfield. Let n

26 -1 2 -1

=- = 22 2 + 22 + 1 = 21. 2

Projective Geometry and Projective Geometry Codes

Section 8.8

3l:27

Let a be a primitive of GF(2 6) (use Table 6.2). Let 13 = ot 21 . Then OJ,13, and 13 2 form the field GF(2 2 ). The geometry PG(2, 22 ) consists of the 21 (ao), (a1). (a2), (a3), (a4), (as), (a6). (a7), (as), (a9), (0t10), (all), (0112), (al3), (a14), (a15), (0116), (al7), (ot18), (a19), (0120).

Consider the line passing the point (ot) and (a 20 ) that consists of five 20 of the form (JJ 1a + 172a ), with 171 and 172 from GF(22 ) = {O. 1. 13,13 2 }. The five distinct points are (OI)' (a20). (ot + a20) (ot + l3a20) (ot + 1320/20)

(0157)

=

(a+ 0/41) (a+ a62)

{(a), (all), (a 14 ), (a 15 ), (a 20 )} points (a) and (a 20 ).

= =

(,620/15) (a56)

=

(a1s), (,B2al4)

=

(o114).

(an).

is the line in PG(2, 2') that passes

the

Let (a 1) be a point not on the line {(1} 1 a;+ 172ai) }. Then, the line {(1) 1 a;+ 172ai) J and the line {(n 1 a 1 + 172ai)} have (ai) as a common point only common point). We say that they intersect at (ai). The number of lines in PG(m, 25 ) that intersect at a given point is 2111s -1

___ = 1 + 25 -1

2s

+ ... +

icm-l)s.

(8.44)

Let (a 11), (a 12), ... , (a 11°+1) beµ+ 1 linearly independent points (i.e., 171a 11 + + · · · + 17r,+1a 11 + 1 = 0 if and only if 171 = 112 = · · · = 17r1 +1 = 0). Then, a µ-flat in PG(m, 2s) consists of points of the form 172a 12

1

(8.45)

where IJ; E GF(2 5 ), and not all 171,172, · · · , 11/l+l are zero. There are 2(1L+lJs - 1 choices for 1J1, 172, · · · , 17 1L+l (171 = 172 = · · · = 17 1L+l = 0 is not allowed). Because in PG(m, 25 ), there are always 2s -1 choices of 171 to 1J,L+l resulting in the same there are 2(/l+l)s - 1

- ~ - - = 1 +22

25 -1

+ ... +211.s

points in a µ.-flat in PG(m, 2s). Let a 1;,+1 be a point not in the µ-flat

(8.46)

328

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

17µ,c/1,)}. The number of µ,-flats in PG(m, 2·1 ) that intersect on a given(µ, -1)-flat in

PG(m, 2s) is 1

2(111-1.1+l)s _

_____ = 1 + 2s + ... + i{m-1.1)s. 2s

(8.47)

-1

Every point outside a (µ, - 1)-flat Fis in one and only one of the µ,-flats intersecting on F. Let v = (vo, v1, · · · , v11 _1) be an n-tuple over GF(2), where 11

=

2(111+l)s -1 2s

-1

= 1 + 2s + · · · + 2ms.

Let a be a primitive element in GF(2C111 +1>-1 ). We may number the components of v with the first n powers of a as follows: Vi is numbered cxi for O ::: i < n. As usual, ai is called the location number of vi. Let F be a µ,-flat in PG(m, 2s). The incidence vector for Fis an n-tuple over GF(2), VF=

(vo, v1, · · · , v11 _1),

whose ith component Vi

={

1 if(cxi)isapointinF, otherwise.

0

DJEnNnmN 1:t5 A (µ,, s)th-order binary projective geometry (PG) code of length n = (2(m+l)s - 1)/(2·1 - 1) is defined as the largest cyclic code whose

null space contains the incidence vectors of all the µ,-flats in PG(m,

2s ).

Leth be a nonnegative integer less than 2(m+l)s - 1 and 1iU) be the remainder resulting from dividing ih by 2(m+l)s - 1. Clearly, fi(O) = h. The 25 -weight of h, W2 , (h), is defined by (8.28). The following theorem characterizes the roots of the generator polynomial of a(µ,, s )th-order PG code (the proof is omitted). The proof of this theorem can be found in [27, 35], and [36]. 8.5 Let a be a primitive element GF(cx(m+l)s). Leth be a nonnegative integer less than 2(m+l)s - 1. Then, the generator polynomial g(X) of a (µ,, s)th-order PG code oflength n = (2(m+l)s - l)/(25 - 1) has ah as a root if and only if his divisible by 2s - 1, and

THEOREM

0 :S max W2, (h(IJ)

= j (25

-

1),

O:
with O < j :S m - µ,. EXAMP'U: 8.25

Let m = 2, s = 2, andµ, = 1. Consider the (1, 2)th-order PG code of length 11

=

2<2+1)-2 -1 22 -1

= 21.

(8.48)

Section 8.8

Geometry and Projective Geometry Codes

329

Let a be a element of GF(2 6 ). Leth be a nonnegative integer less than 63. It follows from Theorem 8.5 that the generator i,;(X) of the (L 2)th-order PG code of length 21 has as a root if and if h is divisible 3, and

a"

with O ::: j ::: L The integers that are divisible by 3 and satisfy the preceding condition are 0, 3, 6, 9, 12, 18, 33, 36, and 48. Thus, i,;(X) has oto = 1, ot3, ot6. 0/9. 0/12, 0;1s. a24. a33, ot36, and a 48 as roots. From Table 6.3 we find that (1) 0; 3 , a 6 , 01 12 , , a 33 , and a 48 have the same minimal polynomial, 2 ¢, 3 (X) = 1 + X + X + + X 6 ; and (2) a 9 , a 18 . and a 36 have ¢, 9 (X) = 1 + x 2 + X 3 as their minimal Thus, i,;(X) = (1

+

(X)¢,9(X) 2

=l+x +

+x 6 +x 7 +x 10 .

Hence, the (1, PG code of length 21 is a 11) cyclic code. H is interesting to note that this code is the 11) difference-set code considered in Example 8.9. Decoding PG codes is similar to decoding EG codes. Consider the decoding of a(µ, s)th-order PG code of length n = c2<m+l)s - l)/(2s - 1). The null space of this code contains the incidence vectors of all the µ-flats of PG(m, 2s). Let p(11,-l) be a(µ in PG(m, 2·') that contains the point (a 11 - 1 ). From (8.47) we see that there are 2Cm-11+l)s _ 1 l=-----

2s -1

µ-flats intersecting on the (µ - 1)-fiat p(1;,-l)_ The incidence vectors of these J µ-flats are orthogonal on the digits at the locations corresponding to the points in pC11 -ll. Therefore, the parity-check sums formed from these J incidence vectors are orthogonal on the error digits at the locations corresponding to the points in p(1;,- l). Let S(F(1,,-l)) denote the sum of error digits at the locations corresponding to the points in pC11-ll. Then, we can correctly determine this error sum, S(FC11- 1l), from the J check-sums orthogonal on it provided that there are no more than

!__J

l

2

= l2(m-11+l)s _ 1 2(2'-l)

J

errors in the received vector. In this manner we can determine the error sums, S(F<11- 1l)'s, corresponding to all the (l,l - 1)-flats that contain the point (a 11 - 1). We then use these error sums to determine the error sums, S(F<1~- 2l)'s, corresponding to all the(µ- 2)-fiats that contain (a 11 - 1 ). We continue this process until the error sums, S(FC1l)'s, corresponding to all the 1-flats that intersect on (a 11 - 1 ) are formed. These error sums, S(FCll)'s, are orthogonal on the error digit e11 _ 1 at the location a 11 - 1 . Thus, we can determine the value of e11 _1, A total ofµ steps of orthogonalization are required to decode e11 _1. Because the code is cyclic, we can decode other error

330

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

digits in the same manner. Thus, the code is µ,-step decodable. At the rth step of orthogonalization with 1 :s r :s µ,, the number of error sums, S(F(µ,-r+ll)'s, that are orthogonal in the error sum corresponding to a given (µ, - r )-flat p(µ,-r) is 2(111-µ,+r)s _ 1

=

fµ,-r+l

25 -1

::'.:

f.

Therefore, at each step of orthogonalization, we can always correctly determine the error sums needed for the next step provided that there are no more than LI /2J errors in the received vector. Thus, the µ,th-order PG code of length n = c2<111 +l)s - 1)/(25 - 1) is capable of correcting t

= ML

l!_J 2

= l2(111-1-,+l)s _ 1 2(25 - l )

J

(8.49)

or fewer errors with majority-logic decoding. Its minimum distance is at least 2tML

+ 1=

im-µ,)s

+ ···+

25

+ 2.

(8.50)

IEXAMPLIE 8.26

Consider the decoding of the (1, 2)th-order (21, 11) PG code with m = 2 ands = 2. The null space of this code contains the incidence vectors of all the 1-flats (lines) in PG(2, 22). Let a be a primitive element in GF(2 6). The geometry PG(2, 22) consists of 21 points, (a 0) to (a 20 ), as given in Example 8.24. Let f3 = a 21 . Then, 0, 1, /3, and /3 2 form the field GF(2 2). There are 22 + 1 = 5 lines passing through the point (a 20 ), namely, {(171ao + ,72a20)}

= {(ao), (as), (a7), (a17), (a20)},

((171al + 172a20)}

=

{(al), (all), (a14), (als), (a20)},

= {(a2), (a3), (a8), (a10), (a20)}, {(i]la4 + 172 a20)} = {(a4), (a6), (a16), (a19), (a20)), {(ryia9 + 1)2<.¥20)} = {(a9), (a12), (a13), (a18), (a20)}. {(171a2 + 172a20)}

The incidence vectors of these lines (in polynomial form) are w1 (X) w2(X) w3(X) w4 cx) ws(X)

=

1

=x = x2 = x4 = x9

+ + + + +

xs + x 7 + x 17 + x20, xn + x14 +xis+ x20, x3 + x8 + xio + x20, x6 + x16 + x19 + x20, x12 + xB + x18 + x20_

These vectors are orthogonal on digit position 20. They are exactly the orthogonal vectors for the (21, 11) difference-set code given in Examples 8.9 and 8.10.

Section 8.9

Remarks

331

TABLE 8.9: PG codes.

m

s

µ

2 2 3 3 2 4 4 4 3 3 2 5 5 5 5 2 6 6

2 3 2 2

1 1 2 1

4 2

1 3

6 6 6

2

2

2 3

1 2

3 5 2

1 1 4

2

3

2 2

2 1

6 2 2 2 2 2

1 5 4 3 2 1

/J1J

21 73

11

45 68

85 85

24 191

273

341 341 341 585 585 1057 1365 1365 1365 1365 4161 5461 5461 5461 5461 5461

315

5 9 5

2 4 2

21

10

17 5

8 2

21 85 9 73

195

45 520 184 813

33 5

1328 1063

21 85

483 76

341

3431

65

5411

5

4900 3185

21 85 341 1365

1064 119

10 42

4 36 16 2 10

21 170 32 2

10 42 170 682

For µ, = 1, we obtain a class of one-step majority-logic decodable PG codes. For m = 2, a (1, s )th-order PG code becomes a difference-set code. Thus, the difference-set codes form a subclass of the class of (1, s )th-order PG codes. For s = 1, a (1, l)th-order PG code becomes a maximum-length code. There is no simple formula for enumerating the number of parity-check digits for a general(µ,, s)th-order PG code. Rather complicated combinatorial expressions for the number of parity-check digits of PG codes can be found in [17] and [18]; however forµ, = m - 1, the number of parity-check digits for the (m - 1, s )th-order of PG code of length n = c2<m+lls - 1)/(2s - 1) is n-k=l+

m (

+

l)s

(8.51)

111

This expression was obtained independently Goethals Delsarte [35], Smith [19], and MacWilliams and Mann [20]. A list of PG codes is given in Table 8.9. PG codes were first studied by Rudolph [3] and were later extended and generalized by many others [19, 24, 25, 27, 35, 36]. ~.9

~IEMfi~J~~S fa this chapter we considered only finite geometries over Galois field GF(2s) and the construction of majority-logic decodable codes based on these geometries. Finite geometries over Galois field GF(ps), where p is a prime, can be constructed in

332

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

exactly the same way, simply by replacing 2 with p and GF(2s) with GF(ps). Construction of codes based on the flats and points in these finite geometries results in a much larger class of majority-logic decodable codes. Construction of finite geometries over GF(ps) and their application to the construction of low-density parity-check codes will be discussed in Chapter 17.

PROBLIEMS IU Consider the (31, 5) maximum-length code whose parity-check polynomial is p(X) = 1 + X 2 + X 5 . Find all the polynomials orthogonal on the digit position x 30 . Devise both type-I and type-II majority-logic decoders for this code. 8.2 P = {O, 2, 3) is a perfect simple difference set. Construct a difference-set code based on this set. a. What is the length n of this code? It». Determine its generator polynomial. c. Find all the polynomials orthogonal on the highest-order digit position x 11 - 1 . id. Construct a type-I majority-logic decoder for this code. 8.3 Example 8.1 shows that the (15, 7) BCH code is one-step majority-logic decodable and is capable of correcting any combination of two or fewer errors. Show that the code is also capable of correcting some error patterns of three errors and some error patterns of four errors. List some of these error patterns. 8.4 Consider an (11, 6) linear code whose parity-check matrix is

0 1 0 0 0

0 0 1 0 0

0 0 0 1 0

0 0 0 0 1

1 1 1 0 0

1 1 0 1 0

1 0 1 1 0

1 1 0 0 1

1 0 1 0 1

(This code is not cyclic.) a. Show that the minimum distance of this code is exactly 4. lb. Let e = (eo, ei, e2, e3, e4, es, e6, e7, es, e9, e10) be an error vector. Find the syndrome bits in terms of error digits. c. Construct all possible parity-check sums orthogonal on each message error digit e 1 for i = 5, 6, 7, 8, 9, 10. d. Is this code completely orthogonalizable in one step? 8.5 Let m = 6. Express the integer 43 in radix-2 form. Find all the nonzero proper descendants of 43. 8.6 Let a be a primitive element of GF(2 4 ) given by Table 2.8. Apply the affine permutation Z = a 3 Y + a 11 to the following vector of 16 components:

llll

= (1

1

1

0

0

1

0

1

0

1

1

0

0

0

0

1)

What is the resultant vector? 8.7 Let m = 6. Then, 26 - 1 can be factored as follows: 26 - 1 = 7 x 9. Let J = 9 and L = 7. Find the generator polynomial of the type-I DTI code of length 63 and J = 9 (use Table 6.2). Find all the polynomials (or vectors) orthogonal on the digit position x 62 ( or a 62 ).

Problems

3l3l3l

d];,g Find the generator polynomial of the type-I DTI code of length 63 and J = 7. Find all the polynomials on the digit x62 . al:,91 Show that the all-one vector is not a code vector in a maximum-length code. go]_@ Let w(X) = vo -1- v1X -1- · · · -1- v2m-2X 2"' - 2 be a nonzero code polynomial in the (2 111 - 1, m) maximum-length code whose parity-check polynomial is l]lJ(X). Show that the other 2111 - 2 nonzero code are cyclic shifts of w(X). Let wUl (X) and vUl (X) be the ith and )th shifts of w(X), respectively, with 0::; i < j < 2111 - 2. Show that ,/il(X) -:j= (X).) d];J_]_ Arrange the 2111 code vectors of a maximum-length code as rows of a 2111 x (2111 -1) array. 111 1 !llo Show that each column of this array has 2 ones and 2111 - 1 zeros. 111 1 11:D, Show that the weight of each nonzero code vector is exactly 2 - . ~tU Example 8.12 shows that the (15, 5) BCH code is two-step majority-logic decodable and is capable of correcting any combination of three or fewer errors. Devise a type-I majority-logic decoder for this code. ftB Show that the extended cyclic Hamming code is invariant under the affine permutations. !tU Show that the extended primitive BCH code is invariant under the affine permutations. 1U§ Let P = {lo, !1 , l 2 , ... , 12 ,} be a perfect simple difference set of order 28 such that 0

:s lo

Construct a vector of n

< l1 < l2 < ... < l2,

:s 2s (2s + 1).

= 22s + 2s + 1 components, v

=

whose nonzero components are V/o

(vo, v1, ... , Vn-1),

v10 , v1 1 , ••• , v1 1., ;

that is,

= V/1 = ... = V/2·' = 1.

Consider the following n x 211 matrix:

G= [(())Tin], where (1) ]!11 is an n x n identity matrix, and (2) (()) is an n x n matrix whose n rows are v and n - 1 cyclic shifts of v. The code generated by CG is a (2n. n) linear (not cyclic) code whose parity-check matrix is

Show that J = 2s + 1 parity-check sums orthogonal on any message error digit can be formed. l!JJ, Show that the minimum distance of this code is d = J + 1 = 2s + 2. (This code is a half-rate quasi-cyclic code [20].) 1U.6 Prove that if J parity-check sums orthogonal on any digit position can be formed for a linear code (cyclic or noncyclic), the minimum distance of the code is at least J +l. 1U.7 Consider the Galois field GF(24 ) given by Table 2.8. Let f3 = a 5 . Then, {0, 1, {3, {3 2 } form the subfield GF(2 2 ) of GF(2 4 ). Regard GF(2 4 ) as the two-dimensional Euclidean geometry over GF(2 2 ), EG(2, 22 ). Find all the 1-flats that pass through the point a 7 . fil,

334

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

8o18 Consider the Galois field GF(2 6 ) given by Table 6.2. Let f3 = a 21 . Then, (0. 1, /3, /3 2 } form the subfield GF(2 2 ) of GF(2 6 ). Regard GF(2 6 ) as the threedimensional Euclidean geometry EG(3, 22 ). 63 . lllo Find all the 1-flats that pass through the point a bo Find all the 2-flats that intersect on the 1-flat, {a 63 + rya ), where 77 E GF(2 2 ). fU9 Regard GF(2 6 ) as the two-dimensional Euclidean geometry EG(2, 23 ). Let f3 = a 9 . Then, {O.l,/3,/32,f3 3 ,f34,f3 5 ,f3 6 ) form the subfield GF(2 3 ) of GF(2 6 ). Determine all the 1-flats that pass through the point a 21 . 820 Let m = 2 ands= 3. ao Determine the 23 -weight of 47. bo Determine max W2,(47U>). O:cc/<3

Co

Determine all the positive integers h less than 63 such that 0 < max W2,(h<1>)::: 23 -1. O:ccl <3

8021 Find the generator polynomial of the first-order cyclic RM code of length 25 - 1. Describe how to decode this code. ~t22 Find the generator polynomial of the third-order cyclic RM code of length 26 - 1. Describe how to decode this code. 823 Let m = 2 ands = 3. Find the generator polynomial of the (0, 3)th-order EG code of length 22 x 3 - 1. This code is one-step majority-logic decodable. Find all the polynomials orthogonal on the digit location a 62 where a is a primitive element in GF(22 x 3 ). Design a type-I majority-logic decoder for this code. K24 Let m = 3 ands = 2. Find the generator polynomial of the (1, 2)th-order twofold EG code of length 23 xz - 1. Describe how to decode this code. lt25 Prove that the (m -2)th-order cyclic RM code of length 2111 -1 is a Hamming code. (Hint: Show that its generator polynomial is a primitive polynomial of degree m.) ft26 Prove that the even-weight codewords of the first-order cyclic RM code of length 2111 - 1 form the maximum-length code of length 2111 - 1. 8.27 Let O < µ < m - l. Prove that the even-weight codewords of the (m - µ - l)thorder cyclic RM code of length 2111 - 1 form the dual of the µth-order RM code of length 2111 - 1. (Hint: Let g(X) be the generator polynomial of the (m - µ - l)thorder cyclic RM code C. Show that the set of even-weight codewords of C is a cyclic code generated by (X + l)g(X). Show that the dual of the µth-order cyclic RM code is also generated by (X + l)g(X).) 8J!8 The µth-order cyclic RM code of length 2111 - 1 has a minimum distance of dmin = 2m-v - 1. Prove that this RM code is a subcode of the primitive BCH code of length 2111 - 1 and designed distance 2111 -v - 1. (Hint: Let g(X)RM be the generator polynomial of the RM code and let g(X) sc H be the generator polynomial of the BCH code. Prove that g(X)scH is a factor of g(X)RM-) 8.29 Show that extended RM codes are invariant under the affine permutations. 8.30 Let m = 3, s = 2 and µ = 2. Find the generator polynomial of the (2, 2)th-order PG code constructed based on the projective geometry PG(3, 22 ). This code is two-step majority-logic decodable. Find all the orthogonal polynomials at each step of orthogonalization. 8.31 Let L be a line in the two-dimensional Euclidean geometry EG(2, 2s) that does not pass through the origin. Let v;::. be the incidence vector of L. For O < i ::: 225 - 2, let v~l be the ith cyclic shift of v ;::.. Prove that

Bibliography [t32,

3l3l5

Let[, be a line in the two-dimensional projective geometry PG(2, 2s). Let v;:: be the incidence vector of£. For O < i ::c 22s + 2s, let w~> be the ith cyclic shift of w;::. Prove that

wl() -j=.w;::.

i!o:l2l Prove that a cyclic shift of the incidence vector of a µ-fiat in EG(m, 2s) not passing through the origin is the incidence vector of another µ-fiat in EG(m, 2s) not passing through the origin. [t34l Consider the cyclic product code whose component codes are the (3, 2) code generated g1 (X) = 1 + X and the (7, 4) Hamming code generated by ~ 2 (X) = 1 + X + x 3 . The component code C1 is completely orthogonalizable in one step, and the component code C2 is completely orthogonalizable in two steps. Show that the product code is completely orthogonalizable in two steps. (fo general, if one component code is completely orthogonalizable in one step, and the other component code is completely orthogonalizable in L steps, the product code is completely orthogonalizable in L steps [37].)

]_o

I. S. Reed, "A Class of Multiple-Error-Correcting Codes and the Decoding ~v,,vu,v "IRE IT-4: 38-49, September 1954.

'.t J. L. Massey, Threshold Decoding, MIT Press, Cambridge, 1963.

2lo L. D. Rudolph, "A Class of Majority Logic Decodable Codes," IEEE Trans. Inform. IT-13: 305-7, 1967.

0o T. Kasami, L. Lin, and W.W. Peterson, "Some Results on Cyclic Codes Which Are Invariant under the Affine Group and Their Applications," Inform. 2(5 and 6): 475-96, November 1968. !!lo W. W. Peterson and E. J. Weldon, Jr., Error-Correcting Codes, 2d ed. MIT Cambridge, 1972. t6io S. Lin and G. Markowsky, "On a Class of One-Step Majority-Logic Decodable Cyclic Codes," IBM J. Res. Dev., January 1980.

7o R B. Yale, "Error-Correcting Codes and Linear Recurring Sequences," Lincoln Laboratory Report, pp. 33-77, Lincoln Labs., MIT, Cambridge, 1958. :!t N. Zierler, "On a Variation of the First-Order Reed-Muller Codes," Lincoln

Laboratory Report, pp. 38-80, Lincoln Labs., MIT, Cambridge, 1958. «Jlo J. Singer, "A Theorem in Finite Projective Geometry and Some Applications

to Number Theory," AMS Trans., 43: 377-85, 1938. ]_1[t

T. A. Evans and H. B. Mann, "On Simple Difference Sets," Sankhya, 11: 464-81, 1955.

Ho L. D. Rudolph, "Geometric Configuration and Majority Logic Decodable Codes," M.E.E. thesis, University of Oklahoma, Norman, 1964.

336

Chapter 8

Majority-Logic Decodable and Finite Geometry Codes

12. E. J. Weldon, Jr., "Difference-Set Cyclic Codes," Bell Syst. Tech. J., 45: 1045-55, September 1966. 13. F. L. Graham and J. MacWilliams, "On the Number of Parity Checks in Difference-Set Cyclic Codes," Bell Syst. Tech. I., 45: 1046-70, September 1966. 141. R. D. Carmichael, Introduction to the Theory of Groups of Finite Order, Dover, New York, 1956. 15. H. B. Mann, Analysis and Design of Experiments, Dover, New York, 1949. 16. A. P. Street and D. J. Street, Combinatorics of Experimental Design, Oxford Science Publications, Inc., Clarendon Press, Oxford, 1987. 17. N. Hamada, "On the p-rank of the Incidence Matrix of a Balanced or Partially Balanced Incomplete Block Design and Its Applications to Error-Correcting Codes," Hiroshima Math. I., 3: 153-226, 1973. 18. S. Lin, "On the Number of Information Symbols in Polynomial Codes," IEEE Trans. Inform. Theory, IT-18: 785-94, November 1972. 19. K. J. C. Smith, "Majority Decodable Codes Derived from Finite Geometries," Institute of Statistics Mimeo Series, No. 561, University of North Carolina, Chapel Hill, 1967. 20. F. J. Mac Williams and H. B. Mann, "On the p-rank of the Design Matrix of a Different Set," Inform. Control, 12: 474-88, 1968. :21. T. Kasami, S. Lin, and W.W. Peterson, "Linear Codes Which Are Invariant under the Affine Group and Some Results on Minimum Weights in BCH Codes," Electron. Commun. lap., 50(9): 100-106, September 1967. :2:2. T. Kasami, S. Lin, and W. W. Peterson, "New Generalizations of the Reed-Muller Codes, Pt. I: Primitive Codes," IEEE Trans. Inform. Theory, IT-14: 189-99, March 1968. :23. V. D. Kolesnik and E.T. Mironchikov, "Cyclic Reed-Muller Codes and Their Decoding," Prob!. Inform. Transm., no. 4: 15-19, 1968.

:241. C. R. P. Hartmann, J. B. Ducey, and L. D. Rudolph, "On the Structure of Generalized Finite Geometry Codes," IEEE Trans. Inform. Theory, IT-20(2): 240-52, March 1974. :25. D. K. Chow, "A Geometric Approach to Coding Theory with Applications to Information Retrieval," CSL Report No. R-368, University of Illinois, Urbana, 1967. :26. E. J. Weldon, Jr., "Euclidean Geometry Cyclic Codes," Proc. Symp. Combinatorial Math., University of North Carolina, Chapel Hill, April 1967. 27. T. Kasami, S. Lin, and W. W. Peterson, "Polynomial Codes," IEEE Trans. Inform. Theory, 14: 807-14, 1968.

Bibliography

Zit P. Delsarte. "A Geometric Approach to a Class of natorial 6: 340-58, 1969.

3l3l1

Codes," J. Cambi-

Zl9\ P. Delsarte, J. M. Goethals, and J. MacWilliams. "On GRM and Related

Codes," Inform.

16: 403-42.

311:Do S. Lin and K. P.

1970. to Multifold Euclidean 1975.

Codes," Inform. 31L E. J. Weldon, Jr., "Some Results on

Error-Correcting

ueco(1n1g," Chap. 8 in 1968.

H. Mann,

3Zo C. L. Chen, "On Majority-Logic Decoding of Finite Trans. Inform. IT-17(3): 332-36, May 1971.

"IEEE

21::t T. Kasami and S. Lin, "On Majority-Logic Decoding for Duals of Primitive Polynomial Codes," IEEE Trans. IT-17(3): 322-31. May 1971.

3.o}o S. "Multifold Euclidean IT-19(4): 537-48, July 1973.

Codes," IEEE Trans. Inform. Theory,

3tt J. M. Goethals and P. Delsarte, "On a Class of Majority-Logic Decodable Codes," IEEE Trans. Inform. Theory. IT-14: 182-89, March 1968.

3(6io E. J. Weldon, Jr., "New Generations of the Reed-Muller Codes. Part H: Nonprimitive Codes," IEEE Trans. Inform. Theory, IT-14: 199-205, March 1968. 370 S. Lin and E. J. Jr. "Further Results on Cyclic Product Codes," IEEE Trans. Inform. Theory, IT-16: 452-59, July 1970.

CHAPTER

9

Trellises for inear Block Codes Constructing and representing codes with graphs have long been interesting problems to many coding theorists. The most commonly known graphical representation of a code is the trellis representation. A code trellis diagram is simply an edge-labeled directed graph in which every path represents a codeword (or a code sequence for a convolutional code). This representation makes it possible to implement maximum likelihood decoding (MLD) of a code with a significant reduction in decoding complexity. Trellis representation was first introduced by Forney [1] in 1973 as a means of explaining the decoding algorithm for convolutional codes devised by Viterbi [2]. This representation, together with the Viterbi decoding algorithm, has resulted in a wide range of applications of convolutional codes for error control in digital communications. Trellis representation of linear block codes was first presented by Bahl, Cocke, Jelinek, and Raviv [3] in 1974. The first serious study of trellis structure and trellis construction of linear codes was due to Wolf. In his 1978 paper [4], Wolf presented the first method for constructing trellises for linear block codes and proved that an nsection trellis diagram of a q-ary (n, k) linear block code has at most qmin{k.n-k) states. Right after Wolf's work, Massey presented a simple but elegant paper [5] in which he gave a precise definition of a code trellis, derived some fundamental properties, and provided some implications of the trellis structure for encoding and decoding of codes; however, these early works in trellis representation of linear block codes did not arouse much enthusiasm, and for the next 10 years, there was basically no research and development in this area. H was Fomey's paper in 1988 [6] that aroused enthusiasm for research in trellis structure of linear block codes. In this paper, Forney showed that some block codes, such as RM codes and some lattice codes, have relatively simple trellis structures. Motivated by Forney's work and the desire to achieve MLD for linear block codes to improve error performance over traditional hard-decision algebraic decoding, researchers made significant efforts to study the trellis structure and to devise trellis-based decoding algorithms for linear block codes. Developments have been dramatic and rapid, and new results are exciting. This chapter gives an introductory coverage of trellises for linear block codes with a simple and unified approach. For further study, the readers are referred to the bibliography at the end of this chapter. 9.1

f!N!TIE-STATE MACHINE MODIEL ANID TRIELUS REPRESENTATION Of A CODIE

Any encoder for a code C, block or convolutional, has finite memory that stores information of the past. Suppose the information symbols are shifted into the encoder serially, and the coded symbols are shifted out of the encoder serially. As the information symbols are shifted into the encoder they are stored in the memory for a certain finite period of time (or interval). The stored information symbols in the memory affect the output code symbols. Let r = {O, l, 2, ... } denote the entire 338

Section 9.1

Finite-State Machine Model and Trellis Representation of a Code

3l3l9l

encoding interval (or span) that consists of a sequence of time instants. The interval between two consecutive time instants is defined as a unit During this unit encoding code symbols are at the encoder based on the current information symbols that are stored in the memory, according to a certain vuvv,.»u.,c, the information symbols stored in the memory at any vuc,v,.,rn,", a specific state of the encoder at that time instant. More encoder at time-i is defined by those information stored in the memory at time-i, that affect the current output code the interval from time-i to time-(i + 1) and future code Because the memory has finite the ~""'",]'.)Q~n•vcrn of the stored information symbols are that at any time the encoder has only a finite number of allowable states. As new information are shifted into the memory some old information may be shifted out of the and there is a transition from one state to another state~a state transition. W'ith these definitions of a state and a state the encoder can be modeled as a finite-state machine (finite as shown in 9.1. the behavior of the encoder can be graphically a state in time, called a trellis diagram ( or simply trellis), as shown in Figure 9.2, that consists oflevels of nodes and edges connecting the nodes of one level to the nodes of the next level. At time-0, the encoder starts from some specific initial state, denoted so. At time-i, the encoder resides in one and only one allowable state in a finite set. In the trellis diagram, the set of aHowable states at time-i is represented by a set of nodes vertices) at the ith level, one for each allowable state. lJ1ereafter, we use the terms state, node, and vertex interchangeably. The encoder moves from one allowable state at one time instant to another allowable state at the next time instant in one unit of time. This state in the trellis diagram, is represented a directed edge called a branch) connecting the starting state to the destination state. Each branch is labeled with the code symbols that are generated the state transition. Therefore, each branch has a label. The set of allowable states at a given time instant i is called the state space of the encoder at time-i, denoted by I:;(C). A state s; E I:; ( C) is said to be reachable if there exists an information sequence that takes the encoder from the initial state so to state s; at time-i. Every state of the encoder is reachable from the initial state so. In the every node at level-i for i E r is connected by a path (defined as a sequence of connected

Memory (states;)

I; Input Combinational logic circuit

FIGURE 9.1:

/L _ _ _ _ _ _ _ _

'.__:-------,

I I I

I I I I I I I I I I I I I I I I I

O;

Output

A finite-state machine model for an encoder with finite memory.

340

Chapter 9

Trellises for Linear Block Codes ith level

t 2nd level

t

-<---

State transition

/ Time-0

Time-1

Time-i Time---

FIGURE 9.2:

Trellis representation of a finite-state encoder.

branches) from the initial node. The label sequence of this path is a code sequence ( or a prefix of a code sequence). Every node in the trellis has at least one incoming branch except for the initial node and at least one outgoing branch except for a node that represents the final state of the encoder at the end of the entire encoding interval. Encoding of an information sequence is equivalent to tracing a path in the trellis starting from the initial node so. H the encoding interval is semi-infinite, the trellis continues indefinitely; otherwise it terminates at a final node, denoted by s1 . Convolutional codes have semi-infinite trellises (to be discussed in Chapter 12), whereas the trellises for block codes terminate at the end of the encoding interval. In this graphical representation of a code, there is a one-to-one correspondence between a codeword (or code sequence) and a path in the code trellis; that is, every codeword is represented by a path in the code trellis, and conversely, every path in the code trellis represents a codeword. This trellis representation of a code makes it possible to implement MLD with a significant reduction in decoding complexity. This topic will be discussed in later chapters on trellis-based soft-decision decodings. For i E r, let /; and O; denote the input information block and its corresponding output code block, respectively, during the interval from time-i to time-(i + 1). Then, the dynamic behavior of the encoder for a linear code is governed by two functions: O;

=

where f;(s;, !;) :fa f;(s;, I/) for!; :fa

f;(s;, I;),

I/.

Section 9.1

Finite-State Machine Model and Trellis Representation of a Code

s;+1

wheres;

E

:E;(C) and s;+1

E

= g; (s;,

3~-1

I;),

:E;+1CC) are called the current and next states,

fo the trellis diagram for the current and next states are connected by an edge (s;, s;+1) labeled with O;. A code trellis is said to be time-invariant if there exists a finite period (0, L ... , v} c r and a state space :E (C) such that

]., :E;(C) c :E(C) for O ::=: i < v, and I:;(C) 2, !1 = f and g; = g for an i E r.

= :E(C) for i

~

v, and

A code trellis that is not time-invariant is said to be time-varying. A treHis diagram for a block code is, in general, time-varying; however, a trellis diagram for a i::onvolutional code is usuaUy time-invariant. Figures 9.3 and 9A depict a timevarying trellis diagram for a block code and a time-invariant trellis diagram for a convolutional code, respectively.

0

FIGURE 9.3: A time-varying trellis diagram for a block code.

FIGURE 9.4: A time-invariant trellis diagram for a convolutional code.

342

Chapter 9

Trellises for linear Block Codes

In this chapter we are concerned only with trellises for binary linear block codes. Trellises for convolutional codes will be discussed in Chapter 12. 9.2

B!T-lEVIEl TRELUSES FOR BINARY UNEAR IBlOCK CODES

In this section we develop the basic concepts, structural properties, and construction of the primitive bit-level trellises for binary linear block codes. Consider a binary (n, k) linear block code C with generator and parity-check matrices, G and H, respectively. During each encoding interval r, a message of k information bits is shifted into the encoder memory one bit at a time and encoded into a codeword of n code bits. The n code bits are formed and shifted onto the channel in n bit times. Therefore, the encoding span r is finite and consists of n + 1 time instants,

r = (0, 1, 2, ... , n}. Let v = (vo, v1, ... , v11 _1) be the codeword generated at the output of the encoder. The code bit Vi is generated during the bit interval from time-i to time-(i + 1). We can represent C with an n-section trellis diagram over the time span r, in which every branch is labeled with a single code bit. Let E(C) denote the encoder for C. 9.1 Ann-section bit-level trellis diagram for a binary linear block code C of length n, denoted by T, is a directed graph consisting of n + 1 levels of nodes (called states) and branches (also called edges) such that:

lDEFIN.ll'HON

1. For O :::: i :::: n, the nodes at the ith level represent the states in the state space :E; (C) of the encoder E ( C) at time-i. At time-0 (or the zeroth level) there is only one node, denoted by so, called the initial node ( or state). At time-n (or the nth level), there is only one node, denoted by sf ( or s11 ), called the final node ( or state). 2. For O :::: i < n, a branch in the section of the trellis T between the ith level and the (i + l)th level (or time-i and time-(i + 1)) connects a states; E :E;(C) to a state s;+1 E :Ei+1CC) and is labeled with a code bit vi that represents the encoder output in the bit interval from time-i to time-(i + 1). A branch represents a state transition. 3. Except for the initial node, every node has at least one, but no more than two, incoming branches. Except for the final node, every node has at least one, but no more than two, outgoing branches. The initial node has no incoming branches. The final node has no outgoing branches. Two branches diverging from the same node have different labels; they represent two different transitions from the same starting state. 4. There is a directed path from the initial node so to the final node sf with a label sequence (vo, v1, ... , v11 _1) if and only if (vo, v1, ... , v11 _1) is a codeword in C. Two states in the code trellis are said to be adjacent if they are connected by a branch. During one encoding interval r, the encoder starts from the initial state so, passes through a sequence of states (so, s1, ... , Si, ... , Sf),

Section 9.2

Bit-Level Trellises for Binary Linear Block Codes

3l43l

generates a code sequence

and then reaches the final state sr. In the this sequence is a v~'""'~v,,u;, the initial node so to the final node Figure 9.3 depicts the 8-section bit-level trellis for the (8, 4) RM code of minimum distance 4. For O :::: i :::: n, let I:Ei (C) I denote the of the state space :Ei ( C). Then, the sequence is called the state space complexity profile, which is a measure of the state v~,uy,v,-u of then-section bit-level code trellis T. VVe will prove later that for O :::: i :::: n, I'Ei ( C) I is a power of 2. We define which is called the state space dimension at time-i. When there is no confusion, we simply use Pi for Pi ( C) for The sequence (po, Pl,···, P11)

is called the state space dimension profile. Because l'Eo(C)I = l'2: 11 (C)I = 1, Po = p 11 = 0. From Figure 9.3 we find that the state space complexity profile and the state space dimension profile for the (8, 4) RM code are (1, 2, 4, 8, 4. 8, 4, 2. 1) and (0, 1, 2, 3, 2, 3, 2, 1, 0), respectively. To facilitate the code trellis we arrange the generator matrix G in a special form. Let w = (vo, v2, ... , v11 _1) be a nonzero binary 11-tuple. The first nonzero component of w is called the leading 1 of v, and the last nonzero component of w is called the trailing 1 of v. For example, the leading and trailing l's of the 8-tuple (0, 1, 0, 1, 0, 1, 0, 1, 0) are v1 and V6, A generator matrix G for C is said to be in trellis oriented form (TOF) if the following two conditions hold:

JL The leading 1 of each row of G appears in a column before the any row below it. '.t No two rows have their trailing l's in the same column.

1 of

Any generator matrix for C can be put in TOF by two steps of Gaussian elimination (or elementary row operations). Note that a generator matrix in TOF is not necessarily in systematic form. rEXAMIPllE \lll.1

Consider the first-order RM code of length 8, RM(l, 3). His an (8, 4) code with the following generator matrix:

G= [

HH

1 1 1 1 1 1

0 0 1 0 1 0 1 0 1 0

344

Trellises for Linear Block Codes

Chapter 9

It is not in TOF. By interchanging the second and the fourth rows, we have

, G=

l

1 0 O 0

1 1 O 0

1 1 1 1 1 1 ] 0 1 0 1 0 1 110011. 0 0 1 1 1 1

We add the fourth row of the matrix to the first, second, and third rows. These additions result in the following matrix in TOF:

lgo] = llll =

1 0 1 0 1 O O 1 1 0 0 0 0

g1

GrncM

g2 g3

0 1 1 1

0 0 1 1

0 1 0 1

where TOGM stands for trellis oriented generator matrix.

n

In the following development of trellises for codes, we need to establish the correspondence between digit ( or bit) positions and encoding time instants in r = {O, 1, 2, ... n }, as shown in Figure 9.5. For a codeword v = (vo, v1, ... , v11 _1), the ith code digit v; is generated in the interval between the time instant i and the time instant i + 1. Therefore, the ith bit position corresponds to time-i and time-(i + 1). Let g = (go, g1, ... , g 11 _1) be a row in GrncM for code C. Let ¢,(g) = {i, i + 1, ... , j} denote the smallest index interval that contains all the nonzero components of g. This says that g; = 1 and gj = 1, and they are the leading and trailing l's of g, respectively. This interval ¢(g) = {i, i + 1, ... , j} is called the digit ( or bit) span of g. From Figure 9.5 we see that ¢,(g) occupies the time span from time-i to time-(j + 1); that is, {i, i + 1, ... , j + 1}. In terms of time, we define the time span of g, denoted by T(g), as the following time interval: T(g) ~ {i, i

+ 1, ... , j + l}.

For simplicity, we write ¢,(g) = [i, j]. and T(g) = [i, j time span of g, denoted by Ta (g), as the time interval ( ) 6 Ta

Time

0 I I I I I I

CD I I I I

Vo

I I I I I I

ED I I I I

V1

g

{

[i

+ 1, j],

=

¢,(empty set),

2

i

I I I I I I

I I I I I I

ED I I I I

© 0

G

ED I I I I

(9.1)

+ 1]. Next, we define the active

for j > i for j = i. i

V;

(9.2)

+1 I I I I I I

ED-

Q

© 0

I I I I

FIGURE 9.5: Time instants and code digit positions.

n - 1

n

I I I I I I

I I I I I I

ED I I I I

vn -

1

CD I I I I

Section 9.2

Bit-Level Trellises for Binary Linear Block Codes

345

Consider the TOGM of the (8, 4) RM code given in Example 9.1. We find that the bit spans of the rows are ¢(go) = [O, 3], ¢('f!,1) = [1, ¢(g2) = [2, 5], and ¢('1!?,3) = [4, 7]. The time spans of the rows are r('l!?,o) = [O, 4], r('l!?,1) = [l, 7], r('l!?,2) = and ,('1!?,3) = [4, 8]. The active time spans of the rows 6], r 0 (g2) = [3, 5], and r 0 (g3) = [5, 7]. are ra('f!,o) = [1, 3], r 0 ('1!?,1) =

Time

0

go

Gn>GM

l

2

I

l

I

1

5

4

3 I

0

1

o,

l

g2

0

0,

g3

o, o, o,

0

l

0

3

4

gl

=

1

=

I

o,

1

l

I

6

8

7

I

0

O'

0

I

0

l

0

I

0

Bit

1

1

I

1

I

2

1

I

I I

1

0 : 0

I

1

1

5

6

I

1 7

Let go, '1/?,1, .... il;k-l be the k rows of a TOGM GroGM for an (11, k) linear code Cwi.th 'fl,/ = (g70, 811, ... ' g/,11-1) for O :::: l < k. Then

GrocM

=

[~ ] [ 'f!,1

.

=

!l;k~l

goo

801 811

802

810

g12

go."-' g1,11-l

8k~10

8k-l,l

gk-l,2

8k-:.n-l

.

J (9.3)

Let (ao, a1, ... , a1c-1) be the message of k information bits to be encoded. From (9.3) (or (3.3) ), we find that the corresponding codeword is given by v

=

(vo, v1, ... , v11 _1)

=

(ao, a1, ... , a1c-1).

r :: _ ll;k-1

= ao · 'f!',o + a1

· 'f!',1

-1

(9.4)

J

+ · · · + ak-1 · !/l;k-1·

We see that the lth information bit a1 affects the output codeword v of the encoder E(C) over the bit span ¢(1);1) of the /th row r,1 of the TOGM
3:46

Chapter 9

Trellises for Linear Block Codes

from time-i to time-(j + 1). At time-i, the input information bit a1 is regarded as the current input. At time-(i + 1), a1 is shifted into the encoder memory and remains in the memory for j - i units of time. At time-(j + 1), it will have no effect on the future output code bits of the encoder, and it is shifted out of the encoder memory. Now, we give a mathematical formulation of the state space of the n-section bit-level trellis for an (n. k) linear code Cover GF(2) with a TOGM GrnGM· At time-i, 0 ::: i :S n, we divide the rows of GrnoM into three disjoint subsets: Vi, vi+l, · · · , v j

1. G; consists of those rows of GroGM whose bit spans are contained in the interval [O, i - 1]. 2. G{ consists of those rows of GroGM whose bit spans are contained in the interval [i, n - 1]. 3. GJ consists of those rows of GrooM whose active time spans contain time-i. EXAMIP'LIE 9.3

Consider the TOGM GrnoM of the (8, 4) RM code given in Example 9.2. At time-2, we find that Gf = ¢. G{ = {g2, g3}, Gz = {go, gi}, where ¢ denotes the empty set. At time-4, we find that

Let A/', A{, and A} denote the subsets of information bits, ao, a1, · · · , ak-1, that correspond to the rows of Gt, G{, and Gf, respectively (see (9.4) for the do not affect the encoder outputs after correspondence). The information bits in time-i, and hence they become the past with respect to time-i. The information bits in A{ affect the encoder outputs only after time-i. Because the active time spans of the rows in Gf contain the time instant i. the information bits in Af affect not only the past encoder outputs up to time-i but also the future encoder outputs beyond time-i. Therefore, the bits in Af are the information bits stored in the encoder memory that affect the current output code bit vi and the future output code bits beyond time-i. These information bits in Af hence define a state of the encoder E (C) for the code C at time-i. Let

A;

Then, there are 2P; distinct states in which the encoder E(C) can reside at time-i; each state is defined by a specific combination of the Pi information bits in Af. These states form the state space :Ei(C) of the encoder E(C) (or simply of the code C). The parameter Pi is hence the dimension of the state space :Ei (C). In the trellis representation of C, the states in :Ei (C) are represented by 2P; nodes at the i th level of the trellis. The set A1 is called the state defining information set at time-i. From the preceding analysis we see that at time-i for O ::: i ::: n, the dimension Pi of the state space I;i (C) is simply equal to the number of rows in the TOGM

Bit-Level Trellises for

Section 9.2

Linear Block Codes

:Jl,iii,7

TABLE 9.1: Partition of the TOGM of the (8, 4) RM code.

'Jfnmmie

i 0 1 2 3 4 5 6 7

8

G;

G!

'i

l


{ii;o. 11;1.11;2.1);3}

cp

¢

{!!;1,!!;2.11;3} {11;2, 11;3} {1);3} {!];3}

{11;0} {11;0, 11;i} {go. 1J;1. 1J;2}


¢ {11;0} {11;0} {ii;o, 1):2} {go. 11;1, 1J;2} {11;0. 1);1, 1);2. g3}

¢ ¢ ¢
{11;1 ° 11;2} {11;1, 1);2. 11;3} {11;1, 1);3} {1!;3}


Pi

0 1 2 3 2 3 2 1 0

GToGM of C whose active time spans contain time-i. We say that these rows are active at time-i. Therefore, from the TOGM GToGM we can easily determine the state space dimension profile (po. p 1 , · · · , p,,) simply by counting the number of rows in GToGM that are active at time-i for O ::::: i ::::: n.

Again consider the TOGM GrncM of the (8, 4) RM code given in Example 9.2. The partition of the TOGM at each time instant in r = {O, 1. 2, 3, 4, 5, 6, 7, 8} is given the number of rows of GToGM that are active at time-i for in Table 9.1. 0::::: i ::::: 8, we obtain the state space dimension profile (0, 1, 2, 3, 2, 3, 2. 1, 0) for the 8-section trellis of the (8, 4) RM code. For O ::::: i < n, suppose the encoder E (C) is in state Si E 1:1 ( C). From time-i to tirne-(i + 1), E(C) generates a code bit v; and moves from state s; to a state s;+1 E I;;+1CC). Let 1,'S _ { (i) (i) (i)} (9.5) 'UI; 1l:1 ' g2 ' !l;p, 0

0

0

0

and A s; -_ {a 1(i) , a 2(i) ,

(i)} ... , aPI ,

(9.6)

where p; = IGf I = IAf I- The current state s; of the encoder is defined by a specific combination of the information bits in Af. Let g* denote the row in G( whose leading 1 is at bit position i. The uniqueness of this row g* (if it exists) is guaranteed by the first condition in the definition of a generator matrix in TOF. Let g;" denote the ith component of g*. Then, g;" = 1. Let a* denote the information bit that corresponds to row g*. h follows from (9.3) and (9.4) and the structure of the TOGM GrncM that the output code bit v; generated during the bit interval between time-i and time-(i + 1) is given by Pl

v1 = a*

+ La?) 1=1

·

l?,

(9.7)

3:48

Chapter 9

Trellises for Linear Block Codes

g?;

where is the ith component of g?) in G}. Note that the information bit a* begins to affect the output of the encoder E ( C) at time-i. For this reason, bit a* is regarded as the current input information bit. The second term in (9.7) is the contribution from the states; defined by the information bits in Af = {af l, at), ... , a~)} that are stored in memory. From (9.7) we see that the current output v; is uniquely determined by the current states; of the encoder E(C) and the current input information bit a*. The output code bit v; can have two possible values depending on the current input information bit a*; each value takes the encoder E(C) to a different state at time-(i + 1); that is, there are two possible transitions from the current states; to two states in :E;+1CC) at time-Ci+ 1). In the code trellis T for C, there are two branches diverging from the nodes;, labeled with O and 1, respectively. Suppose there is no such row g* in G(. Then, the output code bit v; at time-i is given by v;

=

Pi '\:""""'

(i)

Lal 1=1

(i)

. gl,i.

(9.8)

In this case we may regard that the current input information bit a* is being set to O; that is, a* = 0 (this is called a dummy information bit). The output code bit v; can take only one value given by (9.8), and there is only one possible transition from the current states; to a state in :E;+1CC). In the trellis T, there is only one branch diverging from the nodes;. From the preceding analysis we see that the output function of the encoder E(C) is given by either (9.7) or (9.8). This result shows the time-varying nature of the output function of a linear block code encoder and hence the time-varying nature of the code trellis. EXAMPLE 9.5

Consider the (8, 4) RM code with its TOGM GrncM given in Example 9.2. From Table 9.1 we find that at time-2, Gf = ¢, G{ = {g2, g3}, and G 2= {go, gi}. Therefore, A 2= {ao, a1}, the information bits ao and a1 define the state of the encoder at time-2, and there are four distinct states defined by four combinations of values of ao and a1, {00, 01, 10, 11}. We also find that g* = g2. Hence, the current input information bit at time-2 is a* = a2. The current output code bit v2 is given by v2

= [§] +

ao · g02 + a1 · g12

= [§] +

ao · 1 + a1 · 0

= [§]+ ao. where[§] denotes the current input. For every state defined by ao and a1, v2 has two possible values depending on a2. In the trellis there are two branches diverging from each state at time-2, as shown in Figure 9.3. Now, consider time-3. For i = 3, we find that Gf = ¢, G{ = {g3}, and G3= {go, g1, g2}. Therefore, A 3 = {ao, a1, a2}, and the information bits in A 3define eight states at time-3, as shown in Figure 9.3. There is no row g* in G{ with leading 1 at bit position i = 3. Hence, we set the

Section 9.2

Bit-Level Trellises for Binary linear Block Codes

current input information bit a*

= 0. The output code bit v3 is given by (9.8),

v3 = ao · g03 v3

fo the trellis, there is shown in 9.3.

3l~9

+ a1

· g13

+ a2 · g23

= ao · 1 + cq · 1 + a2 · 1

one branch diverging from each of the eight states, as

So far, we have formulated the state space and determined the function of a linear block code encoder E(C). Next we need to determine the state transition of the encoder from one time instant to another. Let ~o denote the row in
(9.10)

The information bits in Af+i define the state space :E;+1CC) at time-(i + 1). The change from Af to AJ+i defines the state transition of the encoder E(C) from the current states; defined by Af to the next state s;+1 defined by Af+i · We may regard the information bit a 0 as the oldest information bit stored in the encoder memory at time-i. As the encoder E(C) moves from time-i to time-(i + 1), a 0 is shifted out of the memory and a* (if it exists) is shifted into the memory. The state defining sets Af and Af+l' and the output functions given by (9.7) and (9.8), provide all the information needed to construct the n-section bit-level trellis diagram T for the (n, k) code C. The construction of the n-section trellis T is carried out serially, section by section. Suppose the trellis has been constructed up to section-i (i.e., from time-0 to time-i). Now, we want to construct the (i + l)th section from time-i to time-(i + 1). The state space :E; (C) is known. The (i + l)th section is constructed as follows: Determine
350

Chapter 9

Trellises for Linear Block Codes

IEXAMIPlE 9.6

Consider the (8, 4) RM code with its TOGM GTOcM given in Example 9.2. Suppose the trellis for this code has been constructed up to time-4. At this point, we find from Table 9.1 that G 4 = h~1, g2}. The state space I;4(C) is defined by A 4 = {a1, a2}. There are four states at time-4, which are determined by the four combinations of a1 and a2: {(0, 0), (0, 1), (1, 0), (1, 1)}. To construct the trellis section from time-4 to time-5, we find that there is no row g0 in G 4 = {g1 , g2}, but there is a row g* in G{ = {g3}, which is g3. Therefore, at time-5, we have

and A 5 = {a1,

a2, a3}.

The state space I;5 ( C) is then defined by the three bits in As. The eight states in I;5 ( C) are defined by the eight combinations of a1, a2, and a3: {(O, 0, 0), (0, 0, 1), (0, 1, 0), (0, 1, 1), (1, 0, 0), (1, 0, 1), (1, 1, 0), (1, 1, 1)}. Suppose the current state s4 at time-4 is defined by (a1, a2). Then, the next state at time-5 is either the state ss defined by (a1, a2, a3 = 0) or the states~ defined by (a1, a2, a3 = 1). The output code bit v4 is given by v4

= [§] + a1

· 1 + a2 · 1,

which has two values depending on whether the current input bit a3 is O or 1. Connecting each state at time-4 to its two adjacent states at time-5 by branches and labeling each branch with the corresponding code bit v4 for either a3 = 0 or a3 = 1, we complete the trellis section from time-4 to time-5. To construct the next trellis section from time-5 to time-6, we first find that there is a row g0 in = {g1, g2, g3}, which is g2, and there is no row g* in G{ =¢.Therefore,

Gs

and A 6 = {a1, a3}.

From the change from As to A6, we find that two states defined by (a1, a2 = 0, a3) and (a1, a2 = 1, a3) at time-5 move into the same state defined by (a1, a3) at time-6. The two connecting branches are labeled with

vs

=@J + a1 · 0 + (a2 = 0) · 1 + a3 · 1,

and

vs = @J + a1 · 0 + (a2 = 1) · 1 + a3 · 1, respectively, where @] denotes the dummy input. This complete the construction of the trellis section from time-5 to time-6. Continue with this construction process until the trellis terminates at time-8.

Section 9.3

State

During the interval r = {O, L ... , n}, the function of the encoder E(C) changes between (9.7) and (9.8). the set , in the summations of (9.7) and (9.8) may from one time to another. This is because each column in the TOGM in generaL not a downward shift of the column before it. Therefore, the function of E(C) is time-varying. Furthermore, from (9.10) . we see that the size and the of the state~·-,,,..,,,,,, information set may change from one time instant to another depending on whether there is an oldest information bit to be shifted out of the memory and a current information bit to be shifted into the memory. As a the state space of the encoder E ( C) varies from time to time. the trellis T is time-varying.

~t ..... ~~/)

The construction of a code trellis can be facilitated by the states at each level of the trellis. State labeling is also necessary in the implementation of a trellis-based algorithm" Next. vve method based on the information set that defines the state space at a encoding time instant fo a code each state is labeled a fixed sequence (or a This labeling can be by a l(s) with corresponding to the k information oo, a1, ... , 01c-1, in a message. At time-i, all the components of I (s) are set to zero for the components at the positions corresponding to . f . b' . As Cil} . . t he m ormat10n its m ; = {a 1Ci) , o 2Ul , ... , op; comb'mat10n o f t he p; b'its at the corresponding to the bits in Af gives the label l (s;) for the state s; (i)

(i)

(i)

0 1 '{/2 ' ···•{Ip, ·

IEJ(AMIPt.lE Sl.1

Consider the (8, 4) code in Example 9.2. At the state-defining information set is A 4= {a1. a2}. There are four states corresponding to four combinations of cq and c12. Therefore, the label for each of these four states is by (0, a1, a2, 0). At time-5, A~ = {a1, a2, c13}, and there are states. The label for each of these eight states is given by (0, a1, a2, a3). With the described state labeling, we can restate the trellis construction procedure. the trellis T has been constructed up to section-i. At this point, G}, Af, and :E; ( C) are known. Each states; E :E; (C) is labeled a k-tuple I (s; ). The (i + l)th section is constructed as follows: ]_, Determine
352

Chapter 9

Trellises for Linear Block Codes

Recall that at time-i there are two branches diverging from a state in I:i (C) if there exists a current information bit a*. One branch corresponds to a* = 0, and the other corresponds to a* = 1. For the convenience of graphical representation in the code trellis T, we use the upper branch to represent a* = 0 and the lower branch to represent a* = 1. If a* is a dummy information bit, then there is only one branch diverging from each state in I:; ( C). This single branch represents a dummy information bit. Using the described representation, we can easily extract the information bits from each path in the trellis (the dummy information bits are deleted). EXAMPLE 9.8

Consider the state labeling and trellis construction for the (8, 4) RM code given in Example 9.2 whose TOGM GroGM is repeated here:

GrncM

gol g1 g2

=

=

ll l l0O 0 l1 1O 1 O 0 1O 0l

o o

l ~

0

0

1 1 1 1 O O 0 0 1 1 1 1

·

For O ::: i ::: 8, we determine the submatrix Gf and the state-defining information set Af as listed in Table 9.2. From Af we form the label for each state in I:; ( C) as shown in Table 9.2. The state transitions from time-i to time-(i + 1) are determined by the change from Af to Af+i · Following the trellis construction procedure given described, we obtain the 8-section trellis diagram for the (8, 4) RM code as shown in Figure 9.6. Each state in the trellis is labeled by a 4-tuple. In many cases, we do not need k bits for labeling the states of the n-section bit-level trellis for a binary (n, k) linear block code C. Let (po, Pl, ... , p11 ) be the state space dimension profile of the trellis. We define Pmax(C)

6

= max

0-<:i-<:n

(9.11)

p;,

TABLE 9.2: State-defining sets and labels for the 8-section

trellis for the (8, 4) RM code.

i

G~l

0

¢ {go} {go, gi} {go, g1, g2} h~1. g2} {g1, g2, g3} {g1, g3} {g3} ¢

1 2 3 4 5 6 7 8

a*

all

ao a1

-

a2

-

-

ao

a3

-

-

a2

-

a1

-

a3

-

A~I

State Label

¢ {ao} {ao, ai} {ao,a1,a2} {a1, a2} {a1, a2, a3) {a1,a3} {a3} ¢

(0000) (aoOOO) (aoa100)

(aoa1a20) (Oa1a20)

(Oa1a2a3) (Oa1 Oa3) (OOOa3) (0000)

Section 9.3

State Labeling

353

FIGURE 9.6: The 8-section trellis diagram for the (8, 4) RM code with state labeling

the

information set

which is the maximum state space dimension of the trellis. Because p; iGJ I = IA/ I for O_:s i < n, and Gf is a submatrix of the generator matrix of C, we have Pmax(C) :S le

In general, Pmax is smaller thank. Because the number of states at any level of the trellis is less than or at most to 2Prna,(C), Pmax(C) bits are sufficient for labeling the states in the trellis. Consider the state space :E; (C) at time-i with O _:s i _:s n that is Ul} o f Pi m . f ormatwn . b.its. F or each states; E :E; (C), denne d t h e set {a 1Ul , a 2Ul , ... , ap; we form a Pmax ( C)-tuple, denoted by l (s; ), in which the first Pi components are simply · . . Pmax (C) - p; components are set to O·, t h at 1s, . a 1Cil , a 2Ul , ... , ap;Ul , an d t h e xemammg p

(i) 0 0 l (s,·) -~ (alUJ' 0 2(i) ' ···'a Pi ' ' '

· · ·'

0)

·

Then, l (s;) is the label for the states;.

For the (8. 4) RM code, the state space dimension profile of its 8-section trellis is (0, 1, 2, 3, 2, 3, 2, 1, 0) (see Figure 9.6). Hence, Pmax(C) = 3. Using 3 bits for labeling the states as described previously, we give the state labels in Table 9.3. Compared with the state labeling given in Example 9.8, 1 bit is saved. States also can be labeled based on the parity-check matrix IHI of a code. In this case, n-k bits are needed to label each state. Therefore, for high-rate codes with small n - k, labeling states based on the parity-check matrix is more efficient than labeling based on the TOGM
Consider the second-order RM code of length 16. It is a (16, 11) code with a minimum distance of 4. Its conventional generator matrix is given in Example 4.2.

354

Chapter 9

Trellises for Linear Block Codes TABLE 9.3: State labeling for the (8, 4) RM code using Pmax(C) = 3 bits.

i 0 1 2

A;

State Label

q;

(000) (aoOO) (aoa10) (aoa1a2) (a1a20)

6 7

{ao} {ao, ai} {ao, a1, a2} {a1, a2} {a1, a2, a3} {a1, a3} {a3}

8

q;

3 4 5

(a1a2a3)

(a1 a30) (a300)

(000)

Performing elementary row operations, we obtain the following TOGM:

GrocM =

go g1 g2 g3 g4 gs g6 g7

gs g9 g10

1 1 1 1 0 1 0 1 0 0 1 1 0 0 0 1 0 0 0 0 0 0 0

0 0 0 0 0 0 0

0 0 0 0 0 0 0

0 0 0 0 0 0 0

0

0 0

0

1 1 1 0 1

0

0 0 0 0

0 0 0 0

1 1 1 1

0

0

1 0 0 0

0

1 0 1 1 0 1 0 0 1 1

1

0 0 0 0

0 0 0 0

0

0 0 0 0 0 0

0 0 0 0 0

0 0 0 0 0 0 0

0 0 0

1 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

1 1 1 1 0 1 1 1 1 1 0 1 0 1 1 0 0 1 1 1 1 0 0 0 0 0 1 1 1 1

From this TOGM, we easily find the state space dimension profile, (0, 1, 2, 3, 3, 4, 4, 4, 3, 4, 4, 4, 3, 3, 2, 1, 0). The maximum state space dimension is Pmax(C) = 4. Therefore, we can use 4 bits to label the trellis states. The state-defining information sets and state labels at each time instant are given in Table 9.4. With the information given in this table, we can readily construct the 16-section bit-level trellis for the (16, 11) RM code, as shown in Figure 9.7. 9.4

STRUCTURAL PROPERTIES OIF BIT-lEVEl TRElUSIES

In this section we develop some fundamental structural properties of an n-section bit-level trellis for an (n, k) binary linear block code C. For O ::: i < j < n, let C;,J denote the subcode of C consisting of those codewords in C whose nonzero components are confined to the span of j - i consecutive positions in the set {i, i + 1, ... , j - l}. Clearly, every codeword in C;,J is of the form (0, 0, ... , 0, V;, V;+I, ... , Vj-1, 0, 0, ... , 0), ~

· ~

11-j

and C;,J is a subcode of C. It follows from the definition of C;,J and the structure of the TOGM GrncM for C that C;,J is spanned by those rows in GrocM whose

Section 9.4

Structural Properties of Bit-Level Trellises

3:55

TABLE 9.4: State-defining sets and labels for the 16-section trellis

for the (16, 11) RM code. (G?

r!J/'

0

¢

1

{1lio} {go, 1li1l {1lio, 1li1, 1li2} {1li1,1li2,!i;3} {!([;1, 1ii2, g3, !i;4} {g1, ii;3, ii;4, gs} {g3, 1,,;4, gs, g6} {g3, !i;S, g6} {!i;3, !i;S, !i;6, g7} {g3, g5. ii;7, gs} {g3, g7, gs. g9) {1§3. gs, g9} {gs. g9. g10} {gs, g10} (g10}

ao cq a2

-

i

2

3 4 5 6 7 8 9 10

11 12 13 14 15 16

¢

'

A~

§t1:1a1t1:re ll.Anlbiell

-

¢

-

{ao} {ao, ai} {ao, a1. a2} {a1, 02, a3} {a1. a2, a3, a4} {cq. a3, a4, as} {a3. 04, as. c16} {a3, 05, 06) {03. a5. a6, a7} {a3, 05, a7, as} {a3, a7, as, a9} {a3, as, a9} {as, 09, 010} {as, 010} {010}

(0000) (aoOOO) (aoa100) (aoa1 a20)

a3

ao

a4

-

as

a2

Cl6 --

a1 a4

[/7

-

as

06

a9

as

-

a7 a3

CllQ --

a9

-

as

-

010

-

-

'

¢

(a1a2a30)

(a1 a2a3a4) (a1 a3a4a5) (03040506) (03asa60) (a3a506a1) (a3a5a7as) (a3a7asa9) (a3osa90) (asa9a100)

(osa1000) (010000) (0000)

FIGURE 9.7: 16-section trellis for the (16. 11) RM code with state labeling by the state-defining information sets.

356

Chapter 9

Trellises for Linear Block Codes

bit spans are contained in the interval [i, j - l ]. The two subcodes Co,; and C;, 11 are spanned by the rows in Gr and G{, respectively, and they are called the past and future subcodes of C with respect to time-i. For a linear code B, let k(B) denote its dimension. Then, k(Co,;) = iGri, and k(C;, 11 ) = iG{i. Recall that the dimension of the state space I:;(C) at time-i is p; ( C) = I Gf 1· Then, it follows from the definitions of Gf, and G{ that

Gl,

p;(C)

=k=k-

IGri -

iG{i

(9.13)

k(Co,;) - k(C;,n).

This expression gives a relationship between the state space dimension p; ( C) at time-i and the dimensions of the past and future subcodes, Co,; and C;, 11 , of C with respect to time-i. Note that Co,; and C;, 11 have only the all-zero codeword O in common. The direct-sum of Co,; and C;, 11 , denoted by Co,; EB C;, 11 , is a subcode of C with dimension k(Co,;) + k(C;, 11 ). The partition C/(Co,; EB C;, 11 ) consists of IC/(Co,; EEl C;,n)I

= 2k-k(Co;)-k(C;,,) (9.14)

cosets of Co,; EEl C;, 11 • Equation (9.14) says that the number of states in the state space I:; (C) at time-i is equal to the number of cosets in the partition C / ( Co,; EB C;, 11 ). Let S; denote the subspace of C that is spanned by the rows in the submatrix Gf. Then, each codeword in S; is given by (i)) Gs v -_ (a1(il , a2Ul , ... , ap; · ;

= ai° . gii) + aii) . gii) + ... + ax) . g~/ where

a?)

E

(9.15)

Af for 1 ::: l ::: p;. It follows from the definitions of Gr, G{, and Gf that C

= S; EEl (Co,; EEl C.11).

The 2P; codewords in S; can be used as the representatives for the cosets in the partition C/(Co,; EB C;, 11 ). Therefore, S; is the coset representative space for the partition C /(Co,; EB C;, 11 ). From (9.15) we see that there is one-to-one correspondence between v and the states; E I:; (C) defined by (aii), aii), ... , aih. Because there is a one-to-one correspondence between wand a coset in C/(Co,; EB C;, 11 ), there is a one-to-one correspondence between a state in the state space I:; ( C) and a coset in the partition C/(Co,; EEl C;,n). With the foregoing one-to-one correspondence in the trellis T, the codeword wgiven by (9.15) is represented by a path that passes through the states; defined by the information bits aii), at), ... , aX) (i.e., a path that connects the initial state so to the final states f through the states;). If we fix the information bits aii), at), ... , aX) and allow the other k - p; information bits to vary, we obtain 2k-p; codewords of C in the coset le, (9.16) 'YEE) (Co,; EE) C;,11) = {v + U: U E Co.i EE) C;,11}

Section 9.4 Time

FIGURE 9.8: v

Structural Properties of Bit-Level Trellises

0

357

11

A coset in

A coset in

Po ,(C)/C/);

P;_,,(C)IC,!'.", 1

The paths in the code trellis that

,rp,·wpcpr,t

the

2k-p;

codewords m

EB (Co.; EB

with v as the coset In the these 2k-p; codewords are represented the that connect the initial state so to the final state Sf through the state s; at time-i defined the information bits af l, at, ... , as shown in Figure 9.8. Note that k - p; = k(Co,;) + k(C;.11), (9.17)

aXl

which is simply the dimension of Co.; EB For O::: i < j < n, let Pi.j(C) denote the linear code of length j - i obtained the first i and last n - j components of each codeword in C. from C Every codeword in Pi.i ( C) is of the form (v;, V;+J· .... Vj-J).

This code is called a punctured ( or truncated) code of C. Let Cf /i denote the punctured code of the subcode : that (9.18) It follows from the structure of the TOGiv1[
(9.19)

and tr

k(Ci.i) = k(C;.J)-

Consider the

code

po,;(C).

k(JJo.;(C))

(9.20)

From (9.19) we find that

=k-

k(C;.11)-

(9.21)

358

Chapter 9

Trellises for Linear Block Codes

This punctured code contains cg-i as a linear subcode with dimension k(Cb1) k(Co,;), We partition Po,; (C) based on cgi, , Then, the partition PO,i (C)/ cgi , consists of 2k-k(Co_;)-k(C;. 11 )

= 2P;

cosets of c0tr., ,, We can readily see that there is a one-to-one correspondence between the cosets in PO,i (C)/ Cb\ and the cosets in C /(Co,i EB Ci,n), and hence a one-to-one correspondence betwe~n the cosets in po,;(C)/Cb'."i and the states in :E;(C), The codewords in a coset in Po,; ( C) / Cb'; are simply the prefixes of the codewords in their corresponding coset in C/(Co.'i EB C;, 11 ), Hence, the codewords in a coset of Po,;(C)/Cb'.'; will take the encoder E(C) to a unique states; E :E;(C), In the trellis T, they are the paths connecting the initial state so to the states; as shown in Figure 9,8, Let L(so, s;) denote the paths in the trellis T that connect the initial state so to the states; in :E;(C), Then, L(so, s;) is a coset in the partition po,;(C)/Cb'.";, Now, we consider the punctured code p;, 11 (C), The dimension of this code is k(p;, 11 (C))

=k-

(922)

k(Co,;),

1 as a linear subcode with dimension k(C z,n r) 1 p;, 11 (C) based on c;i~, The partition p;, (C)/Cf ; consists of 11 , ,1

It contains

ctr z,n

2k-k(Co;)-k(C;. 11 )

=

k(C; ,11 ), We partition

= 2P;

cosets of Cf\, Again, there is a one-to-one correspondence between the cosets in p;, 11 (C)/Cf 1;1 and the cosets in C/(Co,i EB C;, 11 ), and hence a one-to-one correspondence between the cosets in Pi,n ( C) I er;, and the states in the state space :E; (C), In the trellis T, the codewords in a coset 11 ( C) / Cf';, form the paths that connect a state s; E :E; (C) to the final states f as shown in Figure 9,8, Let L(s;, s1) denote the paths in the trellis T that connect the state s; E :E; (C) to the final state s1 , Then, L(s;, s J) is a coset in the partition Pi,n (C) / Cf\1 , For O ::: i < j ::: n, let s;0l and s;0l denote two states on the all-zero path 0 in the trellis T at time-i and time-j, respectively, as shown in Figure 9,9, Let L(s(O), s<, 0)) denote the paths (v;, v;+1, , , , , v.1_ 1 ) of length j - i in T that connect s
p;,

to /Ol, Consider the paths in T that start from the initial state so, follow the all-zero ] Time 0

j

0

n

0

FIGURE 9,9: Paths in the code trellis that represent the codewords in C;,i,

Section 9.4

Structural Properties of Bit-Level Trellises

3l5Sl

st

path ill) to the state s?11 , pass through the paths in L(s?'i), s?"1) to the state 1, then follow the all-zero ill) until reach the final states f as shown in Figure 9.9. These paths the codewords in the subcode C;,J of C, which implies that L(/Ol /Ol) l ' } let

= ctr,.J"

(9.23)

I

Let v = (vo, v1, ... , v11 _1) be a path in the code trellis T. For O :s i < j :Sn, and s(vl be two states on the path vat time-i and time-j, respectively. Let .I

L(s?1. s;'")) denote the

of length j - i that connect s?l to sr>. Consider the

in T that start from the initial state so, follow the path v to the state the in L(s?), sr>), then follow the path v until they reach the pass final state sf as shown in Figure 9.10. These paths represent the codewords in the coset of C;,j'. v EB C;,J

{v

+ un : un E

(924)

C;,J }.

. This implies that L(s?1, st) is a coset of Cf'.i

This is a coset in the partition C / in Pi.J that L( s;M , siM)

6

=

=

p;,J (-) o/

+ ctri.j

E Pi.} (C)/Ctr i.}'

(9.25)

where p;, 1 (v) = (v;, v;+1, ... , v1-1). For any two connected states s; E :E;(C) and :Ej(C) with O s i < j s n, they must be on a path in the trellis T. H follows from (9.25) that

SJ E

tr

(9.26)

L(s;, s 1) E p;, 1 (C)/Ci,j·

Therefore, the number of is by

that connect a state s; 1

2k(C '·I'.) ,

E

:E; (C) to a state SJ

E :Ei ( C)

ifs; and SJ are connected, otherwise.

(9.27)

For O s i < j < k < n, let :Ej(s;. sk) denote the set of states in 'Ej(C) through which the paths in , Sk) connect the state s; to the state sk as shown in Figure 9.11. Let -

6

L(s;, Sj) o L(sj, sic)= {un o v: un E L(s;, s 1). v E L(s1 , s1c)}, Time 0

(9.28) l'I

(v)

S;

FIGURE 9.10: Paths in the code trellis that represent the codewords in the coset v EB C;, 1 .

360

Chapter 9

Trellises for Linear Block Codes

Time i

FIGURE 9.11:

j

k

Connectivity between two states in the code trellis.

where l!ll o v denotes the concatenation of two sequences u and v. In the trellis, L(s;, s1) o L(si, Sk) consists of those paths in L(s;, sk) that connect the states; to the state Sk through the states 1. Then, L(s;, s1c) is the following union:

u

(9.29)

This expression displays the connectivity between two states in a code trellis. 9.5

STATE LABEUNG AND TRELLIS CONSTRUCTION BASED ON THE PARITY-CHECK MATRiX

Consider a binary (n, k) linear block code C with a parity-check matrix (9.30)

where, for O :S j < n, h1 denotes the jth column of Hand is a binary (n - k)-tuple. A binary n-tuple v = (vo, v1, ... , v11 _1) is a codeword in C if and only if v · ]Hf

=

(0, 0, ... , 0).

(9.31)

'-r--' 11-k

Let On-k denote the all-zero (11 - k)-tuple (0, 0, ... , 0). For O :s i < n, let H; denote the submatrix that consists of the first i columns of H; that is, H;

=

[ho, h.1, ... , h;-1].

(9.32)

It is clear that the rank of H; is at most n - k; that is,

Rank(H;) :Sn - k.

(9.33)

Section 9.5

For each codeword :r:

E

State Labeling and Trellis Construction

3l61

C01 r., ,1

lC·IHI;

(9.34)

=i[))n-k·

Therefore, cgf is the null space of lBii. Consider the partition PO,i ( C) /Cb\. Let B be a coset in PO,i / cgf, and B i- cg.·f. For every vector till E B, till · IHI; i- ilD11-1c and is the same for aH the vectors in

that is, for

till1, m2 E

B and fill

f-

212,

(9.35)

The (n - k)-tuple till· IHI; is called the label for the coset B. Let and B2 be two different cosets in PO.f(C)/Cbrf. Let 211 E B1 and 212 E B2. H follows from the theory 3) that 211 f- till2, and of linear block codes (see ·

This expression says that different cosets in PO.i ( C) / Cb\ have different labels. Recall the mathematical formulation of the state spaces of a code trellis. There is a one-to-one correspondence between a state Sf in the state space :Ef ( C) at time-i and a coset B E PO.i ( C) / Cb"i, and the codewords of PO.i ( C) in B form the paths that connect the initial state so io the state Sf. This one-to-one correspondence leads to another definition of a state label. Let L(so, Sf) denote the set of paths in the code trellis T for C that connect the initial state so to a state Si in the state space :E; ( C) at time-i.

(IJ)J; For O ::: i ::: n, the label of a state Sf E :E; (C) based on a parity-check matrix IHI of C, denoted by l(s; ), is defined as the binary (11 - k)-tuple

]]J)JEJFTINnmN

(9.36)

for any m E L(so, Sf). For i = 0, IHI; = cf>, and the initial state so is labeled with the aH-zero (n - k)-tuple, i[))11 -1c. For i = n, L(so, s1) = C, and the final state Sf is also labeled with i[)) 11 _1c. H follows from the preceding definition of a state label, the one-to-one correspondence between the states in :Ef ( C) and the cosets in PO,i ( C) / Cb\ for O ::: i ::: n, and (9.36) that every state s; E :Ei ( C) has a unique label, and different states have

different labels. It follows from (9.33) and the foregoing definition of a state label that the number of distinct state labels at any time instant i is less than or at most equal to 2 11 -k. Due to the uniqueness of the label for each state at any time-i, the number of states in the state space I:;(C) at time-i must be upper bounded by 211 - " ; that is, [I:f(C)[ ::: 211-k

for O ::: i ::: n. Therefore, Pf(C)

= log2 [:E;(C)[

::: n - k,

362

Chapter 9

Trellises for Linear Block Codes

for O :S i :S n. This expression implies that the maximum state space dimension Pmax(C) defined by (9.11) satisfies the following bound: (9.37)

Pmax(C) :Sn - k.

For O :S i < n, lets; and s;+l be two adjacent states withs; E ~;(C) and s;+1 E ~i+l (C). Let v; be the label of the branch in the code trellis T that connects the state s; to the state s;+l· The label v; is simply the encoder output bit in the interval from time-i to time-(i + 1) and is given by (9.7) or (9.8). For every path ( vo, v1, ... , v;-1) E L(so, s; ), the path (vo, v1, ... , v;-1, v;) obtained by concatenating (vo, v1, ... , v;-1) with the branch v; is a path that connects the initial state so to the state s;+1 through the states;. Hence, (vo, v1, ... , v;-1, v;) E L(so, s;+1). Then, it follows from the preceding definition of a state label that l(s;+1) = (vo, v1, ... , v;-1, v;) · Hf+i = (vo, v1, ... , v,-1) · H; + v; ·

h;

(9.38)

=l(s;)+v;·hf. The foregoing expression simply says that given the label of the starting state s;, and the output code bit v; during the interval between time-i and time-(i + 1), the label of the destination state s;+1 is uniquely determined. Now, we present a procedure for constructing the n-section bit-level trellis diagram for a binary (n, k) linear block code C by state labeling using the parity-check matrix of the code. Let v = (vo, v1, ... , v11 _1) be a binary n-tuple. For O :S i :Sn, let Po,; (v) denote the prefix of v that consists of the first i components; that is, Po,;(v) = (vo, v1, ... , v;-1).

For i = 0, Po,o(v) is defined as the null sequence. Suppose that the trellis has been completed up to the ith section (time-i). At this point, the rows of the TOGM GrncM in the set Gf = {gfl, gfl, ... , g~/} and their corresponding information bits · ly de fi net h estate space "' a 1Ul , a 2(il , ... , aPi(il umque L..Ji (C) • L et v = a(i) . o,(i) + a(i) . o(i) + ... + a
"'l

2

"'2

Pi

"'Pi '

Then, PO.i (v) is a path connecting the initial state so to the state s; defined by . given . by a 1Ul , a2Ul , ... , aPiUl . The la b e1 o f states; 1s l(s;) = Po,;(v) · Hf. The (i + 1)-section of the code trellis is constructed by taking the following four steps:

1. Identify the special row g* (if any) in the submatrix G{ and its corresponding information bit a*. Identify the special row g0 (if any) in the submatrix Gf. Form the submatrix Gf +l by including g* in Gf and excluding g0 from Gf. 2. Determine the set of information bits Af+i = {aii+l), ai'+l), ... , aX!/l} that correspond to the rows in Gf +i · Define and label the states in ~i+l (C).

Section 9.5

State Labeling and Trellis Construction

3l!Bi3

code bit from either (9.7) ?L For each state s; E ~; there is no such row in (if there is such a row ~* in G{ at G{ at value of v; if from 7) and one if 4!.o For each connect the states; to the state s;+1 E ~;+1 with label from l( S;+l ) = I

)

"T + V; ·till;. , s;+1), with v;. This

~~'"'""~"~u

section of the treHis. until the entire code trellis is constructed.

matrix is also a follows:

in Example 9.1 is a self-dual code. we choose the matrix.

l

~,~ t

1 0 0 1

1 1 1 1 1 0 0 1 1 1 1 1 0 0 1 1 0 1 0 1 0 1

i

l

a generator matrix as

.

Using this parity-check matrix for state and the foregoing trellis construction steps, we obtain the 8-section trellis vvith state labels shown in Figure 9.12. To illustrate the construction process, we assume that the trellis has been up to time-3. At this G 3= {~o. ~1, ~2} and A 3= {ao, a1, a2} are known. The states in ~3(C) are defined by the combinations of ao, a1, and a2. These eight states and their labels are in Table 9.5. Now, we want to construct the fourth section of the trellis up to time-4. At from the TOGM GTOcM (see we find that 'lf,o = 'lf,o and there is no such row with leading 1 at the fourth bit = {~1, i;i;2}, and A 4 = {cq, a2}. The four states m at the four

FIGURE 9.12: An 8-section trellis for the (8, 4) RM code with state labeling parity-check matrix.

the

364

Chapter 9

Trellises for Linear Block Codes TABLE 9.5: Labels of the states at

tirne-3 for the (8, 4) RM code based on the parity-check matrix. Sfate§ Defined by (ao, a1, a2)

Sfate JLabel

S3 (1) S3 (2) S3 (3) S3 (4) S3

(000)

(0000)

(001)

(1010)

(010)

(1001)

(011)

(0011)

(100)

(1011)

(5)

(101)

(0001)

(110)

(0010)

(111)

(1001)

(0)

S3 (6)

S3 (7)

S3

Labels of states at time-4 for the (8, 4) RM code.

TABLE 9.7:

TABLE 9.6: Paths defined

by the four states at time-3. Path

(a1, a2)

(0, (0, (l, (1,

0) 1) 0) 1)

= v1 = v2 = V3 = VQ

(00000000) (00111100) (01011010) (01100110)

State Defined by (a1, a2)

State Label

(00)

(0000)

(01)

(0001)

(10)

(0010)

(11)

(0011)

(0)

S4 (1) S4 (2) S4 (3) S4

combinations of a1 and a2. The four codewords generated by the rows in G4 are given in Table 9.6. The four paths that connect the initial state so to the four states, (0) (1) (2) (3) . denoted by s4 , s4 , s4 , and s4 , m ~4(C) are Po,4(vo) Po,4(v2)

= (0000), = (0101),

Po,4(v1) p0,4(v3)

= =

(0011), (0110).

The submatrix H4 is

H4

=

0l

r

01

01

011

0 0 1 1 0 1 0 1

.

From po,4(v1), with O ::: j ::: 3 and H4, we can determine the labels of the four states, sl0l, sl1l, sl2l and s~3l, in ~4(C), which are given in Table 9.7. The four states and their labels are shown in Figure 9.13 at time-4. Now, suppose the encoder is in the state s?l with label l(s?) = (1001) at time-3. Because no such row g* exists at time

Section 9.5

State Labeling and Trellis Construction

3lr55

s.,

(ll)

(0000) (0000) (1010)

1

(1001)

ill

S4

(0001) 0

(0011) ~ ~~41

(1011)

~o

CY

~

(2)

S4

0

(0010)

0 (0001)

(0010) (0011) (7)

S::;

(1001)

0

tI Time-3

Time-4

FIGURE 9. 13: State labels at the two ends of the fourth section of the trellis for the (8, 4) RM code.

i

= 3, the output code bit v3 is computed from (9.8) as follows:

=0·1+1·1+0·1

= 1. the states?) is connected to the state in :E4(C) with label /(sj

2

))

+

V3 ·

lln{ = (1001) + 1 · (1011) = (0010),

which is state s~ 2l, as shown in Figure 9.13. The connecting branch is labeled with v3 = 1. The connections from the other states in :E3(C) to the states in :E4(C) are made in the same manner.

366

Chapter 9

Trellises for Linear Block Codes

State labeling based on the state-defining information sets requires k ( or Pmax(C)) bits to label each state of the trellis; however, state labeling based on the parity-check matrix requires n k bits to label each state of the trellis. In the next section we show that Pmax(C) < min{k, n - k}. Therefore, state labeling based on state-defining information sets and using Pmax ( C) bits to label each state in the trellis is the most economical labeling method. EXAMPLE 9.12

A parity-check matrix of the (16, 11) RM code considered in Example 9.10 is

H= [ ;

1 1 1 0 0

1 1 0 1 0

1 1 0 0 1

0 1 1 1 1

0 1 1 0 0

0 1 0 1 0

0 1 0 0 1

1 0 1 1 1

1 0 1 0 0

1 0 0 1 0

1 0 0 0 1

0 0 1 1 1

0 0 0 0 1 0 0 1 0 0

n

State labeling based on this matrix requires 5 bits. The 16-section trellis for the (16, 11) RM code with state labels based on His shown in Figure 9.14. Labeling states based on H requires 1 bit more than state labeling based on the maximum state-defining information set shown in Example 9.10.

FIGURE 9.14: A 16-section trellis for the (16, 11) RM code with state labeling by parity-check matrix.

Section 9.6

Trellis Complexity and Symmetry

31(61

TreHis complexity is, in general, measured in terms of the state and branch complexities. These two complexities determine the storage and computation requirements of a trellis-based decoding algorithm. For a binary k) linear block code C, the state complexity of an n-section bit-level code trellis is measured by its maximum state space dimension Pmax(C) defined by (9.11). which gives the size 2Pmax(C) of the maximum state space of the and (9.37), we showed that Pmax (C) is upper bounded by code trellis T. In both k and n - k. This implies that Pmax(C) must satisfy the following bound: Pmax(C) :S min{k, n - k}.

(9.39)

This bound was first proved by Wolf [4]. fa general, this bound is quite loose. For exarnple, consider the (8, 4) RM code. For this code, k = n - k = 4; however, PmaxCC) = 3. The third-order RM code RM(3, 6) of length 64 is a (64, 42) linear code. For this code, min{k, n - k} = 22; however Pmax(C) = 14. We see that there is a big gap between the bound and PmaxCC); however, for cyclic (or shortened cyclic) the bound min{k, n -k} gives the exact state complexity, as will be shown later. Next, we show that an (n, k) linear code C and its dual code have the same state complexity. Let CJ_ denote the dual code of C. Then, CJ_ is an (n, n - k) linear block code. Consider then-section trellis diagram for C_1_. For O ::: i < n, let I:; (CJ_) denote the state space of C_1_ at time-i. Then, there is a one-to-one correspondence between where c0Ltr the states in I:; (CJ_) and the cosets in the partition po,; ( C_1_) / c0_l_Jr, ,l ,l denotes the truncation of in the interval [O, i - 1]. Therefore, the dimension of I:;(C_1_) is given by

ct;

(9.40) Note that Po.;(C_1_) is the dual code of Cb\, and ct( is the dual code of Po,;(C). Therefore, · ' k(po,;(C_1_)) = i - k(Cb')

(9.41) =i-k(Co,;)

and J_ tr k(C0 _; ) = i - k(po,;(C)).

(9.42)

H follows from (9.40) through (9.42) that p;(C_1_)

Because k(po,;(C))

=k-

= k(po,;(C))

- k(Co,;).

k(C;. 11 ), we have

p;(C_1_) = k - k(Co,;)

k(C;, 11 ).

(9.43)

From (9.13) and (9.43), we find that for O ::: i ::: n, (9.44)

368

Chapter 9

Trellises for Linear Block Codes

This expression says that C and its dual code c..L have the same state complexity. Therefore, we can analyze the state complexity of a code trellis from either the code or its dual code; however, we must note that a code and its dual code, in general, have different branch complexities, even though they have the same state complexity. EXAMPLE 9.13

l

Consider the (16, 11) RM code given in Example 9.10. The dual code of this code is the first-order RM code RM(l, 4) of length 16. It is a (16, 5) code with a minimum distance of 8. Its TOGM is

GrncM

=

1 0 O [ 0 0

1 1 O 0 0

1 0 1 0 0

1 1 1 0 0

1 0 O 1 0

1 1 O 1 0

1 0 1 1 0

1 0 0 0 0 0 1 1 0 1 0 1 1 1 1 O O 1 1 1 1 1 1 0 0 1 1 1 1 1

0 0 0 0 1 0 1 O O 0 0 0 1 1 1

.

From this matrix, we find the state space dimension profile of the code, (0, 1,2,3,3,4,4,4,3,4,4,4,3,3,2, 1,0), which is exactly the same as the state space dimension profile of the trellis of the (16, 11) RM code. The state-defining information sets and state labels are given in Table 9.8. The 16-section trellis for the code is shown in Figure 9.15. State labeling is done based on the state-defining information sets. State labeling based on the parity-check matrix of the code would require 11 bits.

TABLE 9.8: State-defining sets and labels for the 16-section

trellis for the (16, 5) RM code. i

G~

a*

ao

0 1 2 3 4 5 6 7 8 9

1> {go} {go, g1} {go, g1, g2} {go, g1, g2} {go, g1, g2, g3} {go, g1, g2, g3} {go, g1, g2, g3} {g1, g2, g3} {g1, g2, g3, g4} {g1, g2, g3, g4} {g1, g2, g3, g4} {g1, g2, g4} {g1, g2, g4} {g1,g4} {g4} 1>

ao

-

a1

-

a2

-

10

11 12 13 14 15 16

l

-

-

C/3

-

-

-

-

-

-

ao

a4 -

-

-

-

-

a3

-

-

-

a2

-

-

a1

-

a4

-

-

A~l

State Label

1> {ao} {ao, a1} {ao, a1, a2} {ao, a1, a2} {ao, a1, a2, a3} {ao, a1, a2, a3} {ao, a1, a2, a3} {a1, a2, a3} {a1, a2, a3, a4} {a1, a2, a3, a4} {a1, a2, a3, a4} {a1,a2,a4} {a1, a2, a4} {a1, a4} {a4} 1>

(0000) (aoOOO) (aoa100) (aoa1 a20) (aoa1 a20) (aoa1a2a3) (aoa1a2a3) (aoa1a2a3)

(a1 a2a30) (a1a2a3a4)

(a1a2a3a4) (a1 a2a3a4) (a1 a2a40) (a1 a2a40) (a1 a400) (a4000) (0000)

Section 9.6

Trellis Complexity and Symmetry

31521

FIGURE 9.15: A 16-section trellis for the (16, 5) RM code with state labeling by the state-defining information set.

Let T be an 11-section trellis for an (n, k) code C with state space dimension profile (Po, p1 .... , p 11 ). The trellis Tis said to be minimal if for any other n-section trellis T' for C with state space dimension profile (p0, p~, ... , p;,) the following inequality holds: two minimal for O ::: i ::: n. A minimal trellis is unique within isomorphism; that trellises for the same code are structurally identical. A minimal trellis results in a minimal total number of states in the trellis. In fact, the inverse is also true: a trellis with a minimum total number of states is a minimal trellis. The formulation of state spaces and the construction based on the trellis-oriented generator matrix of an (n, k) linear block code given in Section 9.2 result in a minimal trellis. The proof is not given here but can be found in [7]. From (9.13) we see that the state space dimension p; at time-i depends on the dimensions of the past and future codes, Co,; and C;_ 11 • For a given code C, k( Co,;) and k(C;. 11 ) are fixed. Given an (n, k) linear code C. a permutation of the orders of the bit (or symbol) positions results in an equivalent code C' with the same weight distribution and the same error performance; however, different permutations of bit positions may result in different dimensions, k( Co.;) and k( C;. 11 ), of the past and future subcodes, Co.; and C;, 11 , and hence different state space dimensions p; at time-i. A permutation that yields the smallest state space dimension at every time of the code trellis is called an optimum permutation (orbit ordering). His clear that an optimum permutation reduces the state complexity and is often desirable. Optimum

370

Chapter 9

Trellises for Linear Block Codes

permutation is hard to find, however. Optimum permutations for RM codes are knov1n [11], but they are unknov,n for other classes of codes. The construction of RM code given in Section 4.3 gives the minimum state complexity. The branch complexity of an n-section trellis diagram for an (n, k) linear code C is defined as the total number of branches in the trellis. This complexity determines the number of computations required in a trellis-based decoding algorithm to decode a received sequence. Ann-section trellis diagram for an (n, k) linear block code is said to be a minimal branch (or edge) trellis diagram if it has the smallest branch complexity. A minimal trellis diagram has the smallest branch complexity. Consider then-section trellis diagram T for C that is constructed based on the rules and procedures described in Section 9.2. Recall that at time-i with O ::: i < n, there are two branches diverging from a state in I;; ( C) if there exists a row g* in G{; and there is only one branch diverging from a state in I;; ( C) if there exists no such row g* in G{. The existence of g* in G{ implies that there is a current input information bit a* E A{ at time-i. We define I;(a*)

~

11, 2,

ifa*dA! 'F l ,

if a*

E A! i.

(9.45)

Let [ denote the total number of branches in T. Then, n-1

[ =

L II;;(C)I · /;(a*) i=O

n-1

=

(9.46)

L2P,. I;(a*). i=O

For O ::: i < n, 2P, · I; (a*) is simply the number of branches in the ith section of trellis T. EXAMPLE 9.14

Again we consider the (8, 4) RM code given in Example 9.1. From Table 9.2 we find that Io(a*) = /1 (a*)= h(a*) = /4(a*) = 2 and /](a*)= /5(a*) = /6(a*) = h(a*) = 1.

The state space dimension profile of the 8-section trellis for the code is (0, 1. 2, 3, 2, 3, 2, 1, 0). From (9.46) we have [ = 2° · 2 + 2 1 · 2 + 22 . 2 + 23 . 1 + 22 . 2 + 23 . 1 + 22 . 1 + 21 . 1 =2+4+8+8+8+8+4+2

=44.

Section 9.6

Trellis Complexity and Symmetry

3n

codes form a large subclass of linear block codes. Many good codes are cyclic codes. The trellis structure of cyclic codes can be analyzed easily. Consider an (n, k) cyclic code Cover GF(2) with generator polynomial i!l; (X)

= 1 + gl/' + g2Av2 + ... + gn-/c-l x11-k-l + xn-k c · V

matrix for this code is given in (5.9). For convenience, we reproduce it

A here

1 g1 g2 0 1 g1

l!l;o 1!;1

1

gn-k-l g2

gn-k-l

0 0 1 0

0 0

(9.47)

IG= 0 0 ...

i!l;k-1

0

1

g1

... gn--k-1 1

g2

We see that the generator matrix is already in trellis-oriented form. For 0 ::: i < k, the time span of the i-row i!l;i is T(i!!;i)=[i,n

k+l+i].

the bit span¢ (i!!;i) = [i, n - k + i]). The active time spans of all the rows have the same length, n - k. Now, we consider then-section bit-level trellis for this (n, k) cyclic code. There are two cases to be considered: k > n - k and k ::: n - k. Consider the case k > n k. For 1 ::: i < n - k, the number of rows in 1G whose active time spans contain the time index i is i. These rows are the first i rows. For n - k < i ::: k, the number of rows whose active time spans contain the time index i is n - k. Fork < i < n, the number of rows whose active time spans contain the time index i is n - i. Because i > k, 11 - i < n - k. From the preceding analysis, we see that the maximum state space dimension is Pmax(C) = n - k, and the state space profile is (0, 1, ... , 11

-

k - 1, n - k, ... , n - k, n

k

l, ... , 1, 0).

Next, we consider the second case for which k < n - k. For 1 ::: i < k, the number of rows in 1G whose active time spans contain the time index i is i (the first i rows in IG). Fork ::: i ::: 11 - k, the number of rows whose active time spans contain the time index i is k. For n - k < i ::: 11, the number of rows in 1G whose active time spans contain i is n - i. Because i > 11 - k, n - i < k. From the foregoing analysis, we find that the maximum state space dimension is Pmax(C) = k, and the state space dimension profile is (0, 1, .. ., k - 1, k, ... , k, k - l, ... , 1, 0).

(n, k)

Combining the results of the preceding two cases, we conclude that for an cyclic code, the maximum state space dimension is Pmax(C)

= min{k, n - k}.

372

Chapter 9

Trellises for Linear Block Codes

This is to say that a code in cyclic form has the worst state complexity; that is, it meets the upper bound on the state complexity. To reduce the state complexity of a cyclic code, a proper permutation on the bit positions is needed. EXAMPLE 9.15

Consider the (7, 4) cyclic Hamming code generated by g(X) generator matrix in TOF is

G=

1

+ x + x3.

Its

go] r1101000] g1 = 0 1 1 0 1 0 0 g2

r g3

0011010. 0 0 0 1101

By examining the generator matrix, we find that the trellis state space dimension profile is (0, 1, 2, 3, 3, 2, 1, 0). Therefore, Pmax(C) = 3. The 7-section trellis for this code is shown in Figure 9.16, the state-defining information sets and the state labels are given in Table 9.9. Next, we derive a special symmetry structure for trellises of some linear block codes. This structure is quite useful for implementing a trellis-based decoder, especially in gaining decoding speed. Consider the TOGM of a binary (n, k) linear block code of even length n, 80,n-l 81.11-l

8k-l,n-1

l .

Suppose the TOGM G has the following symmetry property: for each row g in G with bit span ¢ (g) = [a, b], there exists a row g' in G with bit span ¢ (g') =

FIGURE 9.16: The 7-section trellis diagram for the (7, 4) Hamming code vvith 3-bit

state label.

Section 9.6

anc!

Trellis

sets and state labels for the 8-section trellis for the (8, 4) RM code.

TABLE 9.9:

a'' 0

1 2 3 4 5 6 7

¢ {~o} {~0,1§1} {~o. ~1- ~2} {~1- ~2. {g2, g3} {~3}

1>

ao a2

a3

(000) (c!oOO) (aoc110)

1>

a1

ao a1 a2

a3

{ao} {ao, ai} {ao,01, {a1,a2, a3} {a3} ¢

(000)

[n - l -b, n - l -a]. With this in G, we can see that for O ::: i ::: n the number of rows in G that are active at - i) is to the number of rows that lbn_;(C)I = l:Ei(C)I P11-i = Pi) in G that are active at time-i. This for O ::: i ::: V\/e can permute the rows of CG such that the resultant denoted by G', is in a reverse trellis-oriented 1 of any 1-0-w

]_" The trailing 1 of each row appears in a column before the below it.

If we rotate the matrix G' 180° we obtain a matrix G 11 in which the ith row'/,,;' is the (k -1- i)th row ~;,-l-i of G' in reverse order trailing 1 of ~~-l-l becomes the leading 1 of~;', and the leading 1 of ~;,_ 1 _ 1 becomes the trailing 1 of ~;'). From the foregoing, we see that G" and G are identical in the sense that 1> (cg;') = ¢ (~i) for O ::: i < k. the 11-section trellis T for C has the mirror the last n /2 sections of T form the mirror image of the first 11 /2 sections of T

!EXAMr>IL!E :BU«:» Consider the (8, 4) RM code given in Example 9.1 with TOGM

1 1 1 0 0 0 0 1 0 1 1 0 1 0 0 1 1 1 1 0 0 0 0 0 1 1 1 i _l_

.l

Examining the rows of GmGM, we find that (~o) = [O, 3], ¢(g3) = [4. and ~3 are symmetrical with each other. Row g1 has bit span [L 6] and is with itself. Row 11;2 has bit span [2. 5] and is also with itself. we permute f,1 and g2. VVe obtain the matrix in reverse trellis~oriented form:

1 1 0 0 1 1 1 1 0 1 1 0 0 0 1 1

0 0 1 1

374

Chapter 9

Trellises for Linear Block Codes

-l

Rotating G' 180° counterclockwise, we obtain the following matrix:

go

II

G"=

g{

g~

g~

-J

1 1 1 1

:n~ ]·

=[0101 0 0 1 1 0 0 0 0 1 1 1 1

We find that G" and G are in fact identical, not just structurally identical. Therefore, the 8-section trellis T for the (8, 4) RM code has mirror symmetry with respect to boundary location 4: the last four sections form the mirror image of the first four sections, as shown in Figures 9.3 and 9.6. Figure 9.7 shows that the 16-section bit-level trellis for the (16, 11) RM code also has mirror-image symmetry. In fact, trellises for all RM codes have mirror-image symmetry. For the case in which n is odd, if the TOGM GTOGM of a binary (n, k) code C has the mirror symmetry property, then the last (n - 1)/2 sections of the 11-section trellis T for C form the mirror image of the first (n - 1) /2 sections of T. From (9.47) we readily see that the generator matrix of a cyclic code has the mirror symmetry property. Therefore, the trellises of all cyclic codes have mirror-image symmetry. Figure 9.16 displays the mirror-image symmetry of the (7,4) Hamming code. 9.7

rn.lELUS SIECTIONAUZAT!ON AND PARALLEL DIECOMPOS!TiON

In a bit-level trellis diagram, every time instant in the encoding interval r = {O, 1, 2, ... , n} is a section boundary location, and every branch represents a code bit. It is possible to sectionalize a bit-level trellis with section boundary locations at selected instants in r. This sectionalization results in a trellis in which a branch may represent multiple code bits, and two adjacent states may be connected by multiple branches. Proper sectionalization may result in useful trellis structural properties and allow us to devise efficient trellis-based decoding algorithms or to simplify decoder implementation. For a positive integer v ::: n, let le,

A= {to, ti, t2, .... t")

be a subset of v + 1 time instants in the encoding interval r = {O, 1, 2, ... , n} for an (n, k) linear block code C with O = to < ti < t2 < · · · < fv = n. A v-section trellis diagram for C with section boundaries at the locations (time instants) in A, denoted by T(A), can be obtained from then-section trellis T by (1) deleting every state in I:, ( C) for t E {O, 1, ... , n} \ A and every branch entering or leaving a deleted state, and (2) for 1 ::: j ::: v, connecting a states E I:,i-t to a states' E I:,i by a branch with label a if and only if there is a path with label a from state s to state s' in the 11-section trellis T. In this v-section trellis, a branch connecting a state s E I:,J-t to a states' E I: 11 represents (ti - fJ-1) code symbols. The state space I:,i-t (C) at time-tj-1, the state space ~ 1 /C) at time-ti, and all the branches between states in I: 1H (C) and states in 1\ (C), form the jth section of T(A). The length of this trellis

Section 9.7

Trellis Sectionalization and Parallel Decomposition

375

section is t.i - tJ- l · If the lengths of all the sections are the same, T (A) is said to be uniformly sectionalized. The state space dimension for this sectionalized trellis is an ( v + 1)-tuple, (po, Pr 1 , Pr, . · · · , Pr,,_ 1 , Pn),

where Pr;

= log2 I'Er1 ( C) I =k-

(9.48)

k(Co.ri) - k(C1i.11),

The maximum state space dimension of the v-section trellis T(A) is given !::,

Pv.max(C)

= max

Oso::jso::v

Pt 1 .

(9.49)

H the section boundary locations to, t1, ... , tv are chosen at the places where p 11 , Pt" . . . . , Pt,._ 1 are small, then the resultant v-section code trellis T (A) has a small state space complexity; however, sectionalization, in general, results in an increase in branch complexity. Hence, it is important to properly choose the section boundary locations to provide a good trade-off between state and branch complexities.

Consider the 8-section bit-level trellis T for the (8. 4) RM code shown in Figure 9.6. Suppose we choose v = 4 and the section boundary set A = (0. 2. 4, 6. 8}. Following the foregoing rules of sectionalization, we obtain a uniform 4-section trellis diagram as shown in Figure 9.17, in which every branch represents 2 code bits. The state space dimension profile for this 4-section trellis is (0, 2. 2. 2, 0), and the maximum state space dimension is P4,maxCC) = 2. The trellis still possesses mirror-image symmetry. Furthermore, the sectionalization decomposes the trellis into two parallel and structurally identical (or isomorphic) subtrellises without cross connections between them.

Consider the 16-section trellis for the (16, 11) RM code shown in Figure 9.7. Suppose we choose v = 4 and the section boundary set A = {O, 4. 8. 12. 16). The result is a

FIGURE 9.17: A 4-section trellis for the (8. 4) RM code.

316

Chapter 9

Trellises for Linear Block Codes 4 states

4 states

4 states

Two parallel branches

FIGURE 9.18: A 4-section minimal trellis diagram T({O, 4, 8, 12, 16}) for the (16, 11)

RM code. 4-section trellis as shown in Figure 9.18. Each section is 4 bits long. The state space dimension profile for this 4-section trellis is (0, 3, 3, 3, 0), and the maximum state space dimension is P4.max ( C) = 3. The trellis consists of two parallel and structurally identical subtrellises without cross-connections between them. A v-section trellis obtained from a minimal n-section trellis by deleting states at places other than the section boundary locations is minimal. A minimal code trellis has the least overall state and branch complexity but is, in general, densely connected. For long codes with large dimensions, it is very difficult to implement any trellis-based MLD algorithm based on a full code trellis on (an) integrated circuit (IC) chip(s) for hardware implementation. To overcome this implementation difficulty, one possible approach is to decompose the minimal trellis diagram of a code into parallel and structurally identical subtrellises of smaller dimensions without cross connections between them so that each subtrellis with reasonable complexity can be put on a single IC chip of reasonable size. This approach is called parallel decomposition. Parallel decomposition should be done in such a way that the maximum state space dimension of the minimal trellis of a code is not exceeded. Parallel decomposition has other advantages. Because all the subtrellises are structurally identical, we can devise identical decoders of much smaller complexity to process these subtrellises in parallel. This not only simplifies the decoding complexity hut also speeds up the decoding process. Parallel decomposition also resolves wire

Section 9.

Trellis Sectionalization and Parallel

37/7J

routing problems in IC implementation and reduces internal communications. Wire is a in IC of a trellis-based decodec Consider the minimal n -section trellis T for a (n, k) linear block code C with maximum state space dimension Pmax(C)a .,We are interested in decomposing T as a union of a certain desired number of and identical subtrellises under the constraint that its state space dimension at every time-i for 0 :S i :s n is less than or to Pmax(C); that we want to T without ex(;eecur1g its maximum state a row 11; from the TOGM we choose a subcode of C GqocM of C The matrix CG1 for this subcode is CG1 = CGrncM \ {!);}. Let and denote the dimensions of C and , respectively. Then, dim( C 1) = -- 1 = k - 1. The partition C / C 1 consists of two cosets, C 1 and its coset I); EB C1 . The two coset representatives are by {ll;}, which are !llJ and g. Let T1 be the minimal trellis for C1. Recall that the state space dimension ,o;(C) is to the nmnber of rows of CG whose active time spans contain the time index i. we see that ,o;(C1) = ,o;(C) -1 (9.50) for i

E Ya

and ,o;(Ci)

= ,o;(C)

(9.51)

for i eJ_ Ya(!);). The of (9.50) and (9.51) give the state space dimension profile of the subcode C1. VVe can obtain the minimal trellis T; for the cos et 11; EB C1 adding 11; to every path in T1. Therefore, T1 and are structurally identical. The union of T1 and denoted by T1 U gives a trellis representation for C in which T1 and form two subtrellises without cross connections between them. We may regard that the minimal trellis T for C is decomposed into two parallel and structurally identical subtrellises T1 and Note that T1 U may not be a minimal trellis for C. We define the index set

r;

r;,

r;,

r;

r;.

lmax(C) ~ {i: ,O;(C)

= ,Omax(C), for O :Si

:S 11},

(9.52)

which is simply the set of time instants (or boundary locations) at which the state space dimensions of the minimal trellis T for Care equal to its maximum, ,Omax(C). Then, Theorem 9.1 follows from (9.50), (9.51), and (9.52) [7, 37, 38].

«JU_ U there exists a row g in the TOGM CGToGM for an (n, k) linear code C such that Ya(!);) 2 lmax(C), then the subcode C1 of C generated by CGToGM \ {g} has a minimal trellis T1 with maximum state space dimension PmaxCC1) = ,Omax(C) -1, and

1rJHiiEOiRiEM

lmax(C1) = lmax(C) U {i: ,O;(C)

= ,Omax(C) - Li 'f-

Ya(!);)}.

(9.53)

Because CG is in TOF, CG1 = CG\ {g} is also in TOF. H the condition of Theorem 9.1 holds, then it follows from the foregoing that it is possible to construct a trellis for C that consists of two parallel and structurally identical subtrellises, one for C 1 and the other for its coset C 1 EB g. Each subtrellis is a minimal trellis and

378

Chapter 9

Trellises for Linear Block Codes

has maximum state space dimension equal to Pmax(C) -1. Therefore, the maximum state space dimension of the resultant trellis is still Pmax(C). EXAMPLE 9.19

Consider the (8, 4) RM code with TOGM

i ~ i ~ ~ ~ ~]

0 0

1 0

1 0

1 1

1 1

0 1

0 1

.

Its minimal 8-section trellis Tis shown in Figure 9.3. Its state space dimension profile is (0, 1, 2, 3, 2, 3, 2, 1, 0) and Pmax(C) = 3. The index set lmax(C) is lmax(C) = {3, 5}. By examining GToGM, we find only the second row g1, whose active time span, ra(gi) = [2, 6], contains lmax(C) = {3, 5}. Suppose we remove g1 from GmGM· The resulting matrix is G1

1 1 1 1 0 0 1 1 [ 0 0 0 0

=

0 0 0 1 1 0 1 1 1

0 ] 0 , 1

which generates an (8, 3) subcode C1 of the (8, 4) RM code C. From G1 we can construct a minimal 8-section trellis T1 for C1, as shown in Figure 9.19 (the upper subtrellis). The state space dimension profile of T1 is (0, 1, 1, 2, 1, 2, 1, 1, 0) and Pmax(C) = 2. Adding g1 = (01011010) to every path in T1, we obtain the minimal 8-section trellis for the coset g1 E9 C1, as shown in Figure 9.19 (the lower subtrellis ). The trellises T1 and form a parallel decomposition of the minimal 8-section trellis T for the (8, 4) RM code. We see that the state space dimension profile of T1 is (0, 2, 2, 3, 2, 3, 2, 2, 0). Clearly, T1 U is not a minimal 8-section trellis for the (8, 4) RM code: however, its maximum state space dimension is still Pmax(C) = 3. If we sectionalize T1 U at locations A = {O, 2, 4, 6, 8), we obtain the minimal 4-section trellis for the code, as shown in Figure 9.17.

r;

r;

r;

Ur;

r;

We can apply Theorem 9.1 to the subcode C1 of C if there exists a row g' E G 1 such that Ta (g') 2 lmax (C1). Then, G2 = G1 \ {g'} = GmcM \ {g, g'} generates a subcode C2 of Ci with dim(C2) = dim(C1) - 1 = k 2. The maximum state space dimension of the minimal trellis for C2 is then Pmax(C2) = Pmax(C1)-l = Pmax(C)-2. As a result, the partition C / C2 decomposes the minimal trellis T for C into four parallel structurally identical subtrellises, one for each coset in C / C2. The maximum state space dimension of the decomposed trellis is still Pmax(C). Theorem 9.1 can be applied repeatedly until either the desired level of decomposition is achieved or no row in the generator matrix can be found to satisfy the condition in Theorem 9.1. A general theorem for parallel decomposition is given next [7, 37, 38]. THEOREM 9.2 Let GmcM be the TOGM of an (11, k) linear block code C over GF(2). Define the following subset of rows of GmcM:

R(C)

6

=

{g

E

GmcM: ra(g) 2 lmax(C)}.

(9.54)

Section 907

FIGURE 9019: Parallel RMcodeo

Trellis Sectionalization and Parallel

~-·""'"-"' of the minimal 8-section trellis for the (8.,

For any r with 1 :S r :S I I, there exists a subcode of of C such - r if and if there that Pmax(Cr) = Pmax(C) - r and exists a subset c; R (C) such that for every i with p;(C) > Pmax(Cr), there exist at least p;(C) - PmaxCCr) rows in whose active time spans contain i The subcode \ and the set of coset for C / Cr is O

H the condition of Theorem 9.2 the C / Cr results in a decomposition of the minimal trellis T for C that does not exceed the maximum state complexity Pmax(C) of To Two direct consequences of Theorem 9.2 are in Corollaries 9.2.1 and 9.2.20 In a minimal trellis for a linear block code C, the maximum number of parallel subtrellises one can obtain such that the total state space dimension at any time does not 21R(Cll" exceed Pmax(C) is upper bounded COJRl[J)JLJLAIRY 1t2JL

base-2 of the number of subtrellises in a minimal v-section trellis with section ~~·ewes.,~, A = (to, t1, 000, tv} for a binary (n, k) linear block code C is given by the number of rows in the TOGM GTOGM whose active time spans contain the section boundary locations, t1, t2, . 00, t,,~ 10

Consider the (8, 4) RM code given in Example whose minimal 4-section trellis diagram with section boundary locations in A = {O, 2, 4, 6, 8} is shown in Figure 90170

380

Chapter 9

Trellises for Linear Block Codes

By examining the TOGM GToGM of the code, we find only one row, the second row, g1 = (01011010), whose active time span, Ta (gi) = [2, 6], contains {2, 4, 6}. Therefore, the minimal 4-section trellis consists of two parallel isomorphic subtrellises. EXAMPLE 9.21

Consider the (16, 11) RM code given in Example 9.10. Suppose we sectionalize the bit-level trellis at locations in A = {O, 4, 8, 12, 16}. Examining the TOGM of the code, we find only row g3 = (000111101000100 0) whose active time span, ra(g 3 ) = [4, 12], contains {4, 8, 12}. Therefore, the 4-section trellis T({O, 4, 8, 12, 16}) consists of two parallel isomorphic subtrellises. Basically, Theorem 92 provides a method for decomposing a complex minimal trellis into subtrellises without exceeding the maximum state complexity. This decomposition of a trellis into parallel and structurally identical subtrellises of smaller state complexity without cross connections between them has significant advantages for IC implementation of a trellis-based decoding algorithm, such as the Viterbi algorithm to be discussed in Chapters 12 and 15. Identical decoders of much simpler complexity can be devised to process the subtrellises independently in parallel without internal communications (or information transfer) between them. Internal information transfer limits the decoding speed. Furthermore, the number of computations to be carried out per subtrellis is much smaller than that of a fully connected trellis. As a result, the parallel structure not only simplifies the decoding complexity but also speeds up the decoding process. 9.8

lOW-WIE!GHT SUBTRIElUSIES

Consider a binary (n, k) linear block code C with weight profile W = {0, w1, w2, ... , where w1 is the minimum weight and w1 < w2 < . . . < w 111 ::c n. Let T(A) be a v-section trellis for C with section boundary locations in A = {to = 0, t 1, t2, ... , t v- l, t v = n}. A subtrellis of T (A) that consists of the all-zero codeword 0 and all the codewords of C with weights up to Wk (including wk) is called the Wk-weight subtrellis of T(A), denoted by Tw" (0). This subtrellis is said to be centered at 0. For small k, say k = 1 or 2, this subtrellis is called a low-weight subtrellis. The wk-weight subtrellis Tw" (0) can be obtained by purging the full trellis T(A) by using the algorithm described in [7, 37, 39]. For two adjacent states sands', withs E :E 1;(C) ands' E :Er;+i (C), let L(s, s') denote the set of parallel branches connecting the states s and s'. Let lb E L (s, s') and w(lb) denote the Hamming weight of !IJ. Let a(L(s, s')) denote the minimum weight of the parallel branches in L(s, s\ For any states E I:1/C) with OS j s v, let W1 (s) denote the minimum of the weights of all paths from the initial state so to the state s. W1 (s) is called the minimum path weight to the state s. For every states in T(A), W1(s) can be computed recursively. Suppose that W1(s) is known for every state s E :Er; ( C) for O ::c i ::c j. Let s' E :E 1i+ 1 ( C) and F (s') denote the set of states in :E 1i (C) that are adjacent to s'. Then, w 111 },

W1(s')

= min {W1(s) + a(L(s, s'))}. sEF(s')

(9.55)

Section 9.8

Low-Weight Subtrellises

:Jlij1

The recursion begins with W1 (so) = 0. Once W1 (s) is determined for every state s E :E 11 (C), the states in :E 1J+i can be processed. The weight of (9.55) continues until W (s1)(= 0) is determined. For any states E :E 1/C) with O _:::: j _:::: v, let WF(s) denote the minimum of the weights of aH paths from state s to the final state Sf. WF (s) is cailed the minimum path weight from the state s. For every state s in can also be recursively from the final state Sf. Suppose WF (s) is known for every states E "£ 11 ( C) for j < i _:::: v. Lets' be a state in "£ 11 (C), and Jet G(s') denote the set of states in (C) that are adjacent to s'. Then, WF(s')

= min {WF(s)

+ a(L(s'. s))}.

(9.56)

sEG(s')

H follows from the definitions of W1 (s) and WF (s) that there is at least one in T(A) passing through states with weight W1 (s) + WF(s), and there is no path in T(A) passing through states with weight less than W1(s) + WF(s). Let the all-zero path I[]) in T(A) be represented by the state sequence (0)

SO

= SO

(0)

•

S11

(0) 'S12 , •.. ,

(0)

st,,

= Sf.

Then, VV1 (s}~l) + WF(s;~l) = 0 for O _:: : j _:::: v and for any states not on the all-zero path !fD, W1(s) + WF(s) > 0. The Wk-weight subtrellis (I[])) can be obtained by purging the full trellis T(A) as follows: E T (A), VV1 (s) + WF (s) > Wk, delete states and all branches entering and leaving s. Z" Lets and s' be any two adjacent states with s E 1:: 1I. ( C) and s' E "£ 1I,~i ( C) for 0 _:: : j < v. Let !bi E L(s, s'). H W1(s) + WF(s') + w(lbi) > Wk, delete branch !bi. ::it U as a result of applications of the preceding two purging steps, any state s( # so) has no incoming branches, then deletes and all its outgoing branches from T(A). H any states(# sr) has no outgoing branches, then deletes and all its incoming branches from T (A).

lL H for every states

The foregoing purging process continues until none of the three purging steps can be applied. The resultant subtrellis contains the following codewords of C: (1) all codewords of weights up to and including wk; and (2) possibly some codewords of weights greater than Wk that correspond to nonzero paths in T(A) that diverge from and remerge with the all-zero path QJl more than once. For each of these paths, the weight of a partial path between the adjacent diverging and remerging states, called side-loop, is w1c or less but not zero. The number of these paths is, in general, small and they can be removed by using the state-splitting technique [7, 40]. We call the described purged trellis the wk-weight subtrellis of T(A). Fork = 1, Tw 1 (!fD) contains (1) the all-zero codeword@; (2) all the codewords of minimum weight w1; and (3) those codewords of weights greater than w1 that correspond to nonzero paths that diverge from and remerge with the all-zero path more than once, For each of these nonzero paths, the weight of each side-loop is w1. These paths with weights greater than w1 can be removed by splitting the states (0) (0) s(O) S11 , S12 , .•. , 1,,_1

382

Chapter 9

Trellises for Linear Block Codes

on the all-zero path Oas follows: -(0) 1, Create v - 1 new states, denote d b y s-(0) 11 , s12 ,

••• ,

b s-(0) oundary 1,,_ 1 , one at each

location except to = 0 and fv = n; 2. For 1 :::=: j < v - 1, connect the states{~) to state { 0; 1 by a single branch with

the all-zero UJ+l - t1)-tuple label. Also connect s1\~~ 1 to the final states f by a single all-zero label; and 3. Delete every branch b with nonzero weight (i.e., w(b) f. 0) that merges into one of the states, s;I0 l, on the all-zero path from any state s, and create a new branch with the same label from s to state s?I 1). The foregoing state-splitting process ensures that there is no path in the new trellis that after merging with the all-zero path diverges from it again before terminating at the final state, s1. Consequently, the new trellis, denoted by Tmin (0) contains only the all-zero path O and the minimum weight codewords of C. Tinin (0) is called the minimum-weight subtrellis of C and is said to be centered at 0. The subtrellis that contains a nonzero codeword v of C and all the codewords that are at minimum distance from v, denoted by TminCv), can be obtained by adding v to all the codewords in Tmin (0). This subtrellis Tmin (v) is called the minimum-weight (or more accurately minimum distance) subtrellis of C centered at v. The minimum-weight trellis Tinin (0) is sparsely connected and generally has much smaller state and branch complexities than does the full trellis for the code. For example, the state space complexity profile for the 4-section minimumweight trellis for the (64, 42) RM code is (L 157, 157, 157, 1) as compared with (1, 1024, 1024, 1024, 1) for the 4-section full trellis of the code. For the same code, the 8-section minimum-weight subtrellis has state complexity profile (l,45,157, 717,157,717, 157,45,1) as compared with (1,128, 1024,8192, 1024,8192, 1024, 128, 1) for the 8-section full trellis of the code. For this code, the 8-section minimum-weight subtrellis has a total of 4524 branches as compared with a total of 278, 784 branches for the 8-section full trellis. 9.9

CARTESIAN PRODUCT

A trellis for a linear block code constructed from component codes by using a construction technique such as interleaving, product, or direct-sum can be constructed by taking Cartesian product of the trellises of the component codes. Consider the interleaved code CA = C 1 * C2 *··.*CA, which is constructed by interleaving A linear block codes, C1, C2, · · · , CA, of length n (see Section 4.8). An n-section trellis TA can be constructed by taking the Cartesian product of the n-section trellises of the component codes. For 1 :::=: j :::=: A, let T1 be an n-section trellis for Ci. For O :::=: i :::=: n, let ~i(CJ) denote the state space of T1 at time-i.

The Cartesian product of T1, T2, · · · , Ti., denoted by TA ~ T1 x T2 x · · · x T,i_; is constructed as follows:

Section 9.9

JL For O ::::

fi:: =

£Jo

3l~3l

:::: n, form the Cartesian (C'')

r,,

Cartesian Product

(C1) x ::Ei(C2) >< · · · >< {(s?\ s,c2l, · · ·, s?l):

(CJJ

sij) E

(CA) forms the state space of form the nodes of at level-i. 2) .. · s(A)) 1·11 A 0-tate (s(l) s( l

(9.57) j) for 1:::: j ::S ;\.}.

at time-i, i.e., the

m

0

L

j

in :Ei+l

[

0

s

[

if and

if

srj)

is

lO

s,(~)1 for 1 < j < A. Le[

fi:: l(s(j). sC+il1) denote the label of the branch that connects the state

/il to l 1 the state s/!!1 for 1 :::: j :::: A. 'Ne connect the state (s?l, s/2 , · · · , s?'l) E ::Ei (C') (2) (ic) 1 1s . la b e1e d , si+l' · · ·, si+l) E I:i+1CC A) a b··1anch t1at t he

1.1·

l

l

(/i. /2,

· · · , l)J.

a column of the array of (4.80).

This label is

The constructed Cartesian product T' = Ti >< T2 x · · · x is an 11-section trellis for the interleaved code = C1 * C2 * ···*CA in which each section is of length A.

Let C be the (3, form is

even

code whose

GmcM

1

matiix in trellis-oriented

1

=[ 0 1

The 3-section bit-level trellis T for this code can be constructed from GToc1111 and is shown in Figure the code is interleaved to a depth of A = 2. the interleaved code = C * C is a 4) linear code. The Cartesian product T >< T results in a 3-section trellis T 2 for , as shown in 9.20(b ). Let C1 be an (111. !q) linear code, and let C2 be an (112, k2) linear code. The product C1 >< C2 is then an (111112, /qk2) linear block code (see Section 4.7). To construct a trellis for the product code C1 >< C2, we regard the top k2 rows of the array shown in Figure 4.3 as an interleaved array with codewords from the same code Cl· VI/ e then construct an n 1-section trellis for the interleaved code, lo

C(

=~ C1 * C1 *. · · · *, C1. k2

using the Cartesian product. Each k2-tuple branch label in the trellis is encoded into a codeword in C2. The result is an 111-section trellis for the product C1 x C2, each section is 112 -bit in length.

IEXti.,MrLIE 9J.23l Let C1 and C2 both be the 2) even parity-check code. Then, the product C1 >< C2 is a (9, 4) linear code with a minimum distance of 4. Using the Cartesian product

384

Chapter 9

Trellises for Linear Block Codes

o > - - - - --0 - - - - - - < o 0

0

0

0

0

(a) A,B,C,D

A= (00).B = (01),C = (10).D = (11) (b)

FIGURE 9.20: ( a) The minimal 3-section bit-level trellis for the (3, 2) even parity-check code, (b) A 3-section trellis for the interleaved (3, 2) code of depth 2.

construction method given above, we first construct the 3-section trellis for the C1 * C1, which is shown in Figure 9.20(b). Then, we encode interleaved code each branch label in this trellis based on the C2 = (3, 2) code. The result is a 3-section trellis for the product code C1 x C2, as shown in Figure 9.21.

cf=

Let C1 and C2 be an (n, k1, d1) and an (n, k2, d2) binary linear block code with generator matrix, G1 and G2, respectively. Suppose C1 and C2 have only the all-zero codeword Oin common; that is, C1 n C2 = {O}. Their direct-sum, denoted by C1 EB C2, is defined as follows: (9.58)

Then, C = C1 EB C2 is an (n, k1 + k2) code with minimum distance dmin ::: min{d1, d2} and generator matrix

Let T1 and T2 be then-section trellis for C1 and C2, respectively. Then, an n-section trellis T for the direct-sum code C = C1 EB C2 can be constructed by taking the Cartesian product of Ti and T2. The formation of state spaces for T and the condition for state adjacency between states in T are the same as for forming the trellis for an

Section 9.9

Cartesian Product

:l~:5

A,B,C,D

A= (000), B = (011). C = (110). D = (101)

FIGURE 9.21: A 3-section trellis for the

taking the Cartesian interleaved code . codes. The difference 1s branch

(s/~ s/~ 1.

)

1

2).

of the trellises for the (l) states (s; .

+ 1),

in T at time-i and

2) )(

ui, s/t1) be the label of the s/t1 for 1 ::c j ::c 2. 'Ne connect

let li ~

srj)

branch that connects the state and the state the state (/l 1l. / l 2l) and the state (s(+ll ) with a branch that is labeled with l 1, s(~l 1,- 1

si~

2

l (s?l, si~1) + l (si l. known as the Shannon product.

!1

+ 12 =

1).

The described

of two trellises is also

iEXAMfllE '.9U:4 Let C 1 and C2 be two linear block codes of


=[

8

by

111100001 01011010

and

G 2 =[00111100] 00001111 ' H is easy to check that C1 n

G

=[

G1 IG2

J=

=

{@}. The direct-sum

is

r~ i ~ i ~ ~ ~ ~ 1 00111100. 00001111

which is simply the TOGM for the (8, 4, 4) RM code in Example 9.1. Both rG. 1 and G2 are in TOF. Based on IG1 and G2, we construct the 8-section treHises T1 and T2 for C1 and C2 as shown in Figures 9.22 and Taking the Shannon product of T1 and T2, we obtain an 8-section trellis T1 >< T2 for the direct-sum EB as shown in Figure 9.24, which is simply the 8-section minimal trellis for the (8, 4, 4) RM code as shown in Figure 9.6. We label the states in T1, and T1 >< the method of state-defining sets of Section 9.3.

386

Chapter 9

Trellises for Linear Block Codes

00

10

The 8-section minimum trellis for the code generated by 11110000] Gi = [ 0 1 0 1 1 0 1 0 . FIGURE 9.22:

00

The 8-section minimum trellis for 00111100] Gz= [ 00001111.

FIGURE 9.23:

the code generated by

0000

FIGURE 9.24:

The Shannon product of the trellises of Figures 9.22 and 9.23.

The Shannon product can be generalized to construct a trellis for a code that is a direct-sum of m linear block codes. For a positive integer m :::. 2, and 1 :s j :s m, let CJ be an ( N, K 1 , d1) linear code. Suppose C 1, C2, · · · , Ci 11 satisfy the following condition: for 1 :s j, j' :s m, and j =I j',

c 1 n cf' This condition simply implies that for v,,1 Wt

= {O}.

E C1

(9.59)

with 1 :s j

+ w2 + · · · + Wm

=

0

:s m,

Section 9.9

if and only if v1

Cartesian Product

3:87

= v2 = · · · = v = illl. The direct-sum of C1, C2, · · · , C is defined as 111

111

(9.60) E Ci,

1 S j S m}.

d) linear block code with

Then, C = C1 E8 C2 E8 · · · E8 C111 is an (N.

+ .. ·+ d

s

min {dj},

1~.i~J11

.

Let G.i be the generator matrix of Ci for 1 S j Sm. Then, C is generated by the following matrix:

= C1 E8 C2 E8 · · · E8 Cm

(9.61)

The construction of an n-section trellis T for the direct-sum C = C1 E8 C2 E8 .. · E8 C111 is the same as that for the direct-sum of two codes. Let (s?l. s?l, ... , s/ 111 l) and Cll si+l c2i · · · · , si+l (ml) b e two a ct·Jacent states m . T. Tl1en, t h e b ranc h. connectmg . ( si+l' (1) (2) (m) (1) (2) (ml . 1 b l d . h ( s; ,s; , ... ,s; )to(s;+l'si+l·'" ,s;+ 1 )1s a ee wit /( s,(1)

where for 1 S j

s

(1) )

's;+J

+ /( s,(2) . si+l (2) ) + ... + /( (m) (m)) s; , ·\+1 .

(9.62)

m, l (s/Jl, si~1 ) is the label of the branch that connects the state

sril and the state si~1 in the trellis Ti for the jth code Ci.

Again, we consider the (8, 4, 4) RM code generated by the

l

TOGM:

llllOOOOJ

G=

01011010 00111100 00001111

.

For 1 S j s 4, let C.i be the (8, L 4) code generated by the jth row of G. Then, the direct-sum, C1 E8 C2 E8 C3 E8 C4, gives the (8, 4, 4) RM code. The 8-section minimal trellises for the four component codes are shown in Figure 9.25. The Shannon products T1 x T2 and T3 x T4 generate the trellises for C1 E8 C2 and C3 E8 C4, respectively, as shown in Figures 9.22 and 9.23. The Shannon product (T1 x T2) x (T3 x T4) results in the overall trellis for the (8, 4, 4) RM code shown in Figure 9.24.

388

Chapter 9

Trellises for Linear Block Codes

0

0

0

0

0

(b) T,

0

0

0

0 0

(d) T,

FIGURE 9.25:

The 8-section minimal trellises for the four component codes.

Suppose T1, T2, · · · , T; 11 are minimal N -section trellises for the component codes C1, C2, · · · , C111 , respectively. The Shannon product T1 x T2 x · · · x T; 11 is not necessarily the minimal n-section trellis for the direct-sum C = C1 EB C2 EB··· EB C 111 • Let T denote the minimal n-section trellis for C, and Pi ( C) denote the state space dimension of T at time-i for O :s i :s n. Then, 111

p;(C)

:s

L p;(C;).

(9.63)

j=I

If the equality of (9.63) holds for O :S i :s n, then the Shannon product T1 x T2 x. · · x I;,, is the minimal n-section trellis for the direct-sum C = C1 EB C2 EB··· EB C111 • Theorem 9.3 gives a sufficient condition for the equality of (9.63). A proof of this theorem can be found in [7].

9.3 Consider the direct-sum C = C1 EB C2 EB · · · EB Cm. For 1 :S j :S m, let Ti be the minimal N -section trellis for the component code C j. Then, the Shannon product Tt x T2 x · · · x I;,, is the minimal N -section trellis for C if and only if the following condition holds: for O :s i :s N, 1 :S j, j' :s m, and j i- j', (9.64) po.;(Cj) n Po;(Ci') = {O}. THEOREM

Section 9.9

Cartesian Product

3g~

The Shannon product can be applied to sectionalized trellises. In this case. all the trellises must have the same number of and sections must have the same length.

we sectionalize each of the two 8-section trellises of Figures 9.22 and 9.23 each of length 2. The resultant 4-section trellises are shown in

00

10

00

FIGURE 9.27: A 4-section trellis for the (8. 4, 4) RM code.

390

Chapter 9

Trellises for Linear Block Codes

Figure 9.26, and the Shannon product of these two 4-section trellises gives a 4section trellis, as shown in Figure 9.27, which is the same as the 4-section trellis for the (8, 4, 4) RM code shown in Figure 9.17.

PROBLEMS 9.1 Consider the (6, 3) linear code generated by the following matrix: G=

9.2

9.3 9.41 9.5 9.6 9.7

101101] 110001 . [ 101010

a. Put this generator in trellis-oriented form. b. Determine the active time spans of the rows in the trellis-oriented generator matrix. c. Determine the state space dimension profile of the bit-level 6-section trellis for the code. ell. Determine the state-defining information set at each time instant. e. Determine the input information bit at each time instant. f. Determine the output function in each bit interval. Construct the trellis-oriented generator matrix for the first-order RM code, RM(l, 5), of length 32. a. Determine the active time spans of the rows. b. Determine the state space dimension profile of the bit-level trellis for the code. c. Determine the state-defining information set at each time instant. ell. Determine the input information bit at each time instant. e. Determine the output function in each bit interval. Construct the bit-level trellis for the (6, 3) code given in Problem 9.1. Label the states based on the state-defining information set using Pmax(C) bits. Find a parity-check matrix for the (6, 3) code given in Problem 9.1. Label the states of its bit-level trellis based on the parity-check matrix. Construct the bit-level minimal trellis for the (8, 7) even-parity-check code. Construct the bit-level trellis for the first-order RM code, RM(l, 5), of length 32. Label its states based on the state-defining information set using Pmax(C) bits. Determine the past and future subcodes of the (6, 3) linear code given in Problem 9.1 at each time instant. Determine the cosets in the partition C / Co,4 EB c4.6·

9.8 Determine the past and future subcodes of the first-order RM code, RM(l, 4), of length 16 at time instants 4, 8, and 12. Determine the cosets in the partition C/Co.8 EB Cs,16·

9.9 For the first-order RM code of length 16, determine the punctured code P4.s(C) and punctured code 8 between time-4 and time-8. Determine the partition

cr

P4.s(C)/C;{s·

9.10 Determine the state space dimension profile of the bit-level trellis for the primitive (15,5) BCH code. Construct its bit-level trellis. 9.11 Consider the first-order RM code of length 16 given in Example 9.13. Construct a 4-section trellis for the code with section boundary locations at 0, 4, 8, 12, and 16.

!U.Z Continue Problem 9.5. Construct a 4-section trellis for the first-order RJVf code of length 32. (IJ)J.:§ Consider the first-order RM code of length 16 given in Example 9.13. the bit-level trellis into two parallel subtrellises without the maximum state space dimension. (IJ)J14J Continue Problem 9.13. After construct an 8-section trellis for the code. ([]>o]_§ Can the first-order RM code oflength 16 be into 4 parallel subtrellises without exceeding its maximum state space dimension? (IJ),Jli/ii Can the bit-level trellis for the 5) BClJL code be into two parallel subtrellises without exceeding its maximum state space dimension? If yes, decompose the trellis. (IJ),Jl7 Prove that the bit-level trellis for the first-order RM code of length 16 has mirror-image c·rn~n,µ, (IJ),Jl~ Prove that the bit-level trellis for the first-order RM code, RNj[(r. has mirrorimage symmetry.

1

]_, G.D. Forney, Jr., "The Viterbi

"'Proc. IEEE, 61: 268-78, 1973.

Z, A. J. Viterbi, "Error Bounds for Convolution.al Codes and Optimum Decoding Algorithm," IEEE Trans. IT-13: 260-69, 1967. ], L. R. Bahl, J. Cocke, F. Jelinek, and J.

Codes for 284-87, 1974.

Symbol Error

IT-20:

4l, J. K. Wolf, " Efficient Maximum-Likelihood of Linear Block Codes Using a "IEEE Trans. IT-24: 76-80, 1978. §,

J. L. Massey, "Foundation and Methods of Channel ~··~~·~'"'·"•",, " in Proc. Int. Inform. Theory and

6, G.D. Forney "Coset Codes II: Binary Lattices and Related Codes," IEEE Trans. Inform. Theory, IT-34: 1152-87, 1988.

'1, S. T. Kasami, T. Fujiwara, and M. P. C. Fossorier, Trellises and Trellis-Based Decoding Algorithms for Linear Block Kluwer Academic, Mass., 1998.

3, A. Vardy, "Trellis Structure of Codes" in Handbook of Coding V. Pless, W. Huffman, and R. A. Elsevier Science, 1998. gJ,

D. J. Muder, "Minimal Trellises for Block Codes," IEEE Trans. 34: 1049-53, 1988.

liilll, Y. Berger and Y. Be'ery, "Bounds on the Trellis Size of Linear Block IEEE Trans. Inform. Theory, IT-39: 203-9, 1993.

392

n.

Chapter 9

Trellises for Linear Block Codes

T. Kasami, T. Takata, T. Fujiwara, and S. Lin, "On the Optimum Bit Orders with Respect to the State Complexity of Trellis Diagrams for Binary Linear Codes," IEEE Trans. Inform. Theory, IT-39: 242-43, January 1993.

Jl.2. T. Kasami, T. Takata, T. Fujiwara, and S. Lin, "On Complexity of Trellis Structure of Linear Block Codes," IEEE Trans. Inform. Theory, IT-39: 1057-64, May 1993.

B. T. Kasami, T. Takata, T. Fujiwara, and S. Lin, "On Structural Complexity of the L-section Minimal Trellis Diagrams for Binary Linear Block Codes," IE/CE Trans. Fundamentals, E76-A (9): 1411-21, September 1993. Jl.4. G.D. Fomey,Jr., andM. D. Trott, ''The Dynamics of Group Codes: State Spaces, Trellis Diagrams and Canonical Encoders," IEEE Trans. Inform. Theory, IT-39: 1491-513, 1993. Jl.5. T. Kasami, T. Fujiwara, Y. Desaki, and S. Lin, "On Branch Labels of Parallel Components of the L-Section Minimal Trellis Diagrams for Binary Linear Block Codes," IE/CE Trans. Fundamentals, E77-A (6): 1058-68, June 1994. Jl.16. G.D. Forney, Jr., "Dimension/Length Profiles and Trellis Complexity of Linear Block Codes," IEEE Trans. Inform. Theory, IT-40: 1741-52, 1994. Jl.7. S. Dolinar, L. Ekroot, A. B. Kiely, R. J. McEliece, and W. Lin, "The Permutation Trellis Complexity of Linear Block Codes," in Proc. 32nd Allerton Conj on Comm., Control, and Computing, 60-74, Monticello, Ill. September 1994. Jl.8. F. R. Kschischang, "The Combinatorics of Block Code Trellises," in Proc. Biennial Symp. on Commun., Queen's University, Kingston, Ont., Canada, May 1994. 19. F. R. Kschischang and G. B. Horn, "A Heuristic for Ordering a Linear Block Code to Minimize Trellis State Complexity," in Proc. 32nd Allerton Conj on Comm., Control, and Computing, 75-84, Monticello, rn. September 1994. 20. H. T. Moorthy and S. Lin, "On the Labeling of Minimal Trellises for Linear Block Codes," in Proc. Int. Symp. Inform. Theory and Its Applications, Sydney, Australia, 1: 33-38, November 1994. 21. A. Lafourcade and A. Vardy, "Asymptotically Good Codes Have Infinite Trellis Complexity," IEEE Trans. Inform. Theory, 41: 555-59, 1995. 22. 0. Ytrehus, "On the Trellis Complexity of Certain Binary Linear Block Codes," IEEE Trans. Inform. Theory, 40: 559-60, 1995. 23. Y. Berger and Y. Be'ery, "Trellis-Oriented Decomposition and Trellis Complexity of Composite Length Cyclic Codes," IEEE Trans. Inform. Theory, 41: 1185-91, 1995. 24. F. R. Kschischang and V. Sorokine, "On the Trellis Structure of Block Cod est IEEE Trans. Inform. Theory, 41: 1924-37, 1995.

Bibliography 1,§,

A. Lafourcade and A. Vardy, "Lower Bounds on Trellis Codes," IEEE Trans. Inform. Theory, 41: 1938-54, 1995.

3'.')13

of Block

C. Lu and S. H. Huang, "On Bit-Level Trellis Complexity of Reed-Muller Codes," IEEE Trans. Inform. Theory, 41: 2061-64, 1995.

1,11\i, C.

Z'l, A. Lafourcade and A. Vardy, "Optimal Sectionalization of a Trellis," IEEE Trans. Inform. Theory, 42: 689-703, 1996. Z:tt R. J. McEliece, "On the BCJR TreHis for Linear Block Codes," IEEE Trans. Inform. Theory, 42: 1072-92, 1996. 1,
M. P. C. Fossorier and S. Lin, "Coset Codes Viewed as Terminated Convolutional Codes," IEEE Trans. Comm., 44: 1096-106, September 1996.

3Gt A. B. Kiely, S. Dolinar, R. J. McEliece, L. Ekroot, and W. Lin, "Trellis Decoding

Complexity of Linear Block Codes," IEEE Trans. Inform. Theory, 42: 1687-97, 1996. 3Jl, Y. Berger and Y. Be'ery, "The Twisted Squaring Construction, Trellis Complexity and Generalized Weights ofBCH and QR Codes," IEEE Trans. Inform. Theory, 42: 1817-27, 1996. 32, F. R. Kschischang, "The Trellis Structure of Maximal Fixed-Cost Codes," IEEE Trans. Inform. Theory, 42: 1828-38, 1996. 33, A. Vardy and F. R. Kschischang, "Proof of a Conjecture of McEliece Regarding the Expansion Index of the Minimal Trellis," IEEE Trans. Inform. Theory, 42: 2027-33, 1996. 34L V. R. Sidorenko, G. Markarian, and B. Honary, "Minimal Trellis Design for

Linear Block Codes Based on the Shannon Product," IEEE Trans. Inform. Theory,42:2048-53, 1996.

3:t G. B. Hom and F. R. Kschischang, "On the Intractability of Permuting a Block Code to Minimize Trellis Complexity," IEEE Trans. Inform. Theory, 42: 2042-48, 1996. 36, T. Komura, M. Oka, T. Fujiwara, T. Onoye, T. Kasami, and S. Lin, "VLSI Architecture of a Recursive Maximum Likelihood Decoding Algorithm for a (64, 35) Subcode of the (64, 42) Reed-Muller Code," Proc. Int. Symp. Inform. Theory and Its Applications, Victoria, Canada, 709-12, September 1996. 37, H. T. Moorthy, "Efficient Near-Optimum Decoding Algorithms and Trellis Structure for Linear Block Codes," Ph.D. diss. University of Hawaii at Manoa, November 1996. 3lt H. T. Moorthy, S. Lin, and G. T. Uehara, "Good Trellises for IC Implementation

of Viterbi Decoders for Linear Block Codes," IEEE Trans. Comm., 45: 52-63, 1997.

394

Chapter 9

Trellises for Linear Block Codes

390 H. T. Moorthy, S. Lin and T. Kasami, "Soft-Decision Decoding of Binary Linear Block Codes Based on an Iterative Search Algorithm," IEEE Trans. Inform. Theory, 43: 1030-40, May 1997.

40. T. Kasami, T. Koumoto, T. Fujiwara, H. Yamamoto, Y. Desaki, and S. Lin, "Low Weight Subtrellises for Binary Linear Block Codes and Their Applications," IE/CE Trans. Fundamentals, E80-A (11): 2095-103, November 1997.

410 B. Honary and G. Markarian, Trellis Decoding of Block Codes: A Practical Approach, Kluwer Academic, Boston, Mass., 1997. 42. R. H. Morelos-Zaragoza, T. Fujiwara, T. Kasami, and S. Lin, "Constructions of Generalized Concatenated Codes and Their Trellis-Based Decoding Complexity," IEEE Trans. Inform. Theory, 45; 725-731, March 1999.

~®~ D@ ~o ~ o~Sf = IBs@~®©~ s;@iftt= [Q)®G:o~o@ITTJ [Q) ®«::@ (Q~ Duu ~ n

ffe~ ~~@[FD it ~'D uuu ~

c~@

u uJ [(1 ®@J u

r:r ~,

r5) !(;\ (? 111: r;::,fl @rc" LQ) U"'':dJ \~ LJ\., 1'::c::, \\)J !~:JJ ,:c::;, ;;zi

All the developed so far are based on hard-decision of the matched filter in the receiver demodulator; that the of the matched filter for each interval is in tvm denoted as Oand 1, which results in a hard-decision received sequence. the decoder processes this hard-decision received sequence based on a specific decoding method. This of decoding is referred to as hard-decision decoding. Hard-decision using algebraic structures of the codes are called algebraic The metric used in hard--decision is the Hamming distance. The is to decode the hard-decision received sequence to the closest codeword in the Hamming distance. results in a loss of which degrades A hard decision of a received of the matched filter are or quantized in more than two levels, we say that the demodulator makes soft decisions. A sequence of soft-decision of the matched filter is referred to as a soft-decision received sequence. Decoding by processing this soft-decision received sequence is called Because the decoder uses the additional information multilevel quantized) received samples to recover soft-decision decoding provides better error performance than hard-decision fo general, soft-decision maximum likelihood decoding (MLD) of a code has about 3 dB of coding gain over algebraic decoding of the code; soft-decision decoding is much harder to implement than algebraic decoding and requires more computational complexity. This is the to be for better error performance. Many soft-decision have been devised. These decoding algorithms can be classified into two major categories: reliability-based ( or probabilistic) decoding algorithms and code structure-based algorithms. Important reliability-based decoding algorithms are presented in ihis chapter. The other category of decoding algorithms will be covered in later chapters.

For soft-decision decoding, metrics other than the Hamming distance must be used. The most commonly used metrics are the likelihood function, Euclidean distance,

396

Chapter 1O

Reliability-Based Soft-Decision Decoding Algorithms

correlation, and correlation discrepancy. In this section we develop these metrics and discuss the error performance of the soft-decision MLD. Let C be a binary (n, k) linear block code with minimum Hamming distance dmin(C) that is used for error control. Suppose BPSK signaling is used for transmission over an A WGN channel with two-sided PSD No/2. Assume that each signal has unit energy. Let v = (vo, v1, · · · , v11 _1) be a codeword in C. For transmission, this codeword is mapped into a sequence of BPSK signals that, in vector form, represented by the bipolar sequence c = (co, c1, · · · , c 11 _1), where for O ~ l < n,

-1

C/

= 2v1 - 1 = { +l

for for

V/ V/

= 0, = 1.

(10.1)

Let c; = (c;o, c;i, · · · , c;,n-1) and CJ = (cjo, CJl, · · · , Cj,n-1) be two signal sequences corresponding to codewords v; = (v;o, v;1, · · · , v;, 11 -1) and VJ = (v 1o, v11, · · · , v1,11 _1), respectively. The squared Euclidean distance between c; and ci is defined as 11-l

[C; - Cj[

2 6 ~ = L_,(Cii - Cj/) 2 .

(10.2)

1=0

For convenience, we denote this squared Euclidean distance by if;;("v;, vi). It follows from (10.1) and (10.2) that 11-l

dy::(v;,

Vj) =

(10.3)

4 L(vu - v11>2. 1=0

Let dH (v;, vi) denote the Hamming distance between v; and vi. It follows from (10.3) that we have the following relationship between the squared Euclidean distance and the Hamming distance between v; and vi: (10.4)

The minimum squared Euclidean distance of code C is then defined as follows: dtmin(C)

~ min{di,(v;. Vj):

V;, Vj EC

and

V;

"I- Vj}-

(10.5)

From (10.4) and (10.5), we readily see that 4,min(C) = 4dmin(C).

(10.6)

Consider a soft-decision MLD with the log-likelihood function as the decoding metric. Let r = (ro, r 1 , · · · , r 11 _ 1) be the soft-decision received sequence. The log-likelihood function of r given a codeword vis (defined in (1.10)) n-1

log P(r[v) =

L log P(r; [v; ).

(10.7)

i=O

With the log-likelihood function as the decoding metric, MLD is carried out as follows: the received sequence r is decoded into a codeword v = (vo, v,, · · · , v11 _ , ) for which the log-likelihood function log P(r[v) given in (10.7) is maximized.

Section 10.1

Soft-Decision Decoding

3l9J7

For an A WGN channel with two-sided PSD No/2, the conditional probability P (li'lw) is given by

P(Irlw)

=

=

1

exp

(nNo)- 11 12

1

exp

(n No)-n/2

I I

n-1

- V(r;

(2v, - 1))

~

2

l

1=0 11-l

/

No

J (10.8)

-V(r-c)2/No. ~ 1=0

'

i

The sum I::;'~J(r; - c;) 2 is the squared Euclidean distance between the received sequence Ir and the code signal sequence (co, ci, · · · , c11 _1). We denote this distance with 4(Ir, te). From (10.8) we see that maximizing P(Irlw) (or log P(Irlw)) is equivalent to minimizing the squared Euclidean distance dl(rr, te). Consequently, soft-decision MLD can be carried out with the squared Euclidean distance as the decoding metric as follows: the received sequence Ir is decoded into a codeword w for which the squared Euclidean distance 4(rr, te) is minimized. Consider the squared Euclidean distance 11-l

dl(Ir, te)

=

L(r; - c;)2.

(10.9)

i=O

Expanding the right-hand side of (10.9), we have n-l

di('J,
11-l

= I:1}+n

-2I:r, ·c;.

i=O

(10.10)

i=O

r;

In computing dl('J, <1:) for all codewords in MLD, we see that I::;'~J is a common term, and n is a constant. From (10.10) we readily see that minimizing cli('J, <1:) is equivalent to maximizing n-1

m(rr, w)

= m('l, ic) ~

Lr;· c;.

(10.11)

i=O

The preceding sum is called the correlation between the received sequence If and the code sequence \C. Therefore soft-decision MLD can be carried out in terms of the correlation metric as follows: Ir is decoded into a codeword w for which m(Ir, \C) is maximized. Finally, we can rewrite (10.11) as 11-l

m('l, c) =

I: lr;I -2 L

Let A(Ir, w)

= A(lf,

lr;I.

(10.12)

i:r,·c;
i=O

c)

~

L i:ri-ci

<0

Ir, 1-

(10.13)

398

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

Then, 11-l

m(ir, c)

=

L, Ir; I -

2>..(ir, c).

(10.14)

i=O

I:; ~c5

Therefore, maximizing m(ir, c) is equivalent to minimizing J..(ir, c). Because 1 Ir; I is the largest correlation that can be attained for a given ir, we call J..(ir, c) the correlation discrepancy between ir and c. Soft-decision MLD can be carried out in terms of the discrepancy metric as follows: ir is decoded into a codeword v for which J..(ir, c) is minimized. Let R and {Ao, A1, · · · , A11 } be the rate and weight distribution of code C, respectively. Then, with soft-decision MLD, the probability of decoding a received sequence incorrectly, called block ( or word) error probability, is upper bounded as follows [1, p. 440]: 11

Ps

:::=:

L A;Q ( J2iREb/No),

(10.15)

i=l

where E1, is the received energy per information bit, and Q(a) = _l_

100

e_x2;2

dx

v2Jrrx

(10.16)

is called the Q-function. This upper bound is known as the union bound. If the weight distribution of C is known, we can evaluate this bound. Because the minimum Hamming distance of C is dmin, A1 = A2 = · · · = Adm;,,-1 = 0. Since Q(a) decreases exponentially with a, Q(J2dm;11 REb/No) is the dominant term in the summation of (10.15). Therefore, for large SNR, Eb/ No, the first term in the summation of (10.15), (10.17) gives a good approximation of the upper bound. Adm;,, is commonly called the error coefficient. For large Eb/ No, the union bound given by (10.15) is quite tight. For 1 :::=: i :::=: n, let B; denote the average number of nonzero information bits associated with a codeword of weight i. Then, the bit-error probability ( the probability that an information bit is decoded incorrectly) is upper bounded by 1

Pb:::=:

11

kL

A;B;Q ( J2iREb/No).

(10.18)

i=l

Soft-decision MLD achieves optimum error performance; however, it is very hard to implement and requires large computational complexity, especially for long codes. A brute-force method requires 2k computations of metrics for 2k codewords, and 21c - 1 comparisons to find the most likely codeword. For large k, the number of computations and comparisons is simply too large for practical implementation. Several soft-decision MLD algorithms have been devised. They are very efficient for codes of short to moderate lengths. For long codes, they still require very large computational and storage complexities. To overcome this complexity problem, various nonoptimum or suboptimum soft-decision decoding algorithms have been

Soft-Decision

can and known [3, 4], Nhich to achieve MLD with a lS It was first devised for decodhovvever since block codes also have trellis structure ( as shown in the Viterbi to these codes as well. Besides the VA, there are other trellis-based for both block and convolutional codes. Consider a MLD based on Euclidean distance between the received sequence iI and a codeword in C. H follows from (10.6) that if the received sequence li' falls inside a of radius p;;;;; centered at the transmitted code sequence ,r: in the 11-dimensional Euclidean space, then decoding is correct. A soft-decision that correct decoding for all received sequences located inside the hyperspheres of radius .Jdmin centered at each of the code sequences is called a bounded distance decoding algorithm. Bounded distance been regarded as a criterion for designing good because its asymptotic error performance is the same as that of MLD., as deduced from (10.17); however, recent results tend to indicate that bounded distance well for codes with a srnall always the case for more the error coefficients. is well c.~n,,-,r,~, based on ~UHHHHur,

1

(10.19)

~-

k

If encoding is realized in information bits are determined encoder. In this case, (10.19) becomes P1,

~ dmin ---;;-At1111 ; 11

we can choose B; / k based on the

(-.t

)

Q2dmi11REb/No.;

~

i / n, since form of the

(10.20)

however, this is no the case for other types of encodings, such as encoding based on the trellis-oriented generator matrix i / n in general [2]. it follows from (10.20) that for large Eb/ No, encoding in nonsystematic form becomes suboptimum with respect to encoding in systematic form for the bit-error associated with MLD. This problem can easily be overcome based on the fact that a maximum likelihood decoder provides the most likely (ML) codeword out of the candidate codewords of a codebook, independent of the mapping used between information bits and codewords. As a result, the decoder can be based on a particular generator matrix
400

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

only into the matrix G' = GREF, so that GREF contains the k columns of an identity matrix, but not necessarily in the first k positions. This matrix G,rrnF is said to be in reduced echelon form (REF). If GREF is used for encoding, the information sequence is easily retrieved from the codeword delivered by the decoder based on the knowledge of the positions of the k columns of the identity matrix in GREF· An example that illustrates this simple operation will be given in Section 10.5.

10.2

REUABIUTY MEASURES AND GENER.Al RIEUAB!UTY-IBASED DECODING SCHEMES Let r = (ro, r1, · · · , r11 _1) be a soft-decision received sequence at the output of the matched filter of the receiver. For O ::: i ::: n - 1, suppose each received symbol r; is decoded independently based on the following hard-decision rule: ZI. --

{

01

for for

r; < r; ::::

0, 0.

(10.21)

Then, the magnitude Ir; I of r; can be used as a reliability measure of the hard-decision decoded bit z;, since the magnitude of the log-likelihood ratio I log(P(r; Iv;

= 1)/ P(r;lv; = 0))1

associated with the foregoing hard decision is proportional to Ir; I, where v; is the ith transmitted bit. The larger the Ir; I, the more reliable the hard decision z; is. Based on the given reliability measure, we can reorder the symbols of the received sequencer in decreasing order of reliability. As a result of this ordering, the received symbols and their corresponding hard decisions at the left-side positions of the reordered received sequence are more reliable than the received symbols and their corresponding hard decisions at the right-side positions of the reordered received sequence. With hard-decision, an error is more likely to be committed at a less reliable position than at a more reliable position. Figure 10.1 shows the number of errors occurring at each position after reordering 500,000 received sequences with 50 symbols for each sequence, in decreasing order of reliability for various values of SNR. From this figure we see that few hard-decision errors are recorded in the more reliable positions of the ordered sequences, whereas the number of harddecision errors increases exponentially in the less reliable positions. Based on this error distribution phenomenon, soft-decision decoding algorithms using reliability measures of the received symbols have been devised. These decoding algorithms are thus known as reliability-based decoding algorithms. Based on the reliability measures of the received symbols, the symbol positions of a received sequence can be divided into two groups according to a certain criterion: one group consists of the least reliable positions (LRPs), and the other group consists of the most reliable positions (MRPs). Then, decoding can be devised based on either processing of the LRPs or processing of the MRPs. Let z = (zo, z1, · · · , z11 _1) be the hard-decision received sequence obtained from r. Then, errors are more likely to occur in the LRPs, as shown in Figure 10.l, and the MRPs are likely to contain very few or no errors. The errors in the LRPs can be reduced or removed by modifying the hard-decision received vector in these positions. Let E be a set of error patterns with errors confined only in the LRPs.

Section 10.2

Reliability Measures

-e-

4(0)1

:SNR=OdB

__,_ :SNR=2dB _,_ :SNR=4dB 2

"'0

- : ~ :SNR=6dB

l-,

1--<

1.5

l-,

Q) "-,

0 h Q)

..0

s

;:;

z

0.5

0
30

35

40

45

50

Position in ordering

FIGURE 10.1: Number of errors at each ordered position for 500,000 BPSK sequences of length 50 with SNR values of OdB, 2 dB, 4 dB, and 6 dB.

For each error pattern e in E, we modify ::z by adding e to ::z. Consequently, there are error patterns in E that reduce the number of errors at the LRPs of ::z, and possibly, there is one error in E that removes all the errors at the LRPs of ::z. As a result, the modified vector ::z + e very likely contains either no errors or a number of errors within the error-correcting capability t of the code. fo that case, decoding ::z + e with an algebraic decoder will give the ML codeword. Based on this concept, we can devise a general decoding scheme as follows: lio Construct the error pattern set E based on the LRPs of 1r. '.t For each error pattern e in E, form a modified received vector 7l + e. :t Decode ::z + e into a codeword in C with an algebraic decoder (decoding may fail if the number of errors in ::z+e is greater then the error-correcting capability t of the code). 41, Steps 2 and 3 result in a list of candidate codewords. Compute the soft-decision decoding metrics of these candidates and find the one with the largest metric if correlation is used (or the smallest metric if squared Euclidean distance or correlation discrepancy is used), which is the decoded codeword.

The complexity of this general decoding scheme depends on the size of E and the complexity of the algebraic decoder for generating candidate codewords. The error performance of this scheme depends on how many LRPs are chosen to construct the error pattern set. Various decoding algorithms based on this general

402

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

scheme have been devised. These algorithms are referred to as LRP-reprocessing algorithms. The second type of reliability-based decoding scheme processes the MRPs of ir. This decoding approach is based on the fact that a set of k independent positions in z uniquely determines a codeword in an (11, k) linear code. Based on the reliability measures of the received symbols, we determine a set of k most reliable independent positions (MRIPs) in :r. Then, the hard-decision received vector z contains only very few errors in these k MRIPs. Let z1c denote the k-tuple that consists of the symbols of z at the k MRIPs. Let E be a set of low-weight error patterns of length k. For each error pattern e in E, we form Zk + ,e and encode it into a codeword in C. If z1c + e contains no errors, the resultant codeword is the ML codeword. A general decoding scheme based on this concept follows: lo Determine the error pattern set E based on the k MRIPs of r. 2, For each error pattern e in E, encode z1c + e into a codeword in C. 3, Step 2 generates a list of candidate codewords. Compute the soft-decision decoding metrics of these candidate codewords. Choose the one with the largest metric as the decoded codeword if correlation is used.

The error performance and complexity of this general decoding scheme depend on the choice of the error pattern set E. This general decoding scheme is referred to as the MRIP-reprocessing decoding algorithm. Various such decoding algorithms have been devised. In the rest of this chapter, some important LRP-reprocessing and MRIPreprocessing decoding algorithms are presented for binary linear block codes. In general, for nonbinary codes and multilevel modulation schemes, reliability measures are not as easily expressed as for BPSK transmission of binary codewords, and likelihood ratio metrics have to be considered. For a linear code over GF(q), these metrics depend on q, the modulation used, and the channel model and may also be simplified in conjunction with the algorithm considered. Such considerations can be found in [5- 7] for the A WGN channel with nonbinary error-correcting codes. 10.3

SUFFICIENT CONDITIONS ON THE OIPTIMAUTY OIF A DECODED CODEWORD

Reliability-based decoding algorithms generally require generation of a list of candidate codewords of a predetermined size to restrict the search space for finding the ML codeword. These candidate codewords are usually generated serially one at a time. Each time a candidate codeword is generated, its metric is computed for comparison and final decoding. If a condition can be derived to test whether a generated candidate is the ML codeword, then decoding can be terminated whenever this condition is satisfied. This may result in an early termination of the decoding process without generation of the entire list of candidate codewords and hence reduces computations and delay. Such a condition is referred to as an optimality condition. We now desire a sufficient condition on optimality of a candidate codeword based on the reliability measures of the received symbols. Let v = (vo. v1, · · · , v11 _1) be a codeword in C, and let c = (co, c1, · · · , c11 1) be its corresponding bipolar sequence using BPSK signaling, where c; = (2v; - 1),

Section 10.3

Su1'ficient Conditions on the Optimality of a Decoded Codeword

4013

for O .:s i < n. We define the following index sets for w: Do(w)

g

{i : v;

= z; with

O .:s i < n),

(10.22)

(10.23) = {O, 1, ... , n - 1}\Do(w).

Let (10.24) H follows from (10.1) and (10.21) that r; · c; < 0 if and only if z; i= v;. Consequently, we can express the correlation discrepancy 7'.(Ir, w) given by (10.13) as follows:

7'.(rr, ,y) =

L

Ir; 1-

(10.25)

iED1 (v)

MLD based on correlation discrepancy is to find the codeword in C that has the smallest correlation discrepancy for a given received sequence u·. If there exists a codeword w* for which A(ir, w*) .:S a(Ir, w*)

g

min {7'.(Ir, w)}, vEC,v,lv

(10.26)

then w* is the ML codeword for rr. It is not possible to determine a(rr, w*) without evaluating 7'.(rr, w) for all w in C; however, if it is possible to determine a tight lower bound 7'.* on a(rr, w*), then 7'.(rr·, w) .:S 7'.* represents a sufficient condition for the optimality of the candidate codeword w* in the list of candidate codewords generated by a reliability-based decoding algorithm. We derive such a sufficient condition next. H follows from (10.22) to (10.24) that the index set DoM consists of n - n(w) indices. We order the indices in Do(w) based on the reliability measures of the received symbols as follows:

= {!1, 12, · · ·, ln-n(v)}

Do(w) such that for 1

.:s i

< j

.:s n -

n(w), (10.28)

lr1; I < lri1 1-

Let

(10.27)

Dt (w) denote the set of first j indices in the ordered set Do(w); that is, D6il (w)

=

{1 1 , 12 ,

... ,

lj }.

(10.29)

For j .:S 0, D6J)(w) g 0, and for j :::= n - n(w), D611cw) g Do(w). For 1 .:s j .:S m, let Wj be the jth nonzero weight in the weight profile W = {O, w1, w2, · · · , w111 } of code C, where w1 = drninCC), and w1 < w2 < · · · < w111 • For a codeword win C, we define

G(w,

6

qj

-

Wj)

-

6

Wj -

L

n(w),

1 1 iED '1j (v) 0

Ir; I,

(10.30) (10.31)

404

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

and R(v, Wj)

=6.

'

'

{v EC: dH("i 'w) < Wj}

(10.32)

where dH (v', v) denotes the Hamming distance between v' and w. The set R(v, w i) is the set of codewords in C that are at Hamming distances WJ-1 or less from v. For j = 1, R(v, w1) = {v}, and for j = 2, R(w, wz) contains v and all the codewords in C that are at a distance dmin(C) = w1 from v. R(v, wz) is called the minimum distance region centered at v. The following theorem gives a sufficient condition for R(v, w i) to contain the ML codeword VML for a given received sequencer.

JL0.1 For a codeword v E C and a nonzero weight w 1 E W, if the correlation discrepancy .11.(r, v) of v with respect to a received sequencer satisfies the condition Jc(r, v) :S G(v, w 1 ), (10.33)

THEOREM

then the ML codeword '¥ML for r is contained in the region R(v, w 1 ).

Proof. Let

v be a codeword outside the region R(v, w 1); that is, v' E C\R(w, w 1).

Then, (10.34)

To prove the theorem, we need only to prove that Jc(r, w') 2: Jc(r, v). This would imply that no codeword outside the region R(v, w 1 ) is more likely than v. In that case, VML must be in R(v, w 1). Consider the sets Do(v) n D1(v') and D1(v) n Do(v'). We define 6 6

[Do(v) n D 1(v')!,

(10.35)

[D 1(v) n Do(v')I.

(10.36)

Then, it follows from (10.22), (10.23), and (10.34) to (10.36) that (10.37)

From (10.35) and (10.37), we readily find that [D1(v')! 2: [Do(w)nD1(v,)[ 2: w 1 -n10.

Because that

1110

=

(10.38)

ID1(v) n Do(v')! :S [D1(v)[, it follows from (10.38) and (10.24)

2:_ Wj - 1110

2:

Wj -

= w1 -

[D1(v)[ n(v).

(10.39)

Section 1003

Sufficient Conditions on the

l'fote that D 0 (v) n definition of Dbfl

a Decoded Codeword

~1[))5

and the

('/) <::::

'Ne have 0

7 )

Ir; I

== 0

iED1 (V

)

>

Ir; I

>

Ir; I iE

Wj)

0

~r') :": )l.(rc

thcin

Y.

Hence,

no codeword,/ outside the 0

YiYJ L

must be in

in Theorem 1001 Given a codevvord Y in C, the condition defines a in vvhich the ML codeword VML can be found. It says that vML is among those coclevvords in C that are at a distance of w 1_ 1 or less from the codeword Tvvo cases are For j = L R(v, w1) contains Y itself Therefore, the condition , v):::; G(v, w1) that v is the ML codeword Vp;JLo For j = 2, contains 7 and its nearest neighbors H<,cu,,v.u.c, distance from The summarizes these resuHs.

JliOULJL Let v be a codeword in a r be a received sequence.

!CROILJL,ATitY

linear (11, k) code C Let

1

:IL H

"t) :::;

w1 ). then 'Vis the lV[L codeword v ML for :r,

2, H . Y) > but A(R\ Y) :::; then the ML codeword YML for r is at a distance not than tbe minimum Hamming distance dmin ( C) = w I from In this case. YML is either v or one of the nearest of Y, The first of a sufficient condition for · [7'] , ~. 1 · conu1t10n ..,. · ot,. a co deworc,,1 Y anc,~ m 1.us can hue use d as a condition in any The interpretation of this condition in the 11-dimensional Euclidean space is depicted in Figure 102. In Figure 10.2, s(,I) the bipolar sequence associated with the binary n-tuple 2f, which differs from v in all in ( 1) and in the w 1 - n(Y) , • , (W1-ll(V)) Th' pos1t10ns m D 0 (w). 1s contained in the 11-dimensional hypersphere of Euclidean radius 4dmin (C) and centered at 1.: is a valid code sequence, where 1.: is the code sequence v, The second part of the allows us to search for the ML codeword v ML in the minimum distance region centered at a candidate codeword. Both conditions when 0

406

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

s (x)

C

C

(a)

(b)

FIGURE 10.2: Sufficient condition for optimality of c; (a) condition dE(r, c) > dE(r, s(x)); (b) condition satisfied: dE(r, c):::: dE(r, s(x)).

not satisfied:

used properly result in a significant reduction in average decoding computational complexity. This concept can straightforwardly be extended to the nonbinary case. A sufficient condition for determining the optimality of a codeword based on two codewords can also be derived. Let v1 and v2 be two codewords in C. We define 6

81

-

82

-

Doo

-

Do1

6 6 6

-

w1 - n(v1).

(10.41)

w1 - 11(v2),

(10.42)

Do(v1)

n Do(v2),

(10.43)

Do(v1)

n D1 (v2).

(10.44)

Without loss of generality, we assume that 81 :::: 82. We order the indices in the preceding index sets according to the reliability values of the received symbols as defined by (10.27) and (10.28). We define /(v

"')

] , v2

!:. (D U D(LC81-82l/2J))C81l 00 01

'

(10.45)

where xCqJ denotes the first q indices of an ordered index set X as defined in (10.29). We define (10.46) G(v1,w1;v2.w1)~ lril-

L

iEI(v1 .v2)

Let v be either v1 or v2. whichever has the smaller correlation discrepancy. If (10.47)

then vis the ML codeword vML for r. The derivation of (10.47) is left as a problem (see Problem 10.1). The sufficient condition on optimality based on two codewords given by (10.47) is less stringent than the sufficient condition given in Corollary 10.1.1 for j = 1

Section 10.4

Generalized Minimum Distance and Chase Decoding Algorithms

dl-ff7

s (x')

''\

/

I

\

I

\

I

\

I

I I

I I I

n:

~----1

~---i-1

\

ic'

0

/ s (x)

I

I \

I \

I \

I \

'

/

'

/

Sufficient condition for the optimality of IC satisfied based on both IC and but not satisfied based on IC only; dE IC) :s dE(rr, s(;i but dE(K, IC) > dE s(l'l)).

FIGURE 10.3:

0

IC'

)),

based on a single codeword, which may result in a faster termination of the -,vvv•~rn,,c., process in a reliability-based decoding algorithm. The advantage of considering two codewords rather than one to derive a sufficient condition of optimality is illustrated in Figure 10.3. For any codeword w with corresponding signal sequence IC, no point contained in the 11-dimensional hypersphere of squared Euclidean radius 4dmin ( C) and centered at IC is a valid candidate codeword. Therefore, if for two signal sequences «: and IC' representing two codewords w and w' processed by the algorithm, the corresponding hyperspheres intersect, then it is possible to eliminate binary 11-tuples l'l based on the common knowledge of these two hyperspheres. Such a case is depicted in Figure 10.3, where the knowledge of the hypersphere associated with IC, allows us to determine the optimality of the codeword a:.

In this section and the following ones, we present various decoding algorithms based on processing the LRPs of a received sequence. The first such algorithm is known as the generalized minimum distance (GMD) decoding algorithm devised by Forney [5] in 1966. This decoding approach was later generalized by Chase and others, for binary linear block codes [9, 10]. W.4L 1 lhlie GMIDJ IDJtew«:iln11191 Aigi«:wiitriim

The GMD decoding algorithm is a very simple and elegant method of using reliability information of the received symbols to improve algebraic decoding for both binary and nonbinary codes. For an (n, k) linear block code with minimum Hamming distance dmin, this decoding algorithm generates a list of at most L(dmin + l)/2J candidate codewords based on an error-and-erasure algebraic decoding method

408

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

and then selects the most likely one from the generated codewords. As stated in Section 7.7, an error-and-erasure decoding method can correct all combinations of v errors and e erasures provided that 2v

+e S

dmin -

1.

(10.48)

The GMD decoding algorithm simply considers all possible cases of erasures S d 111 ; 11 - 1 in the dmin - 1 LRPs, which are the most likely positions to be in error. The decoding is carried out as follows: e

1. Form the hard-decision received sequence z from r and assign a reliability value to each symbol of z. 2, Generate a list of L(d111 ; 11 + l)/2J sequences by modifying the hard-decision received sequence z. If d 177 ;n is even, modify z by erasing the least reliable symbol, the three least reliable symbols, · · · , the dmin - 1 least reliable symbols. If d 111 ;n is odd, modify z by erasing no symbol, the two least reliable symbols, · · · , the d111 ;11 - 1 least reliable symbols. 3. Decode each modified z into a codeword v using an error-and-erasure algebraic decoding algorithm. 4, Compute the soft-decision decoding metric for each generated candidate codeword. Select the candidate codeword with the best metric as the decoded solution. Steps 2 and 3 can be executed simultaneously. In most cases, fewer than L(dmin + l)/2J candidate codewords are generated by this procedure, since some algebraic decodings at step 3 may fail and result in no candidate codewords. Also, a sufficient condition on optimality can be used for testing each candidate codeword when it is generated. Decoding is terminated whenever a generated candidate codeword satisfies the optimality condition. For decoding RS codes, the Euclidean error-anderasure decoding algorithm presented in Section 7.7 can be used. 10.4.2

Chase Dewding Aigorithms

In generalizing the GMD decoding approach, Chase devised three algorithms [9], namely, algorithm-1, algorithm-2, and algorithm-3. For decoding binary linear block codes, Chase algorithm-3 is very similar to the GMD decoding, except that the erasure operation is replaced by the complementation operation at the LRPs of the hard-decision received sequence z (i.e., changing a 1 into a 0, or a O into a 1). An error-correction-only algebraic decoder is used to generate a list of at most Ld111 ; 11 /2 + lJ candidate codewords. Chase algorithm-3 performs the following steps in decoding:

L Form the hard-decision received sequence z from r and assign a reliability value to each symbol of z. 2, Generate a list of at most Ld111 ; 11 /2 + lJ sequences by modifying the harddecision received sequence z. If d 111 ;n is even, modify z by complementing no position, the least reliable bit, the three least reliable bits, · · · , the d111 ;n - 1 least reliable bits. If d111 ,,, is odd, modify z by complementing no symbol. the two least reliable bits, the four least reliable bits, · · · , the d111 ;n - 1 least reliable bits.

Section 10.4

Generalized Minimum Distance and Chase Decoding Algorithms

i!illll'9J

Jo Decode each modified z;; into a codeword Y using an error-correction-only algebraic decoder. 4lo Compute the soft-decision decoding metric for each generated candidate codeword. Select the candidate codeword with the best metric as the decoded solution. From step 2 we see that for even dmin, error patterns of odd weights confined to the dmin - 1 LRPs are added to z;; for decoding, and for odd dmin, error patterns of even confined to the dmin - l LRPs are added to z;; for decoding. Because algebraic decoding of a modified 7l may fail, at most Ldmin/2 + lJ candidate codewords can be generated. H a condition on optimality is used for testing each candidate codeword when it is generated, it may shorten the decoding process. For binary codes, Chase algorithm-3 achieves about the same error performance as the GMD decoding algorithm and requires about the same computational complexity. Chase algorithm-2 is an improvement of algorithm-3; it generates a larger list of candidate codewords for decoding. fo this algorithm, a set E of error patterns with all possible errors confined to the Ldmin/2J LRPs of z;; is used to modify 21. This algorithm performs the following decoding steps:

JL Form the hard-decision received sequence 21 from r and assign a reliability value to each symbol of 21. Z, Generate the error patterns in E one at a time, possibly in likelihood order. For each error pattern ie, form the modified vector z;; + ie. Jo Decode each z;; + ie into a codeword Y using an error-correction-only algebraic decoder. 4lo Compute the soft-decision decoding metric for each generated candidate codeword. Select the candidate codeword with the best metric as the decoded solution. The set E is called the test error pattern set, and it consists of the zld,,,;,,/ 2J test error patterns obtained by considering all possible combinations of O's and 1's in the Ldmin/2J LRPs of 21. Therefore, the candidate codeword list for Chase algorithm-2 grows exponentially with dmin and hence is much larger than the candidate codeword list for algorithm-3 when dmin becomes large. The result is a greater computational complexity; however, algorithm-2 achieves a much better error performance than algorithm-3. A condition on optimality can be incorporated in Chase algorithm-2 to reduce the number of algebraic decodings. Chase algorithm-1 generates a list of at most (Ldm'.'.,/2J) candidate codewords by complementing all possible combinations of exactly Ld111 ; 11 /2J positions in the harddecision received sequence 21. Owing to its very large computational complexity compared with other soft-decision decoding methods, Chase algorithm-1 has never received much attention. Among the three Chase algorithms, algorithm-2 gives the best trade-off between error performance and decoding complexity.

1(0),4L3

Gie111erniu2iedl CrniZ1::1ie zi111dl GM[)) [)lerc
410

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

algorithm A(a) of this family generates at most2a-l ( r(dmin + 1)/21-a + 1) candidate codewords. For a chosen a, we form a set E(a) of error patterns by modifying the hard-decision received sequence z. For even dmin, E(a) consists of the following error patterns: 1. The error patterns with errors confined to the a -1 LRPs. There are 2a-l such error patterns. 2, For each of the preceding error patterns, the i next LRPs, with i = 0, 1, 3, · · · , dmin - 2a + 1, are complemented. For odd dmin, E(a) consists of the following error patterns: 1, The error patterns with errors confined to the a - 1 LRPs.

2. For each of the preceding error patterns, the i next LRPs, with 0, 2, 4, · · · , dmin - 2a + 1, are complemented. Then, algorithm A(a) executes the following steps in decoding: 1, Form the hard-decision received sequence z from r and assign a reliability value to each symbol of z. 2, Generate the error patterns in E(a) one at a time, possibly in likelihood order. For each error pattern e, form the modified vector z + e. 3. Decode each z + e into a codeword v using an error-correction-only algebraic decoder. 4. Compute the soft-decision decoding metric for each generated candidate codeword. Select the candidate codeword with the best metric as the decoded solution. We readily see that A(l) and A(rdmin/21) are simply Chase algorithm-3 and Chase algorithm-2, respectively. It has been shown that for BPSK signaling, algorithm A(a) with a = 1, 2, · · · , rdmin/21 achieves bounded distance decoding [10]. Another family of rdmin/21 decoding algorithms, denoted by {Ae(a) : a = 1, · · · , rdmin/21}, can be obtained by modifying the foregoing error pattern set E(a). Let Ee(a) denote the test error pattern set for algorithm Ae(a). The set Ee(a) is very similar to E(a) after the complementation operation is replaced with the erasure operation. For even dmin, Ee(a) consists of the following error patterns: 1, The error patterns with errors confined to the a - 1 LRPs. 2. For each of the preceding error patterns, the i next LRPs, with 1, 3, · · · , dmin - 2a + 1, are erased. For odd dmin, Ee(a) consists of the following error patterns: 1. The error patterns with errors confined to the a - 1 LRPs. 2. For each of the preceding error patterns, the i next LRPs. with 0, 2. 4, · · · , dmin - 2a + 1, are erased.

Genernlized Minimum

Section 10.4

and Chase

(Ac

1dmin /21) can be shown to be among the Problem For a = L · · · . 1dmin /21, both families of u'"'""''-"'J'i". {Ae(a)}, achieve bounded distance "'·''Y'-''·H""" ,~~UH,,CYU

and

families of bounded distance ·,,,~,,v·,.m between the error ·~~,·+,~,··~· and a considerations. shows the bit. error decoding for o

code and

fft

...._....

10-1

10·'

,

--.... . . °"',,..!1·lgebraic decoding

-

>,

E

"0

.D

10-1

I

'"-

H

P.,

\

H

...0 ! H

10 .. ;

\

rii

+

10-)

:Ae(l) or GMD

o : A(l) or Chase-3 · A(2) 10-(;

x : A( 4) or Chase-2 .~~~-~__[_~~-L_____l______L~~~~-~~

10-7

2

0

3

4

5

6

7

8

SJ\TR (in dB)

FIGURE 10.4: Bit-error ~.c,·,. A(4) decoding algorithn1s.

0

0 ·'

,~,1,unA,G

of the

8) RM code with A(l), A(2). and

412

Chapter 10 1011

Reliability-Based Soft-Decision Decoding Algorithms

C---- - - , - - - - , - - - - , - - - - - - - - - - - , - - - - - - - - - . - - - ~ - ~

-,,.... Algebraic decoding __,,_ A(l) or Chase3 _,__ A(6) -s-A(ll) or Chase2 -+-MLD

10"

2

3

4

5

6

SNR (in dB)

Bit-error performances of the (127, 64, 21) BCH code with A(l), A(6), and A(ll) decoding algorithms.

FIGURE 10.5:

A(4) require to generate and process at most 5, 8, and 16 candidate codewords, respectively. The Reed algorithm (see Section 4.3) is used to decode the (64, 42, 8) RM code. In this figure, the error performances of the algebraic, GMD (or Ae(l)), and MLD decodings of the code are also included for comparison. We see that GMD and A(l) achieve the same error performance with the least computational complexity (processing only 4 candidate codewords). They achieve 1.1-dB coding gain over the algebraic Reed decoding algorithm at the BER 10-4 . Algorithms A(2) and A(4) give better error performance than the GMD and A(l) decoding algorithms and require to process 2 and 4 more candidate codewords, respectively. Algorithm A(4) achieves 1.6-dB coding gain over the algebraic decoding of the code at the BER 10-4 but has a 0.9-dB degradation in coding gain compared with the MLD of the code. Figure 10.5 shows the bit-error performances of the of the (127, 64, 21) BCH code with decoding algorithms A(l), A(6), and A(ll). The Berlekamp algorithm is used in decoding the (127, 64, 21) BCH code. Algorithms A(l), A(6), and A(ll) require to generate and process at most 11, 192, and 1024 candidate codewords, respectively. We see that A(l) (also GMD) performs better than the algebraic decoding only for SNR values greater than 4.7 dB. Algorithm A(ll) (or Chase algorithm-2) achieves 1.1-dB coding gain over algebraic decoding at the BER 10-4 but has a 1.7-dB loss in coding gain compared with MLD. In general, for a given a, the performance degradation of A(a) (or Ae(a)) compared with MLD increases with the minimum Hamming distance dmin of the code considered.

Section 10.5

rn.s

Weighted Erasure Decoding

4B

WIE!G!11lE[JJ IE~ASllJ!PllE [JJ[E(l1J[DlmllG

The weighted erasure decoding (WED) is another variation of GMD for a input and output channel. This algorithm was devised Weldon for u,uuu,cou channel outputs [11 ]. fo this section, we present a simplified version of the WED algorithm. Consider a linear (11, k) code with minimum distance dmin that is used for error control. Suppose the channel output is uniformly quantized into Q levels with Q - 2 granular regions of equal space and 2 overload regions at the two ends. For Q = 2111 , these Q regions are assigned Q w-weights w.i = j /(Q - 1), for 0 ::: j ::: Q - 1. For O ::: i ::: m - 1, we define Pi

'£,;";;}

p;

= 2111-i-l /(Q

-1).

(10.49)

= 1. H follows from the definition of w-weights and (10.49) that m-1 Wj =

L

(10.50)

Gj.iPi,

i=O

where the m-tuple (oJ.O, oJ.l· · · · , ai- 111 _i) is simply the binary representation of the integer j for O ::: j ::: Q - l; that .

J

=

m-1 '\:-'

LaJ.i

2i

(10.51)

.

i=O

Each received r1. for O ::: l ::: n - 1, is quantized into one of the Q levels and assigned the w-weight associated with that level. For O ::: l ::: 11 - 1, let w1 denote the of the received symbol ri. We express w1 in the form of (10.50) as follows: m-1

WJ

=

Va(l),·Pi· L J.

(10.52)

i=O

VVe construct an m >< n binary matrix A associated with the received sequence = (ro, r1, · · ·, r11 _1) for which the /th column is given by the binary m-tuple (on, 'o;'.;11-1> obtained from (10.52). Then, WJ is called the weight of the /th column of A. For O :s i :s m -1, the ith row of A is decoded into a codeword in code C with an algebraic decoder. Them decoded codewords are then used in the same order to form another m >< n binary array A'. For O < i ::: m - 1, let Ji denote the number of places where the ith row of A and the ith row of A' differ. We define the reliability indicator of the ith row of A' as

ir

an' ...

R;

=

(10.53)

max{ 0, clmin - 2 f; }.

Si

Sb

For the !th column of A', with O < l ::: 11 - 1, let and denote the index sets of the rows that contain 1 and O at bit position l, respectively. Then, the !th bit is decoded into O if (10.54) Ri Pi > R; Pi

L

L

414

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

and decoded into 1 if )£.....i R;

D;

'

< )k-J R;. 'D;.

iESb

(10.55)

iESj

In case of equality, the hard decision of r1 is taken to be the decoded bit. Let N 1 denotes the number of columns in A with weight w J, for O :S j :S Q -1. Then, the WED algorithm gives correct decoding if the following condition holds: Q-1

L

N·w· J .I <

(10.56)

2

j=O

EXAMPLE 10.1

Consider the (8, 4, 4) RM code generated by G-

-

l

l 1 1 1 1 00001 0 0 1 1 0 0 1 0 1 0

Suppose this code is decoded with the WED algorithm with 4-level output quantization and two granular regions of spacing .6. = 0.3. Thew-weights associated with the quantization levels are wo = 0, w1 = ~, w2 = ~, and w3 = 1, respectively. We find that po= ~. and Pl = ~- It follows from (10.50) that we can represent thew-weights as follows:

= 0 · Po + 0 · Pl, w1 = 0 · PO + 1 · Pl, w2 = 1 · PO + 0 · Pl, W3 = 1 · PO + 1 · Pl·

wo

Therefore the four 2-tuples associated with the w-weights are (0, 0), (0, 1), (1, 0), and (l, 1). The generator matrix G can be put in reduced echelon form (REF) by elementary row operations:

GREF

=

l

1 0 0 0 0 0 0 1

0 0 1 0

1 0 1 1

0 1 0 0

1 1 1 1 0 1 1 0

0 1 1 1

l .

From the last paragraph of Section 10.1, we know that encoding based on GREF provides a smaller bit-error probability than encoding based on G. Hence, suppose encoding is based on GREF· Then, the message u = (1, 0, 0, 0) is encoded into the codeword v = (1, 0, 0, 1, 0, 1, 1, 0). For BPSK transmission, v is mapped into the bipolar sequence c = (1, -1, -1, 1, -1, 1, 1, -1). Suppose this sequence is transmitted, and the received sequence is ir

= ( 0.7, -2.1, -1.1, -0.4, -1.4, 1.7, 0.8, 0.1 ).

Section 10.5

Weighted Erasure Decoding

~15

H a hard decision is made for each received symbol, we obtain the following hard-decision received vector: tZ

= ( 1,

0, 0, O*, 0, 1, 1, 1* ).

This hard-decision received sequence contains two errors (marked*) compared with the transmitted codeword w. Because the error-correcting capability of the code is t = 1, correct decoding of tZ is not guaranteed with algebraic decoding. With WED, each symbol ofir is assigned a specific weight in (wo, w1, w2, w3} associated with the region that contains the symbol. Then, the w-weight sequence for IT' is

Based on the 2-tuple representation of each weight, we form the 2 x 8 binary array

1 0 0 O* 0 1 1 0 J. = [ 1 0 0 O* 0 1 1 1*

A

The first row of A is equivalent to the hard decision of ir with two positions in error. The second row of A contains only one error. The Reed decoding algorithm decodes the first row of A into information sequence 1lll1 = (0, 1, 0, 0), and the second row of A into information sequence @2 = (1, 1, 1, 1). Because decoding assumes that encoding is based on the generator matrix G (Reed algorithm), the two decoded information sequences., 1llll and 1lll2, are re-encoded into two codewords based on G. Using these two codewords as rows, we obtain the matrix

A'=[OOOOl

1 1

1 0 0 1 0 1 1

Comparing A and A', we obtain .fo = 2 and .fi = 1. H follows from (10.53) that the reliability indicators for the two rows of A' are Ro = 0 and R1 = 2. The index sets for the columns of A' are given in Table 10.1. To decode the code bit at position 0, we compute 0,

LR; p; = Ropo iES8

2-

LR; p; = R1 Pl

1 3

2 3

iES~

TABLE 10.1:

Index sets for the columns of A'.

C
~

].

z

3

4l

§

I/ii

7

si0

{O}

{O, 1}

{O, l}

{O}

{1}

0

0

{1}

sil

{1}

0

0

{1}

(O}

(0. 1}

{O, 1}

{O}

416

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

Because Li ESo Ri Pi < L; ESo Ri Pi, the decoded bit at position O is Do 0 I decode the code bit at position 1, we compute

L

R; Pi= Ro Po

+

R1 Pl

= O+ 2 · }

= 1. To

2

3,

iES6

L

RiPi =0.

iESf

Because LiEsg Ri Pi > LiESf R; Pi, the decoded bit at position Ois D1 = 0. Repeating the foregoing decoding process, we obtain the following decoded sequence:

v=

( 1, 0, 0, 1, 0, 1, 1, 0 ),

which is identical to the transmitted codeword. Thus, the decoding is correct. The corresponding information sequence is easily retrieved based on GREF· This example illustrates that an error pattern not correctable by algebraic decoding can be corrected by the WED algorithm. The error performance achieved by the WED algorithm strongly depends on the quantizer levels, which have to be chosen according to the operating SNR [12]. Nevertheless, the WED represents a very attractive solution for practical implementations of low-complexity reliability-based algorithms owing to its simplicity. 100

10-1

c

10-2

]

"'

..D 0 h 0..

10-3

h

0

h h

1

i:o

+:GMD @: Chase-3 * :WED;Q = 4 o:WED;Q=8

10-.\

10-0

x: Chase-2

10-6

1

2

3

4

5

6

7

SNR (in dB)

FIGURE 10.6: Comparisons between WED for Q = 4 and Q = 8 and GMD, Chase-3, and Chase-2 decodings for the (64, 42, 8) RM code.

Section 'I 0.6

Maximum Likelihood

feature of the rnodified varies ·with each received sequence. as This feature of has been in [13]. 10.6 compares WED for Q = 4 and Q = 8 with GMD. and Chase-2 for the 8) RM code. 1Ne observe that VVED for Q = 8 values than 10- 3 11[)).(5

fa\ Mfa\Z{uMUM U!(iEUHOOli) DIEOOD!uW5 t\LGOmTHlslf1 13\A'.SIElCi Oil\\ fflElPV\lff\flE

rROCIES§!~~G OIF Tt0HE UEfo',H RH.!AlSl!UE ;PO:Srno1ws

is limited

codes.

To reduce the must be devised to reduce the search space Such an algorithm has been devised and Hirasawa [14]. For simplicity. we call this This processes the LRPs of the received sequence and uses a condition on the correlation discrepancy of the best candidate codeword generated either to the direction of further search or to tenninate the decoding process when the most likely codeword is found. The Kl\llH achieves maximum likelihood decoding. Let C be a linear (n. k) block code with minimum distance dmin. Suppose decoder is used to candidate codewords. The decoder is to correct error of up tot = l(dmin l)/2J. Let rr = (ro, r1. · · · , r11 _1) be the received sequence and 2 be its corresponding harddecision sequence. Let E(i) denote the set of error with errors confined in the i LRPs of 2, and let denote the set codewords obtained the sequence:;;;+ re withe E E(i). it follows from the definition of J(i) that 1(0) <;; J(l) <;; 1(2) · · · <;; 1(11),

(10.57)

where J (0) one codeword obtained by decoding :; ; if decoding is successful and is of:;;; fails. For O ::=: i ::=: n -1. consider a codeword ,/ in J(i + l)\J(i). First,,/ E J(i + 1) implies that,/ must have at most t positions j outside the (i + 1) LRPs such that v~ I Zj. ·v' Fj. J (i) implies that v' contains at least (t + 1) positions j outside the i LRPs such that v~ I z j. Combining these j outside the i LRPs such two it follows that v' has exactly (t + 1) that v'i I Zj, and one such position is the (i + 1)th LRP. Without loss of generality. assume that the received ri are ordered to increasing reliability values; that ro is the least reliable symbol, and r 11 _1 is the most reliable symbol. H follows from the definition of correlation discrepancy given (10.13) that the

418

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

correlation discrepancy of v' with respect to the received sequence r satisfies 1+1

A(r, v') c:::

L lri+Jl-

(10.58)

i=l

Let v be the codeword in J (i) that has the least correlation discrepancy with r (i.e., vis the most likely codeword in l(i)). Then, dH(v, v') c::: dmin· Let 01

le.

= max{O, dmin

We define Gi(v)

- (t

+ 1) -

ID1(v)I}.

(10.59)

lril·

(10.60)

t+l

L lri+jl + L

~

j=l

jED6b' 1(v)

Based on the same approach as for Theorem 10.1, it can be shown that [14] >-.(r. v') c::: Gi(v).

(10.61)

The preceding inequality simply says that any codeword outside of J (i) cannot have a correlation discrepancy smaller than Gi (v). Therefore, if >-.(r, v) < Gi (v),

(10.62)

then the most likely codeword with respect tor must be in J (i). From (10.60) we see that the threshold Gi (v) monotonically increases with i. Therefore, as i increases, it becomes more and more likely that the condition of (10.62) will be satisfied, and the search for the ML codeword will be terminated. Based on the foregoing development, we can formulate the KNIH algorithm as follows:

1. Decode z into a codeword with the algebraic decoder. Set i = 0. If decoding is successful, record the decoded codeword v and its correlation discrepancy >-.(r, v) and go to step 2. Otherwise go to step 4. 2. Based on v, determine the smallest integer imin for which A(r, v) < Gimin (v). Set i = i + 1 and go to step 3. 3. If i s imin, find the codeword v' in J(i)\J(i - 1) that has the least correlation discrepancy with r. If >-.(r, v < >-.(r, v). set v = v'. Go to step 2. If i > imin. then v is the ML codeword v ML. The decoding stops. 4. Find the codeword v in J (i + 1) \J (i) that has the least correlation discrepancy with r, set i = i + 1 and go to step 2. If J(i + l)\J(i) is empty (i.e., all the decodings of z + e withe E E(i + 1)\E(i) fail), set i = i + 1 and repeat step 4. 1

)

The KNIH algorithm always finds the most likely codeword VML, but the number of candidate codewords to be generated depends on the SNR. Even though the average number of computations may be relatively small, the worst-case computational complexity is bounded by 2k, the total number of codewords in C. The definition of Gi (v) given in (10.60) can be viewed as an improved version of G(v, wi) given in (10.31), since in (10.60) the fact that at least (t + 1) positions

Section 10.7

Reduced List Syndrome Decoding Algorithm

~-121

outside the i LRPs differ from the corresponding positions in bl is used. In other w1) given in has been improved by exploiting the reprocessing strategy of the KNH--I The average number of computations obtained the KNH-l[ here can be further reduced if in condition (10.60) then the a further of Gi (w) is used. Uthe decoding of bl is algebraic codeword wo can be used in conjunction with the best recorded candidate codeword v to determine the optimality of w based on an improved version in (10.46). H the decoding of bl fails, then we use of G(w, w1 ; vo. w 1 ) (v) in the condition of (10.60). Such a condition is used in [14]. This is effective for decoding codes of short to medium For long codes, computational becomes very large, especially for low SNR. For example, consider the decoding of the (128, 64) extended BCH code. At SNR = 5 dB, the KNIH algorithm generates more than 2 million candidate codewords to decode one particular received sequence ir [14]. To overcome the 2 of we may set imin S imax < n at cu,_,~,""'"· This reduces the worst-case computational complexity; however, becomes suboptimum. A proper choice of i 111 ax will provide a good trade-off between error performance and computational complexity. Also, solutions for reducing the average number of candidate codewords processed by the KNIH algorithm have been proposed in [15]. 11117

filE[l)IUJ(IE[l) UST sv~m~OMIE ll)IE(OrnNG AILGOmTHM

The reduced list syndrome decoding (RLSD) algorithm proposed by Snyders [16] provides a different approach to MLD. This algorithm is devised based on the paritycheck matrix IHI of a code, the syndrome§ of the hard-decision received vector bl, and the reliability measures of the symbols in the soft-decision received vector ir. The algorithm requires identification of at most the n - k - 1 least reliable independent positions (LRIPs) in re and is mostly suitable for decoding high rate codes with n - k « k. With respect to the LRP-reprocessing decoding algorithms presented in the four sections, this decoding algorithm requires additional effort to select independent least reliable positions in re for reprocessing. The LRIPs can be identified by eliminating dependent LRPs. A systematic procedure for eliminating dependent positions in re will be presented in Section 10.8.1. The main idea of the RLSD algorithm is to construct a fixed reduced list of error patterns for the syndrome § computed from the hard-decision received vector bl and the parity-check matrix IHI. From this list of error patterns we generate a list of candidate codewords among which one is the ML codeword with respect to the soft-decision received vector re. There are 211 -k possible syndromes. Therefore, 211 -k such lists of error patterns must be constructed. The construction can be tedious; however, code structure can be used to reduce the effort. For a given code, the construction of the reduced lists of error patterns for all possible syndromes need be done only once. Reduced list syndrome decoding is carried out based on the formulation of MLD proposed in [17]. We express the parity-check matrix IHI in terms of its columns as follows: IHI= [lino, lln1 · · · lln11-1],

(10.63)

420

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

where for O ::: i < n, hi denotes the ith column of H. Let v be a codeword. We define an error pattern e = (eo, e1, · · · , en-1) as follows: for l for l

E

D1 (v),

f. D1 (v),

for O ::: l < n, where D1 (v) is the index set defined in (10.23). Then, e v = e + z. Consequently, the syndrome of z is §

=

(10.64)

= z + v,

and

zH 7 = eH 7 (10.65)

For a given codeword v the error pattern e defined in (10.64) is called an n(v)pattern, where n(v) = ID1(v)I as defined in (10.24). From (10.65) we see that an n(v)-pattern e is an error pattern of weight n(v) such that the n(v) columns of H corresponding to the nonzero positions of e (or the positions in D1 (v)) sum to the syndrome s. We also can represent this n(v)-pattern with its corresponding set of columns of H summing to §: {h1: l E D1(v)}. It follows from the definition of an n(v)-pattern, (10.13), and (10.65) that we can formulate MLD as follows: among all n(v)-patterns e for n(v) ::: n, find the pattern e* that minimizes the correlation discrepancy A(r, z+ e) between rand the codeword v = e + z. Then, the codeword v* = e* + z is the ML codeword with respect tor. For a given syndrome§ there are 21c n(v)-patterns; however, we can eliminate some of them from consideration. This follows from the fact that the rank of H is n - k. Using this fact, we can prove that for any n(v)-pattern e with n(v) > n - k, there exists an n(v')-pattern e' with n(v') ::: n - k such that (10.66)

The preceding equality implies that e + e' = vo is a codeword. Furthermore, this codeword vo has Hamming weight n(v) - n(v'). Consider the codewords v = e + z and v' = e' + z. Then, vo = v + v'. We can readily prove that A(r, v)

= A(r, v') + A(r, vo). (10.67)

:::: A(r, v'). which implies that the n(v)-pattern e cannot produce the ML codeword 'VML· Consequently, all n(v)-patterns of weight n(v) > n - k for a given syndromes can be eliminated from the search of the ML codeword VML· In fact, more n(v)-patterns can be eliminated based on the partial knowledge of LRIPs. Based on the definition of an n (v)-pattern, the only n (v)-pattern e of weight 1 that needs to be considered is the one that corresponds to the column in H that is equal to the syndrome s (if such a column exists). Let l!Jli denote the column of lH! that corresponds to the !th LRIP of r, and let r.!,. denote the set 6 (10.68) r.l,- = {q1, q2, · · ·q,-}.

Section 1

Reduced List

An n(v ~1 can be elinrdnated fro1n the r.1°-~~.~·,"· known to determine an 11 and 'lf/ == fl;/ + ::Z~ rPCcnP•f'fl'lP a reduced list of ;1 for r :S 11 - k. For < ~ vvhere for 2 :=:: j with n(v') :::: that contains 0

-

ho,vve1.1er ~

from the next theorem

For

from the fact that the rank of lB1 is

-"JV~o,OWCc~

11

-

k.

lc!li, then aH n {iqJ1, 1~12, · · · COJ;,-d is the

of

fo [16], several theorems that allow elimination of error reduced lists are derived. The sizes of the lists can be further reduced the structures and of the code ~~nc,d,"•c,r:I -J,,,.. ,.,_,,- code.

1rtatrix

Consider the (7, 4, 3) ~

-

]

1 0 0 1 0 1 1 0 1 0 1 1 0 1 0 0 1 0 1 1

2111 - 1, since rm consists of all the nonzero For a Hamming code of as columns, a nonzero 5 must be identical to one of the columns in !EL 1. The reduced list for the (7, 4, Therefore, there is an n 3) Hamming code consists of one n {§} of {mJ1, (!]]1 + §}, {(!]]2, IJJ]2 +§},and {mJ1 + (!]]2, !DJ1 + mJ2 +§},of {1J!1, (!]]2, (!]]1 +mJ2 +s} of 3. All other n or r.!2-eliminated. For example. {(!]]3, (!]]2 + QJJ3} is MLD is achieved by first identifying those 11 for the :s and then ...,..,.H,_, the candidate codewords generated the n Hamming codes, all patterns on the list have to be nonzero as columns. Also, 5 complete list can be further reduced. For and for s = ilJll + I\JJ2, the list reduces to {B) and (!Ill1. then the entire list needs to be considered. Let rr· = (-0.7: -0.8: -1.1: be the received sequence. The hard-decision received vector is ?Z = (0 0 0 1 1 0 0), rnad the of ?Z is § = ?ZJHIT = (101) = 11nr. We find that ([)Jl = /J112 = and§ = IDJ1 + l1Ji2· As a

422

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

only the n(v)-patterns {hs} and {h3, lhl4} have to be considered. The error pattern corresponding to {lms} is ce1 == (0 0 0 0 0 1 0), and the codevvord generated by this error pattern is v1 = e 1 + z = (0 0 0 0 0 1 0) + (0 0 0 1 1 0 0) = (0 0 0 1 1 1 0). The correlation discrepancy of v1 is J,.(ir, v1) = 1.1. The error pattern corresponding to {h3, lh.4} is e2 = (0 0 0 11 0 0), and the codeword generated by this error pattern is v2 = e2 + z = (0 0 0 1 1 0 0) + (0 0 0 1 1 0 0) = (0 0 0 0 0 0 0). The correlation discrepancy of v2 is J,.(ir, v2) = 0.9. Hence, v2 = (0 0 0 0 0 0 0) is the ML codeword. In [16], the reduced lists of n(v)-patterns for the (15, 11, 3) and (31, 26, 3) Hamming codes are given. The complete reduced list for the (15, 11, 3) code is composed of one n(v)-pattern of weight 1, seven n(v)-pattems of weight 2, seven n(v)-patterns of weight 3, and one n(v)-pattern of weight 4. Also in [16], a method is presented for efficiently constructing the reduced list of n (v)-patterns for an extended code based on the reduced list of the original code. Schemes for constructing reduced lists of n (v)-patterns for codes based on binary trees and graphs are also presented in [18]. MLD of high-rate codes based on the RLSD algorithm can be achieved with very few real operations and therefore is quite suitable for high-speed decoding of these codes. 10.8

MOST RIEUAIBllE iNIDIElf!'IENIDllENl rosrnoN RIEIP'ROCIESiSi!NG ID!IECOIDliNG ALGORITHMS

In the previous four sections we presented various soft-decision decoding algorithms for linear block codes that are based on processing the LRPs of a received sequence. These decoding algorithms are referred to as LRP-reprocessing decoding algorithms. In this section we present two soft-decision decoding algorithms that are based on the processing of MRIPs of a received sequence. These decoding algorithms are referred to as MRIP-reprocessing decoding algorithms.

10.8.1

Most Reliable and least Reliable !Bases For the GMD and Chase-type decoding algorithms, only a partial ordering of the reliability values of the received symbols is needed to identify the LRPs of a received sequence for decoding; however, decoding based on MRIPs, requires not only a complete ordering of the received sequence based on their reliability values but also identification of k MRIPs. These k MRIPs can be determined by permuting the columns of the generator matrix of a code based on the ordering of the received symbols and then finding k independent columns by elementary row operations of the permuted generator matrix. Consider a binary (n, k) linear block code C with generator matrix G. Suppose a code sequence is transmitted. Let ir = Cro. r1, · · · , r 11 ~1) be the received sequence. We order the received symbols based on their reliability values in decreasing order (if there is a tie between two received symbols, then we order them arbitrarily). The resultant sequence is denoted by (10.70)

r;,~l

with ! r~ l>I r~ I> · · · >I 1- This reordering of the received symbols defines a permutation n 1 for which 1/ = n 1[ir]. We permute the columns of G based on n1 and

Reprocessing Decoding Algorithms

Section 10.8

&J:231

obtain the following matrix:
= n1[
[

I

I

1/iill, 1/ill · · · 1/inn-ll

/]

(10.71)

,

where for O ::: i < n, gn' denotes the ith column of cc;;'. The code c' generated is equivalent to C generated by
cc;;'

c' = n1[C] = {n1(w): w EC}.

r;

Because the ith column 11;/ of to the ith of ir, with reliability value [ 1.; [, we call I I the reliability value associated with column 1/in,. Although the first k positions of ir, ( or the first k columns of cc;;') are the k most reliable positions, are not necessarily independent, and therefore they do not always represent an information set. To determine the k MRIPs, we perform elementary row operations (or Gaussian eliminations) to put cc;;' in the reduced echelon form. There are k columns in cc;;' in reduced echelon form that contain one 1. These k columns are linearly independent. Consequently, the positions in ir that to these k linearly independent columns are the k MRIPs. Note that the matrix in reduced echelon form generates the same code c' as cc;;', except for the mapping between an information sequence and a codeword. We use these k linearly independent columns as the first k columns of a new generator matrix cc;;", maintaining the decreasing order of their associated reliability values. The remaining n - k columns of cc;;' in reduced echelon form give the next 11 - k columns of G" arranged in order of decreasing associated reliability values. This process defines a second permutation n2. His clear that the code generated by cr;" 1s

r;

Rearranging the components of fi', according to the permutation sequence :I= (yo, Yl, · · · , Yk-1, _Vk, · · · , Yn-1), with I y =

YO

n2[ff']

n2,

we obtain the (10.72)

[>[ Yl [> · · · >[ Yk-1 [, and I Yk [> · · · >[ Yn-1 [. It is clear that = n2[ni[rr]]. The first k components of y are the k MRIPs. These k MRIPs

are said to form the most reliable basis (MRB) for code C. We can permute the rows of G" to obtain a generator matrix
/Jl.O

PO.n-k-1 Pl.n-1,-1

1

Pk-l,O

~k-1 n-1,-1

PO.O

]

(10.73) '

where Ilk represents the k x k identity matrix, and ]l' is the k x (11 - k) parity-check matrix. The code C 1 defined by
424

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

By analogy with the MRB, we define the least reliable basis (LRB) as the set of the n - k least reliable independent positions (LRIP) in the dual code CJ_ of C. Then, it is shown in [19] that the positions not included in the MRB form the LRB. Because the complement of an information set is an information set of the dual code, it follows from (10.73) that )H[i = [PTJl.,,_k] is the parity-check matrix of the code C1. Consequently, if n - k < k, we can determine G 1 after constructing H1 in the dual code CJ_. In this case, the ordering is from right to left in increasing order of reliability values. This process also defines two permutations µ1 and µ2 corresponding to the ordering of the received sequence in increasing order of reliability values and the determination of the n - k first independent positions of this ordering, respectively. It follows that at most n · min{k, n - k} 2 binary additions are performed to construct G1. Also, although the received sequence can be ordered with O (n log2 (n)) comparisons if unquantized values are considered, the ordering corresponding to n1 ( or µ1) becomes trivial if quantized values are used, since only the numbers of quantized values corresponding to each quantization level suffice to determine the complete ordering of the quantized received sequence. 10.8.2

Priority-first Search Decoding AJgorithm

As shown in Chapter 9, every linear block code can be represented graphically by a trellis. Another useful graphical representation of a linear block code is a tree. Every binary linear systematic (n, k) block code C can be represented by a binary tree T, as shown in Figure 10.7. This binary tree has the following structures: 1, T consists of n + 1 levels. 2, For O :s i < k, there are 2; nodes at the ith level of the tree. Fork :s i :s n, the number of nodes at the ith level of the tree is 2k. There is only one node so at nth level

I

i i

Terminal nodes

I I I I

kth level· 0 1

0 1

0

0

second level

0 1

0

0

first level

0

~ 1

zeroth level · s0 (Root)

FIGURE 10.7: A tree representation of a binary linear systematic block code.

Section 10.8

Reprocessing

Algorithms

~25

the zeroth level of the tree called the initial node the of the tree, and there are 2" nodes at the nth level of the tree. which are called the terminal nodes of the tree. 3,, For O :': i < k, there are two branches every node s; at level-i and connecting to two different nodes at + 1). One branch is labeled with an information 0, and the other branch is labeled with an information symbol 1. Fork < :S n, there is one branch leaving every node s; at level-i and to one node at level-(i + 1). This branch is labeled with either O or 1. a ~., The label sequence of a connecting the initial node so to a node Sk at the kth level to an information sequence un of k bits. The label sequence of the the initial node so through a node sk at the kth level to a terminal node sn at the nth level is a codeword in C. The label sequence of the tail node sk to node sn to the n - k of the codevvord. --·~-"'"" definition of a code tree, it is clear there is a one-to-one "'~"·"~"·"~ between a codeword in C and a connecting the initial node so to a terminal node s11 • The set of nodes at the ith level of the tree is called the node ( or space, denoted by (C). !E){AMfLIE W.3l

Figure 10.8 generated

the tree representation of the systematic (6, 3) linear code

(G

[

1 0 0 0 1 1 0 1 0 1 0 1 0 0 1 1 1 0

l

The tree of a linear block code can be used to facilitate decoding. One such tree-based decoding is the priority-first search (PFS) decoding algorithm devised by Han, Hartmann, and Chen [20]. This algorithm processes the tree T1 of the permuted code C1 generated by the generator matrix G 1 given in (10.73), where the first k levels of the tree correspond to the k MRIPs of r. The basic concept of this algorithm is to penetrate the tree from the initial node so, level by level, to search for the ML path. At each level, a list of paths is and the most likely path on the list is extended. Because the starting part of the tree corresponds to the most reliable positions and contains only a smaller number of nodes at each level, it is possible to direct the search of the ML path in a particular highly reliable direction and hence keep the size of the list and the effort small. Each time the kth level of the tree is reached, a candidate codeword is generated and tested. The candidate codeword with the least discrepancy is kept. For each test, all the partial paths on the list with correlation discrepancies larger than this least correlation discrepancy are discarded from the list. This process is repeated until the list becomes empty. Then, the stored candidate codeword v''' with the least correlation discrepancy is the decoded codeword and is

426

Reliability-Based Soft-Decision Decoding Algorithms

Chapter 10

0

0

0

1

1

0

0

0

0

0

0

0

0

0

1

0

0

0

1

0

0

FIGURE 10.8: A tree representation for a systematic (6, 3) linear code.

the ML codeword in C1 with respect to the permuted received sequence y. Then, v = rr 1[rr 1[v*]]

1 2

is the ML codeword in C with respect to r. The PFS decoding is carried out based on a cost function f (s;) associated with each states; E :E; (C). This cost function is a measure of the correlation discrepancy of any path that starts from the initial node so, passes through the node s;, and ends at a terminal node of the tree. It is used to guide the search of the most likely path in the code tree along the most likely direction. Decoding is performed on the ordered received sequence y = rr2[rr1[r]] = (yo, Yl, · · · , Yn-1). Let z = (zo, z1, · · · , z11 _1) denote the hard-decision vector obtained from y. For 1 :::: i :::: n, Jet p(s;) ~ (vo, v1, · · ·, v;-1) denote the path in the code tree that connects the initial node so to nodes; at the ith level. We define the binary sequence v(s;) = (vo(s;), v1(s;), · · ·, v11 _1(s;)), where v1(s;)

=

V/ {

Z!

for O:::: l < i, for i :::: l < n.

(10.74)

Therefore, p(s;) is a prefix of v(s; ). Note that v(s;) and z differ only within the positions in {O, 1, · · ·, i - l}. Let D1(v(s;)) denote the set of positions where v(s;)

Section 10.8

and iz diffeL correlation

H follows from the definition of that

)) = For 1 :::= i :::= k. let wi any of the node of the code tree T1 that have ( vo, v 1. r1n,or~o frorrt node Si at level-i. 1rhese

··· .

of n at a terminal v; -1) as the common arid to 2k-i cod.e,-1NOTdS i-1i1

·where ('i) denotes the set of definitions of ) and ,;/ that C

From for 1 :::= i :::= k,

).

we obtain the

and

101:Ne:t

bound on

) ) :::=

The

wi) can be further based on ideas in Section. 10.30 Let Uh b~ th_e inforrnation seque11cce defined by u1 = z1 for O < l :::= k - l; that nu 1s fon11ed the first k-bHs of the hard-decision received vector iz. From fili., we form the cocle-word ;y = ., which called the initial seed codeword. Because the informai:ion rn of vis (zo. z1. · · · , Zk-1). {k. k + L · · · . n - 1). Based on the {wo = 0. w1. wz. · · ·. w, 11 } of C. we define o(s;)= min {lw,-11 )) Il. IE{O,m}

where11(w) = I 1.and11 )) = ID 1 (v(s;))I.Sincevdiffersfromz;inn and w(s;) differs from iz in n (v(s;)) MRPs, 8 ) the rninimum number of additional in which iz and ) must differ for to be a valid codeword. l.o',Ner bound on ,,/): /i(s;)

v(s;))

+

IY11-1 I. l=l

We define /5(s;) l::::.

J(s;)

=

'\\-°'\

A(y. w(s;))

+L

IYn-11,

1=1

which is called the cost function of node s;. This cost function consists of two parts. The first of f(s;), ),,,(y, the contribution from the

428

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

common prefix (vo, v1, · · · , v;-1) of all the 2k-i codewords (or paths) diverging from nodes;. The second part of f(s;), L[~'21Yn-tl, represents a lovver bound on the contributions from a tail (v;, v;+ 1 , · · ·, v11 _1) of a codeword diverging from nodes;. The cost function f (s;) simply says that each codeword passing through node s; has a correlation discrepancy of at least f(si). For two different nodes, s; and SJ, in the tree, if f (s;) < f (s j ), then it is reasonable to assume that a path passing through nodes; is more likely to produce the received sequence y than a path passing through nodesi. Therefore, a higher priority should be given to nodes; to penetrate the tree in search of the ML codeword. This assumption forms the basis of the PFS decoding algorithm. An important property of the cost function f (s;) is that along a path from node so to nodes;, it is a nondecreasing function of the tree level i. For O ::::: i < k, consider the two nodes si~1 and s;~1 in Z:.:;+1 (C) that are adjacent to nodes; E I:; (C) with a cost function f (s;). Without loss of generality, we assume that the label bit for the branch (s;, si~1 ) is identical to z;, and the label bit for the branch (s;, s;(~1 ) is z; + 1, the complement of z;. Then, it follows from the definition of v(sj) that v(si) = v(si~1 ), and v(s/~1 ) differs from v(si~1) only at the ith position. We readily see that (1) (10.82) f(si+l) = f(s;). From (10.81) we can compute f (s;~1 ). It follows from (10.79) and the fact that v(si~1) and v(si~1 ) differ only at the ith position that (10.83)

The equality of (10.82) and the inequality of (10.83) imply that for a path from node so to node s;, 1 ::::: i ::::: k, the cost function f (s;) is a nondecreasing function of the tree level i. For k ::::: i < n, there is only one branch leaving a node s; E I:; ( C) and connecting a node s;+ 1 E b;+l (C). Therefore, vi = vk for i ::: k. In this case, the cost function f (s;) is simply f (sk) = A(y, vk); that is, .f(s;)

=

f(s1c)

=

A(y, vk).

(10.84)

The PFS decoding algorithm is devised based on the cost function f (s;) to penetrate a code tree using a depth-first processing procedure. This algorithm continuously updates a list of nodes in the tree that have been visited (or reached), and their cost functions. This list is called the cost function list ( CFL). The nodes are ordered in increasing values of their cost functions, such that the first (or top) node of the list has the smallest cost function, and the last (or bottom) node of the list has the largest cost function. The first node on this ordered list has the highest priority to be chosen as the location (or the point) for penetrating the tree in the search of the most likely path. The algorithm executes the following steps: lo Construct the initial seed codeword w. Compute the cost function f (so) of the initial node so and form the initial cost function list CF Lo= {so, .f(so)}. Go to

step 2. 2. Check the first node s; on the list and remove its cost function f (s;) from the list. If i < k, go to step 3; otherwise, go to step 4.

Section 10.8

Reprocessing

Algorithms

::t Extend the nodes; into its two adjacent nodes, si~1 and

si~

1 • at

level-(i

42:9J

+ 1) of

the tree. Compute the cost functions f (s;~1) and f (s;~1) of s;~1 and si~1. and reorder the list. Go to step 2. 4L Hi = k, evaluate the cost function f(sk) = >,_(y, 1/) and record the codeword wbesr that has the smallest cost function among an codewords tested so far. Delete all the nodes on the list whose cost functions are larger than >,_(y, vbesr ). H the list is not empty. go to step 2; otherwise, go to step 5. §. Output 'l!IJesr as the decoded codeword, and stop the decoding process. Note that in step 4, nodes on the list with cost functions larger than >,_(y. 'lfbesr) can be deleted owing to the monotonicity of the cost function. The codewords passing through these nodes do not contain the ML codeword and hence should be removed from further consideration. Also, (10.82) implies that for i < k. si~1 is always put at the top of the list. Because the algorithm is to minimize the cost function (correlation discrepancy) of a codeword at level-k, the output 'lfbesr is the MLD codeword. IEKAMIP'llE 1illl.4\

l

Consider the (8, 4. 4) RM code with weight profile W = {O. 4, 8} and generator matrix 1 0 0 0 0 1 1 1 0 1 0 0 1 0 1 1 (10.85) CG= 00101101. [ 0 0 0 1 1 1 1 0 Suppose the all-zero codeword w = (0 0 0 0 0 0 0 0) is transmitted, and

= ( -1.2. -1.0. -0.9. -0.4, 0.7, -0.2, -0.3, -0.8) is received. In decoding ir, the PFS algorithm first orders the symbols of decreasing reliability values, so we obtain

ir

in

(10.86)

= ( -1.2, -1.0. -0.9, -0.8, 0.7, -0.4, -0.3, -0.2 ).

l

Permuting the columns of CG based on the permutation n1, we obtain the matrix

I

CG =n1[CG]=

0 0 0 1 0 ~1 0 1 0

0

1 0 0 1 1 0 1 1 0 0 1 1

1 0 1 1

n

(10.87)

First, we notice that the first three columns of CG', g~, g~, and g~. are linearly independent; however, the fourth column, g~, depends on the first three columns. The fifth column, g~, is linearly independent of the first three columns. Therefore. g~, g~, g~, and g~ form the most reliable basis. fo 1/, the MRIPs are 0, 1. 2, and 4.

430

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

With inverse permutation n 11 , we find that in r the MRIPs are also 0, 1, 2, and 4. Adding the fourth row of G' to the second and third rows, and permuting the fourth and fifth columns of the resultant matrix, we obtain the following matrix:

G1

=

l

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

1 1 1 0

0 1 1 1

1 1 0 1

1 0 1 1

l .

(10.88)

The permutation n2 results in the following received vector,

(10.89) = ( -1.2. -1.0, -0.9, 0.7, -0.8, -0.4, -0.3, -0.2 ).

The hard-decision received vector z obtained from y is

z = (0, 0, 0, 1, 0, 0. 0, 0). We set ii= (0001). The initial seed codeword vis given by

v= iG1 = (0, 0, 0, 1. 0, 1, 1, 1). Comparing v with z, we find that D 1 (v) = {5, 6, 7}, and n(v) = [D1(v) I = 3. Because v(so) = z, D1(v(so)) = 0, and n(v(so)) = [D1(v(so))! = 0. From (10.79) and (10.81) we find that 8(so) = 1, and f (so) = 0.2, respectively. Then, the PFS algorithm for decoding y is initialized with the cost function list CF Lo = {so, f (so) = 0.2} (or simply, {!(so)= 0.2}). The decoding process is depicted by the tree shown in Figure 10.9. At the first step, so is extended to nodes s?) ands?) with connecting branches (so, s?l) = 0 and (so, s?)) = 1. Because D1 (v(s?l) = {O}, and n(v(sf 2l)) = 1, we find from (10.79) that 8(s?)) = 0. From (10.81) we find that f (s?)) = 1.2. Updating CF Lo, we have a new (1) (1) (2) (2) · (1) (2) CFL = {s1 , f(s 1 ) = 0.2; s1 , f(s 1 ) = 1.2} (or simply, {f(s 1 ) = 0.2; f(s 1 ) = 1.2}). The rest of the decoding steps are summarized in Table 10.2. From this table we find that MLD is achieved in six steps. The ML codeword is '¥best = (0 0 0 0 0 0 0 0) with correlation discrepancy A(y, '¥best) = 0.7. The PFS algorithm performs MLD efficiently for binary linear block codes of lengths up to 128 at medium to high SNR; however, at low SNR, the number of computations required by the algorithm to achieve MLD may become prohibitively large, especially for long codes, owing to the very large number of nodes in the tree to be visited and extended, which results in a very large cost function list. The worst-case scenario is that all the 2" nodes at the kth level of the tree must be visited and their cost functions computed. To overcome this problem, a suboptimum version of the PFS algorithm is presented in [21 ], in which a limit is put on the

Section 10.8 Decoded

(s 11 • 0.2)

FIGURE 10.9:

TABLE 10.2: Decoding l

Ii)

·\+i ?

IDu Iv

I )I

ii) (v(si+i)

(vls121 R i+l

for PFS algorithm.

ols;~ 1 )

Tbest

01sf+1)

;\.(y. VF,es1)

CFL

§le!l'

Si

s;_H

}l'(sf+lll

1

I"()

ill '1

(0)

0

1

-

(1)

1

0

-

(00)

0

l

-

Ii):

(01)

1

0

-

= ((J2: 1.0: l.2}

s,..,111

(000)

0

1

-

l\1

'3

(001)

1

0

-

=

'4

(0001)

0

1

-

'4

(0000)

1

0

-

(2)

·'1

2

(I) .\'!

111

\·2

-

3

(I)

s2

4

ill s,.,

5

ill s..J.

·'

·'

ii)

-

-

6

-

-

-

(00010111) 0.9

-

-

-

-

-

(00000000)

-

-

-

-

0.7

1/1.,:

11

1: /ls:

21

1)

= \0.2: 1.2) .

(2)

I ls 1 I)

(2)

): /(s I

I}

(0.2: 0.9: l.O: l.2} (2)

{fls~I I): /(s4 =

I:

I 1,:21 l l

):

(0.2: 0.7: 0.9: l.O: l.2}

u1.{ 11 =

o.7J

I

VI

size of the cost function list. This suboptimum version provides a good trade-off between error performance and decoding complexity with a proper choice of the maximum size of the cost function list. A very large number of computations can be saved at the expense of a very small degradation in performance, say a few tenths of a decibel.

432

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

The ancestor of the PFS algorithm was initially proposed by Dorsch in [22]. A clever implementation of this algorithm based on k lists was presented by Battail and Fang in [23]. These algorithms can be viewed as using the simplified cost function f(si) ~ A(y, v(si)) in (10.81) and, as initially proposed, are therefore less efficient than the PFS algorithm. Recently, Valembois and Fossorier proposed a computational improvement of the Battail-Fang algorithm in [24]. In this method, a second ordered list of the visited nodes is updated, so that every time a node is visited, only one node is extended. As a result, this method requires less memory than the PFS algorithm and often succeeds in decoding blocks that result in memory overflow with the PFS algorithm. Its average computational complexity is slightly greater than that of the PFS algorithm but its variance is much lower in general. One drawback of the PFS algorithm and similar algorithms is that the order in which the error patterns are processed in the list depends on each received sequence. In the next section, an algorithm with a fixed order of error-pattern processing is presented. This may represent an interesting feature for VLSI realizations of MRIP-reprocessing algorithms. Furthermore, it was shown in [25] that this structured reprocessing results in only a very small increase of average computational complexity with respect to that of the PFS algorithm. 10.8.3

Orrdered Sil:ail:istk Decoding Algorithm

MLD is, in general, achieved at the expense of great computational complexity, especially for long codes; however, if we do not insist on achieving optimum MLD performance, efficient soft-decision decoding algorithms can be devised to achieve near (or practically) optimum error performance with a significant reduction in decoding complexity. Consider a suboptimum soft-decision decoding algorithm A. Let P(A) be the probability that algorithm A commits a decoding error, whereas MLD is correct. Let Ps denote the block error probability of MLD as defined in (10.15). Then, the block error probability Ps (A) of algorithm A is upper bounded by Ps(A) ::S Ps

+ P(A).

(10.90)

~ Ps. In this case, algorithm A practically achieves the same error performance as MLD. Clearly, it is desirable to devise such a practically optimum decoding algorithm with a significant reduction in decoding complexity. One such decoding algorithm has been proposed in [26]. This algorithm is a MRIPreprocessing algorithm that is devised to process the MRIPs progressively in stages based on the joint statistics of the noise after symbol reordering. These statistics are also used to evaluate the error performance achieved after each stage and therefore allow us to determine when practically optimum error performance or a desired level of error performance is achieved. This algorithm is referred to as an ordered statistic decoding (OSD) algorithm. The OSD algorithm reprocesses the ordered received sequence y given by (10.72) based on the permuted code C1 generated by the generator matrix G1 given by (10.73). Because the first k symbols of y are the k most reliable independent symbols, their hard decisions should contain very few errors. Based on this concept, the algorithm generates a sequence of candidate codewords for testing by processing

If P(A) << Ps, then Ps(A)

Section 10.8

the k most reliable -with y is the decoded codewonl least correlation the decoded codeword in c;. Fm- 0 < < le, the OSD executes the of the l( first k forn1 an inforn1ation sequenc~ @o. 2, Construct the codeword the correlation ~1" " " ' ' . " ' ' " ' :1t For 1 s l s i. nrnke all form a nev, information sequence mt Generate each y

is referred to as the

and continue to The recorded codeword

until the is the decoded

'!/,est "Y/mt

codeword. The OSD

of order-i consists of (i of a total of

+ 1)

and

candidate codewords to make a of order-k is MLD. which codewords. This is not -what v,rs want As out earliec the k first of y are the k most reliable indeand their hard decisions rnosi: contain very few errors. That is to say that the information sequence illlO contains very few errors. Conseof !U!O most all will produce the ML codeword. with a small achieve the MLD error "D.rh~n,-i~,cc,,•-/•"· order-i should that for most block codes of up to 128 and n:1tes k / 11 > an order

!.

of io ~ Ld,nin/4J is to achieve the san1e error nD.,-u.c••·" MLD for bit-error rates than 10- 6 . fo this case. the two errnr n,c,•· 11·,-w,-,-, curves fall on of each other. lf io Ld, 11 i11 /4J is smaller thank. then 1+ + ... + is much smaller than 2k. order-io results in a tion in computational to generate the candidate codewords for easy to implement. 8) RM Figures 10.10 and 10.11 depict the bit-error code and the (128. 22) extended BCH code with OSD of various orders. respectively. For the (64. 42. 8) RM since d,nin = 8. the order of to achieve practically optimum error is io = L8/4J = 2. ~u-c,v·~u. in Figure 10.10. the OSD algorithm of order-2 achieves the MLD error

m

C;)

434

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

1011 ~ - - - ~ - - ~ - - - ~ - - - ~ - - - ~ - - - ~

Simulations x: Order-0 + : Order-1 o: Order-2

10-1

10- 2

_q ]

"'

,J:J

...00.. ...0 ......

!

10-0

10-"

iii 10-)

\BPSK Soft-decision decoding (union bound)

10-6

\

, \

Order-2\ \Order-1

\ \

Order-0

\

10-7

0

2

4

8

6

10

12

SNR (in dB)

FIGURE 10.10: Order-i reprocessing for the (64, 42, 8) RM code.

('i

('i

performance. It requires generating 1 + 2) + 2 ) = 904 candidate codewords and computing their correlation discrepancies. Figure 10.10 also shows that the OSD algorithm of order-1 gives an error performance only 0.3 dB away from MLD at the BER 10- 5 . H requires generating only 43 candidate codewords. For the (128, 64, 22) extended BCH code, to achieve practically optimum error performance, the order of reprocessing required by the OSD algorithm is io = L22/4J = 5; however, as shown in Figure 10.11, the OSD algorithm of order-4 is already within the union bound of the code. The OSD algorithm of order-4 requires generating a total of 679,121 candidate codewords, which is much smaller than 264 . In these figures, tight bounds on the error performance of each reprocessing order are also included. These bounds correspond to (10.90), in which P(A) is evaluated based on order statistics following the methods of [26-28]. The number of candidate codewords to be processed by the OSD algorithm of order-i can be drastically reduced by using a sufficient condition on optimality derived in Section 10.3. At the end of each reprocessing phase before the ith (last) one, the best candidate codeword Vhest recorded is tested for optimality. If the sufficient condition on optimality is satisfied, then Vhest is the ML codeword, and decoding can be terminated. The sufficient condition based on one candidate codeword given by (10.31) can be improved by taking into account the structure of the reprocessing method. Assume that for I < i, phase-/ of order-i reprocessing has been completed. At this stage, all candidate codewords that differ from the harddecision received vector ~ in at most l MRIPs have been processed. The remaining candidate codewords to be processed must differ from z in at least (l + 1) MRIPs.

Section 10.8

Reprocessing Decoding Algorithms

43l5i

1011 10-1

~ . ~"0--~-~

b,, -~--.._;;

10-'

E

'

'"""

!'"

\' \ ' \

10-6

ii:i

'

'

\

10-8

o :

"'

\01de1-l

Order-4

".. ".. BPSK

',

'

\

~- "'

Order-0 ,

'\

'

\

- "-

·',"'._

\

\

\

~- \

\,

, 10-7

""

~

'

\\~, \ \

10-s

"', '"

*-

~'

o:Order-2 ,:, : Order-3

~<,--"'- "-

~

15Cl 10--1

...0

-- ._.,,_

~.~ ""\.

10-~

.D 0 '0.. """

Simulations x : Order-0 +: Order-1

''--,~

\

', ·"-

\

Order-2 \

\

Soft-decision decoding\\ Order) (union bound) \' Order- 4 10-9

\

10-10

1

2

3

4

5

6

8

7

9

10

11

SNR (in dB) FIGURE 10.11:

Order-i reprocessing for the (128, 64, 22) extended BCH code.

Based on this fact, the bound on optimality given by (10.31) can be

as

l+l

G(o/best, w1)

=

L

j=l

IYk-j

I+

(10.91) I

jE

D/i° \v1, ,1) 0

where (10.92) a'= max{O. dmin - [D1(w1,e,1)l - (l + 1)). 1 The first term of (10.91), :Z:: i;;,\ [Yk-j[, represents the minimum contribution associ-

ated with inverting at least (l + 1) MRIPs in :E, and the second term is derived in the same way as for (10.31). Comparing (10.91) with (10.31). we see that LRPs in (10.31) are replaced with MRPs in (10.91). Consequently, from Theorem 10.1 (or (10.33)), we see that (10.91) provides a stronger (or less stringent) sufficient condition for optimality than (10.31). Further computation savings can be achieved by improving the optimality threshold G(wbest, w1) of (10.91) in a number of ways. First, in evaluating (10.91), we can use a processed codeword v other than 'If best if G(w. w1) is larger than G(Vbest, w1). Second, if W!Jest c/= wo, we can consider the MRIPs of :E inverted in generating 'II/Jest in evaluating G(vtbest, w 1). Third, we may use a sufficient condition on optimality based on two candidate codewords, as in (10.46), in conjunction with the reprocessing of the MRIPs in z [29]. Finally, we can use the monotonicity of the reliability values in the MRIPs to discard candidate codewords within each reprocessing phase [26, 30]. The effectiveness of using a sufficient condition for optimality test to terminate the decoding process of the OSD algorithm is best demonstrated by an example.

436

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

~

0~

~

100

0

J

CJ)

,::

·;;:

/

95

/

"'u
0

1-,

0.
,

90

1-,

,....,

,/

(I)

"' ..c::

"'0.

,/

85

/

1-,



a"'

/

80

/

;:l

.§

0.. 0

/

75


"'

- - - :Test 1

/

'u 1-,

/

,/

/

70

"iJ
~ 1-,

0

:Test 2

+

:Test 3 :Test 4

X

65

*

0 ~

: Test 5


'u 0

u

60 1

1.5

2

2.5

3

3.5

4

4.5

5

SNR (in dB) FIGURE 10.12: Percentage of codewords declared optimum after phase-1 of order-2 reprocessing for the (24, 12, 8) Golay code.

Consider the (24, 12, 8) Golay code (presented in Chapter 4 ). For this code, an order2 OSD algorithm achieves practically optimum error performance. Figure 10.12 depicts the percentage of codewords that are declared optimum after phase-1 reprocessing for five different cases of applications of sufficient conditions on optimality. For the first three cases, optimality is determined based on one candidate codeword, and for the remaining two cases, optimality is determined based on two candidate codewords. In the first case, the optimality threshold G(v, w1) is evaluated based on (10.31) by considering the recorded codeword Vbesr at the end of phase-1 reprocessing. In the second case, G(v, w1) is computed from (10.91) for l = 1 based on 'V!Jesr. The third case is similar to the second case, except that G (v, w1) is evaluated based on the codeword that minimizes G(v, w1) of (10.91) among the 13 candidate codewords generated at the end of phase-1 reprocessing. For the fourth case, the optimality threshold G(v1, w1; v2, w1) is evaluated based on (10.46) by considering the two codewords that have the smallest correlation discrepancies among the 13 candidate codewords generated at the end of phase-1 reprocessing. The fifth case is similar to the fourth case, except that the optimality threshold G(v1, w1; v2, w1) is computed from a generalization of (10.91) for two codewords [29]. Figure 10.12 shows that (10.91) provides a more efficient optimality condition for terminating the decoding process than (10.31). In terms of cost effectiveness, case-3 is the most effective one, as it is easier to implement than case-5. For case-3, 83% of the time, the decoding process is terminated at the end of phase-1 reprocessing at the SNR of 1 dB. This result means that even at a low SNR of 1 dB, only 22

Section 1

candidate code,vords need to be DP)C(cssea on average for sequence.

a received

in 10.4 for Consider order-1 re10n;ct,ss:mg of the received sequence the (8. 4. 4) Rh/J[ code. At we find vo = V/mt = 0. 0. 1. 0. 1. 1. 1). and A(y. '"hesr) = 0.9. From (10.91) and we find that for I = 0, a' = 0. and must be carried into G(Vbest, w1) = 0.7. Because pn)cess1ng. At i[he end of we find the codeword w = (0. 0, 0. 0. 0. 0. 0. 0). with w) = 0.7. Since A(y, v) < V/,esr), we set 0. 0. 0. 0. 0. 0. 0). F:mm (10.91) and ,ve find that for l = 1. o' = 1. . w1) = 0.9 + 0.7 + 0.2 = 1.8. Because A(y. 'Y!,est) < 1.8. Vt,est is the MLD codeword.

a reduced

of few information sets becomes more efficient than rei:)roce~:srnLg of the MRB, with j < i. A of order-j re1pn)CE,ssmg of several information sets is in This method was further in [33], where an iterative information set reduction is used to achieve the error of + 1) reprocessing with a cost Finally. Valembois and Fossorier proposed closer to that of order-i a new version of OSD called "Box-and-Match'' (BMA) [34]. The BMA achieves the error performance of reprocessing with a ~~v~·~"''"" cost closer to that of order-i reprocessing but with a memory of the same order OSD has no memory fo contrast with the OSD which processes error patterns of each the BMA stores some information about the error patterns at j to be used at phase-/. with / > j. As a many error patterns processed at phase-/ by the OSD algorithm are discarded the BMA algorithm. and the BMA algorithm can achieve near-MLD of larger codes than the OSD algorithm. The OSD algorithm and the decoding algorithm are complemenin nature: the former is based on processing certain MRPs of a received sequence, and the latter is based on processing certain LRPs. This complementary feature allows us to combine the two algorithms to form a hybrid decoding that has the advantages of both algorithms. Consider the OSD algorithm with order-i reprocessing. If there are at least i + 1 MRIPs of the ordered received sequence in error. but fewer than t + 1 errors have occurred outside the Ldmin/2J LRPs. then the OSD algorithm with order-i reprocessing will result in an incorrect decoding; however. Chase algorithm-2 will result in a correct decoding. On the other hand. if the number of errors in the LRPs is large, but the number of errors in the

438

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

MRPs remains less than i + 1, Chase algorithm-2 will fail to decode the received sequence correctly, but the OSD algorithm with order-i reprocessing will result in a correct decoding. Based on these results, we can combine these two algorithms to process the received sequence in parallel. At the end of the processing, there are two candidate codewords, one produced by the Chase decoder and one produced by the OS decoder. These two final candidate codewords are then compared, and the one with the larger correlation (or smaller correlation discrepancy) is chosen as the final decoded codeword. It has been shown in [35] that for i < Ldmin/4J, this hybrid decoding improves the error performance of the OSD algorithm with order-i reprocessing at medium to high SNR. The OSD algorithm also can be combined with the PFS decoding algorithm presented in Section 10.8.2 to limit the size of the cost function list. In the PFS decoding algorithm, only the cost functions f (s;) that correspond to inverting at most i MRIPs can be added to the cost function list, either from the beginning of the reprocessing or at a later stage to first exploit the dynamic nature of the PFS algorithm. In general, a probabilistic decoding method is associated with two related problems: (1) converging as quickly as possible to the optimum solution and (2) recognizing the optimum solution that has been found and stopping the decoding. For MLD, the PFS algorithm converges faster than the OSD algorithm to the optimum solution owing to its dynamical behavior; however, the structured reprocessing strategy of the OSD algorithm allows us to derive a less stringent sufficient condition for optimality of the decoded codewords. These issues are elaborated in [25]. In this section we described a probabilistic decoding method that achieves either a desired error performance or practically the optimal error performance of MLD for a given range of SNR values. Also using a probabilistic approach, Dumer proposed in [36] a general approach for soft-decision decoding that, at most, doubles the block error probability Ps for MLD at all SNR values. This method has also been extended to MLD [37]. The decoding complexity of this method is shown to be less than that of trellis-based MLD, suggesting that probabilistic soft-decision decoding algorithms seem to represent a promising approach for optimum or near-optimum decoding of block codes with lengths of hundreds of bits. 10.8.4

Syndmme-Based Ordered Statistic Decoding Algorithm

The OSD algorithm can be formulated in terms of syndrome like the syndromebased MLD decoding algorithm presented in Section 10.7. Consider the parity-check matrix H1 = [PTI 11 _k] for the code C1 generated by the permuted generator matrix
zHf

s1 = (s1,o,

s1, 1,

· · · , s1.11-k-d

be an (n - k)-tuple obtained by taking the transpose of the sum of the I vectors in QI· Let §2 = § + § l · We form the following error vector e = (e1, e2), where e1 = (e1.o, eu, · · · , eu-1) with eu = 1 if j E Q1, and e1,i = 0 otherwise, and e 2 = s2 . Then, v = z + e is a codeword in C. MLD is achieved by finding the error vector e* such that v* = :z + e* has the smallest correlation discrepancy with y.

Section

OSD ]_,

of order i consists of the

the

. For ea8h set Z, For 1 ::= l ::= i, form all sets of thi0 -vector form the vector sum of the cohnnns. Take obtain §1, Frorn Band §1,, §2 == s + 51 and 8 ·== Fer each error vector e, fo;_-m the codeword = c: + z-; and evalunte its cortelaticE s111a1lest Find and record the code\;;;o•id and The recorded codevvord

~'best "'1,e.11

until the the decoded

codeword. This formulation of order-i search for the CHfiri!oor"~-

In an the

i:nforn1ation of the received

and methods are suitable for information to decode RM codes.

M'-'~~·~
·mi.~. 1l

Msi]«lira'()f-l©lfl]ali:

~~cuivo•c,nH

CH,CH"0[C,vOcU UL•vLrLUIHf',•

vvF: consicler

,,,,,·,vcYlVH

(fr'uSlu'Uii®J

< Jll., In Section 4.2 we showed tlwt for any rth-order RM i . SU.J)distance A1NGH channel. For code is used for error control over i.nto a a codeword v = (vo, v1, · · · , Vn-1) is sequence ir = (co, c1, · · · , c11 _1) with c; = - L After au,ouJIJ'001>Ju, the received sequence at the output of the correlation detector is 11· hard-decision rr is converted into a with z; = 0 for r; < 0, and z; = 1 othenvise. For set Si of check-sums for each information bit IIJ ., 0 ::= j ::= k -- L is forrned. Each check-sum is a modulo-2 sum of the hard-decision decoded bits z.; of certain received r; (see For RM the sets of bits in any two distinct check-sums are The check--smns in Si form a set of imfowe revievv the main estimates of the information bit u i. ue-·v'-,•C-nu,., of RIVI codes as detailed in Section but no·N of This we closely hard-decision decoding. 'U•CHCWLUUr~

440

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

For equiprobable signaling, the transmitted information bit u.i is decoded into its estimate Uj E {O, 1}, which maximizes the conditional probability P(Sj I u.i ). Let Sj = {Aj,l: 1::: l::: IS.ii}, where A 1,1 denotes the /th check-sum in SJ. Because the check-sums in Si are disjoint, we obtain ISJI

Iog(P(Sj

I u.i)) =

Llog(P(AJ.1

I UJ)).

(10.93)

1=1

An algorithm that maximizes (10.93) is referred to as a maximum a posteriori probability (MAP) decoding algorithm (or simply a posteriori probability (APP) decoding algorithm [38]). Let YI be the probability that the number of independent errors in the check-sum AJ,1 is odd. Then, it is shown in [38] that YI= P(Aj,1 =

~ [1 -

= 0 I Uj = 1) =

P(Ai,1

= 1 I Uj = 0)

n c1 _2

(10.94)

p; )] ,

iEB1(J)

where B1 (j) represents the set of positions associated with the digits constituting the check-sum Aj.l, and p; is the probability that the ith digit is in error.His clear that y1 is simply the probability that the check-sum Ai.I is different from the transmitted information bit ui. For the BSC with crossover probability p; = p = Q(j2Eb/ No), YI is the same for all! and independent of j. h follows from (10.93) that [38] ISJI

IS; I

~log(P(Aj,1 lui =1)) = ~(2AJ.1-l)log(l-Y1)· L,, 1=1

P(A-1 J,

I u1

=0)

YI

L,, i=l

(10.95)

Because log( (1 - YI)/ YI) is a positive constant, the decision rule is then IS;I

L(2Aj.l -1) :

(10.96)

0,

1=1

which simply compares

:z=)::i A.i.1

with IS.i1J2. If

:z=l::1

:z=)::1 A.i.1 1

>

ISJl/2, we set the

AJ.1 < ISJ 1/2, we set u.i = 0. Whenever estimate UJ of UJ to 1, and if AJ.1 = ISJ 1/2, we flip a fair coin to determine u.i, which results in an erroneous decision in half of the cases on average. This type of error dominates the error performance. The foregoing decoding is the conventional majority-logic decoding. We readily observe that this decoding method is exactly the same as the Reed algorithm presented in Section 4.2.

Li:'.i

10.9.2

Majoll"ity-logi«: Dewdi111g Based 0111 Reiiabiiity !rifol'matio111

For the A WGN channel model, p; is defined as [38] e-lL;I

Pi= 1 + e-lL;I'

(lU.97)

Section 10.9

Weighted Majority-Logic Decoding

441

where Li = 4ri / No represents the log-likelihood ratio associated with the received symbol ri. The optimum decision rule based on (10.97) becomes computationally expensive and depends on the operating SNR [38]. Using the ""'',_,,-,,vn-n

n

(1 - 2

Pi)~

(10.98)

1- 2 maxiEB1(i)Pi,

iEB1Cf)

we obtain

e--41r1 Imin I No YI~

. . , 1 + e-41111111111/No

where lrilmin = miniEB1(j){\r; \}. H follows from (10.95) that for the AWGN channel, (10.99) becomes the decision rule based on the ISJ I

l)2Aj.l - 1) In Imin ; 0.

(10.100)

i=l

In (10.100), each term of (10.96) is weighted with the reliability value of the least reliable digit in its corresponding check-sum. H :Z:::i~i (2Aj.1 - 1) lr1 lmin > 0, we

set the estimate ui of u i to 1, and if :Z:::!~i (2A1.1 - 1) In lmin < 0, we set 11i = 0. Whenever :Z:::!~i(2A1.1 -1) lnl 111 ; 11 = 0, we flip a fair coin to determine £ii; however, the occurrence of such events becomes less and less probable as the number of quantization levels used at the receiver increases. This decoding algorithm, referred to as weighted majority-logic decoding, was first proposed in [39].

Majority-logic-decoding >< :

10- 2

Conventional

,:, : Tie-breaking o :Weighted

.q ]

"' 10-> .n 0 !-; 0..

!-;

2

!-;

!

iii

10-s

Uncoded BPSK

2

3

4

5

6

7

8

9

10

SNR (in dB)

FIGURE 10.13:

16) RM code.

Conventional and weighted majority-logic decoding for the (64, 22,

442

Chapter 10 1011

Reliability-Based Soft-Decision Decoding Algorithms

~--~---~---~---~---~--~

Majority-logic-decoding x : Conventional

10-1

g

* :Tie-breaking o :Weighted

10-2

15

"'

..0 0 >-<

0-

10-1

>-< 0 >-<

...

!

&i 10-4

10-5

MLD

Uncoded BPSK

10-6 0

2

4

6

8

10

12

SNR (in dB)

FIGURE 10.14: Conventional and weighted majority-logic decoding for the (64, 42, 8)

RM code. As mentioned in Sectionl0.9.1, the error performance of the conventional majority-logic decoding is dominated by the event A J,l = ISJ 1/2. In such a case, we can break the tie by using (10.100) instead of flipping a fair coin. A simplified version of this tie-breaking rule is obtained by simply discarding the check-sum with AJ,l = IS11/2 in the Reed algorithm. the smallest reliability value whenever Such a test requires very few additional computations. Figures 10.13 and 10.14 depict the simulation results for conventional and weighted majority-logic decoding of the (64, 22, 16) and (64, 42, 8) RM codes, respectively. Conventional majority-logic decoding in conjunction with (10.100) to break the ties is also presented. We observe that both soft-decision approaches provide significant coding gains over conventional majority-logic decoding while maintaining its simplicity and decoding speed. Another version of weighted majority-logic decoding based on the parity-check sums computed from a parity-check matrix of a one-step majority-logic decodable code will be presented in Chapter 17.

Li~{

Li~i

10.10

10.10.1

ITERATIVE RIEUABiUTY-BASIED DIECODiNG OF ONIE-STIEi? MA.IORITY-LOGiC DIECOIDABLIE CODES

iterative MAP-Based Decoding Suppose a one-step majority-logic decodable code (see Chapter 8), or a low-density parity-check (LDPC) code (an LDPC code is defined as the null space of a

Section 1

rn

Decodable Codes

443

matrix Fl with low density of nonzero entries that a sparse matrix [45]) (see 17) is used for error control over an AWGN channel. this code can be decoded based on the approach presented in Section 10.9.2. the computes the a I v;) associated a p; with each bit v;, 0 :::: i :::: n - 1, from the based on (10.93) and Because for the classes of codes considered, this a natural extension is to use the set of a the entire probabilities p; and to iterative method based on the Let lmax denote the maximum number of iterations to ]_, Construct the hard-decision received sequence 7l from rr. Z, Initialize the iOl 3,,

p;0l = p;, for i = 0, 1, ....

process by

11

-1, and

=?l.

For j

=

1. 2 .....

,[l,

compute hOJUl

=

p;,~

as the vector of a error using [l)U - l) as the vector of a priori error probabilities; "' if p then set z(j) = z(j-ll ffi 1 and p(j) = 1 p(j) for i = ILJ!, 1 ·, ·, W 1 1 ' 0. 1. ... , n - 1. till,

(p~j). Pij), · · · ,

Set the estimated word v*

=i

1)

111 "" l.

In this iterative the average number of iterations can be reduced checking at the end of each step j whether the syndrome computed from i.il is zero. fa this case, we can set v* = 72Ul. For the in Section 10.9.2 based on (10.100), we obtain the

JL Construct the hard-decision received sequence 7l from rr. 0

2, Set lr/ li = lr;I, i = 0. 1. ... , n - l, and i 0l = ;z, 23, Fo1· j = l, 2 ..... lmax, for O :::: i :::: n - 1. and for 1 :::: I < IS; I, check-sums /l;.1 based on the values z;i-lJ and identify

C'r.m,'1,iCP

the

. {I r11iu-i) I} . lr;.·1
(0)

'\ , -

(j-1)

r1 = Ir; I+ L)Au - Au) lr1. 1

lmin,

1=1

where A;.1 represents the complementary value of the check-sum A;.z. If < 0 then set /fl = /J-l) EB 1.

,.ui l

'

I

l

'1!, Set the estimated word v* =

i

1111 axl.

This algorithm represents the simplest implementation of iterative soft-decision decoding of either one-step majority-logic decodable codes or LDPC codes. Because

444

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

this algorithm is independent of the operating SNR, it is referred to as the uniformly most powerful (UMP) MAP-based decoding algorithm. Other iterative decoding algorithms based on the same generai approach have been proposed in [40-42]. In general, such algorithms converge with relatively few iterations (say, between 10 and 20) but have a nonnegligible error performance degradation with respect to optimum MLD or MAP decoding owing to the increasing correlations between the decisions made at each step. To reduce the effects of correlated decisions, it was proposed in [43] to combine this approach with variable threshold decoding. In the algorithm of [43], only the most likely errors (i.e., those whose associated a posteriori probabilities are larger than a certain threshold) are corrected, and the variable threshold is reduced during the decoding iteration process. As a result, the error performance improves at the expense of a larger number of iterations. A second approach, which almost eliminates correlations between decisions made at successive steps, is presented next. 10.10.2

Iterative Elle!ief-Prnpagation-lBased Decoding

We first use a simple example to illustrate the correlations between decisions made at successive iteration steps. Consider three check-sums defined by three codewords of Hamming weight 2, v1 = (v1,o, v1,1, · · · , v1_ 11 _i), v2 = (v2.o. v2,1, · · · , v2_ 11 -1), and '¥3 = (v3,o, v3,1, · · · , v3, 11 _i), in the dual code, with vi,; = v1,1 = 1, v2,; = v2,x = 1, and v3,1 = v3,y = l. At the jth iteration, the general algorithm presented in Section 10.10.1 computes

P(j) =

j(p;O))

+ (!(p}i-1)) + f (py-l)))'

(10.101)

=

j(pjO))

+ (!(p;f-1)) + f (p~j-l)))'

(10.102)

p;f)

where](·) and f (-) are two functions defined by the specific algorithm considered. Similarly, at the (j + l)th iteration, we have Plj+l)

=

f(p;O))

+ (f(pjil) + f(piil))

=

i(pjO))

+ (! (f(p;O)) + f(p;i-1)) + f(p/-1))) + f(pij))).

(10.103)

Because the a posteriori probability pfil for bit vi at the jth iteration is evaluated based on some information from bit v;, as shown in (10.102), this value becomes correlated with p}°l. As a result, p;O) and can no longer be assumed to be independent when used to evaluate as shown in (10.103). To overcome this problem, for each position-i, 0 :Si :Sn - l, [S;[ a posteriori probabilities pj~) rather than one are computed at the jth iteration, where for

Pii+l),

Piil

A;.1 E S;, Piil r~presents the a posteriori probability obtained by considering the [S; [ - 1 check-sums other than A;.1 in S;. For simplicity, in this particular example we represent instead with the a posteriori probability obtained by discarding

PiY

One-Step Majority-Logic Decodable Codes

Section 10. 0

445

the check-sum of Si containing bit v1. As a result, (10.101) and (10.102) become

Pi)= f(p;O)) + f(p;\-1)), /il = Jc/o)) + Jc/i.-1\ Pi)= j(pjO)) + f(P; 11-l\ P/:7} = fcpjO)) + f (p;_r 1\ I

l,X

/,1

and (10.103) is replaced with ( +l)

pl 1,/

=

(0)

·

( )

f(p I ) + f (p ,\,I J ),

/i+l) = Jc/o)) + 1 c/i.l) I

l..\

(10.104)

/,1

= j(p;O))

+f

(f(pjO))

+ f(p;:~1-1))) ·

Hence, no correlated values appear in (10.104). Ha decision about bit made at the (j + l)th iteration, then the a posteriori probability

Vi

has to be

/f+ 1) = Jc/0l) + J c/n) + J c/i)) I

I

l,1

X,I

is considered, as it contains all available information evaluated separately about bit Vi. The general form of this algorithm is known as the belief propagation (BP) algorithm and was introduced in [44]; however, the application of this algorithm to decoding LDPC codes can be found in [45] and was recently described as an instance of the BP algorithm in [46-48]. A good description of this algorithm can be found in [46]. The BP approach also can be applied to the decoding of one-step majority-logic decodable codes [50]. For LDPC codes, a simplified version of the BP algorithm based on the approximation (10.99) has been proposed in [49]. This algorithm can be summarized as follows:

z;

]_, For each i, 0 sis n - l, and each l, 1
z;

/ill,

Foreachi,O

s

i S n-landeachl,1 S ls ISil,evaluatethecheck-sums

Au = zi(j-1) EB

(

~ L

(j-1))

Z/,1

•

(10.105)

i'EB1(i)\i

and identify IriUll I min •

. {I r.,u-i)I} . = min i' EB1(i)\i

1

,/

(10.106)

446

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

b. For each i, 0::: i ::: n - l, and each l, 1::: l::: IS; I, compute rCi) = lr(O)I l, 1

l

+

vL..,

(--:Ai,l'

.

Ull min· - Ai,!' )I ri,l'

(10.107)

c. For each i, compute IS;I r;Ul

=

Ir;
+ v(-A L.., ;.z -

A i,l )I r;,1 Ull min·

(10.108)

i=l

d.

Form ,z(il z(J) I

ie.

=

=

(z(Jl z(Jl · · · z(Jl ) 0 ' 1 ' ' n-1 z
l

such that z(Jl ,

= z<, 0l if

r(J)

,

> 0 and '

•

For each i, 0 ::: i ::: n - 1, and each l, 1 ::: l ::: IS; I, evaluate z;~l such that z 0, and z(Jl = z(O) EB 1 if r(Jl < 0. , z,/ I 1,/ I,/ I 1,/ -

3. Set the estimated word to v*

= zUma.,l.

This algorithm is referred to as the UMP BP-based decoding algorithm, as it is independent of the operating SNR. In general, it improves the error performance of the algorithm presented in Section 10.10.1 at the expense of a larger number of iterations. Figures 10.15 and 10.16 show the bit-error performance for iterative 100

10-1

£' :B

...."' .. !

.D 0

~

10- 2

10-3 - - - - : UMP APP-based decoding

0

co

- -: UMP BP-based decoding - - : BP decoding

10-4

x : 50 iterations

10-j

10- 6

o : 200 iterations

'----'----'-----'----'-------'----'----'-----'----'

0

0.5

1.5

2

2.5

3

3.5

4

4.5

E 1,IN0 (in dB)

FIGURE 10.15: Error performance for iterative decoding of the (504,252) LDPC code with BP, UMP BP-based, and UMP APP-based decoding algorithms, and at most 50 and 200 iterations.

Problems

447

l 110-2 J3

'0"

.a

Cl.

E1,IN0 (in dB)

FIGURE 10.1 Error performance for iterative of the (1008, LDPC code with BP, UMP BP-based, and UMP APP-based decoding algorithms, and at most 50 and 200 iterations. '"""""""·"'uu, of the (504,

and (1008, 504) LDPC codes (constructed by cornputer with the BP, UMP BP-based, and UMP APP-based decoding algorithms, and at most 50 and 200 iterations. The (504, 252) LDPC code has three check-sums of 6 orthogonal on each and the (1008, 504) LDPC code has four check-sums of 8 orthogonal on each We observe that the error performance of the simplified UMP BP-based algorithm is close to that of the BP and achieves a significant gain over the UMP APP-based decoding algorithm of Section 10.10.1; hovvever, the number of iterations is quite large, and little improvement is observed by increasing the number of iterations from 50 to 200 for BP-based decoding algorithms. Construction of LDPC codes and various algorithms for decoding LDPC codes will be discussed in Chapter 17. l"~OlBLIEMSi ]_i!JJJl. Prove the sufficient condition for optimality of a codeword given by (10.47). ]_([D,2

Consider the value G(,11, w1; Y2, w1) in (10.46). Discuss G(w1, w1; w2, w1) in the case where the codeword delivered by an algebraic decoder is known. What is the problem in trying to use this result for all received sequences? rrD, Discuss G(w1, w1; v2, w1) for w1 = w2. GMD decoding considers only L(d111 in + l)/2J erasures in the d 111 ; 11 - 1 LRPs. Explain why not all dmin - 1 possible erasures are considered. !11,

]_il]lo]

448

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

10.4 Consider an (n, k) binary linear code with even minimum distance dmin· Show that it is possible to achieve the same error performance as for the conventional Chase algorithm-2 by erasing one given position among the Ld111 ; 11 /2j least reliable positions (LRPs) of the received sequence and adding to the hard-decision decoding of the received sequence ir all possible combinations of O's and 1 'sin the remaining Ldmin/2J - 1 LRPs. 10.5 Consider an error-and-erasure algebraic decoder that successfully decodes any input sequence with t errors and s erasures satisfying s + 2t < dmin and fails to decode otherwise. Define Se(a) as the set of candidate codewords generated by the algorithm Ae (a) presented in Section 10.4. For a = 1, · · · , rdm; 11 /21 - 1, show that Se(a) <; Se(a + 1). 10.6 In the KNIH algorithm presented in Section 10.6, show that any codeword v in J (i) rather that the one that has the smallest correlation discrepancy with the received sequence ir can be used for evaluating G; (v). Discuss the implications of this remark (advantages and drawbacks). Ut7 In the RLSD algorithm presented in Section 10.7, show that there exists at most one (n - k)-pattern that is not (n - k - 1)-eliminated. 10.8 For the RLSD algorithm presented in Section 10.7, determine the complete reduced list for the (15, 11, 3) Hamming code. 10.9 Determine the complete reduced list for the (8, 4, 4) extended Hamming code. Show that this complete list can be divided into two separate lists depending on whether the syndrome§ is a column of the parity check matrix H. (Hint: Each list is composed of five distinct patterns). 10.10 In the RLSD algorithm presented in Section 10.7, prove that all n(v)-patterns with n(v) > 11 - k can be eliminated from all reduced lists. For n(v') < n(v), determine an n (v' )-pattern that justifies this elimination. HHl Let C and C1 be the two codes defined in Section 10.8.1. Explain why if vis the decoded codeword in C1 , then n;-1n 21 [v] is simply the decoded codeword in C. 10.12 Prove that the most reliable basis and the least reliable basis are information sets of a code and its dual, respectively. 10.13 Prove that order-1 reprocessing achieves maximum likelihood decoding for the (8, 4, 4) RM code. 10.14 Which order of reprocessing achieves maximum likelihood decoding of an (11, 11 1, 2) single parity-check code? Based on your answer, propose a much simpler method for achieving maximum likelihood decoding of single parity-check codes. 10.15 Describe the types of errors that can be corrected by Chase algorithm-2, but not by order-i reprocessing. 10.16 Assume that an rth-orcler RM code RM(r, m) is used for error control. a. Show that all error patterns of weight at most t, as well as all error patterns of weight t + 1 with one error in a given position can be corrected. b. Assuming reliability values are available at the decoder, propose a simple modification of majority-logic decoding (Reed algorithm) of RM(r, m) RM codes in which the error performance can be improved based on (a). BIBLIOGRAPHY

1. J. G. Proakis. Digital Communications, 3d ed., New York: McGraw-Hill, 1995. 2. M. P. C. Fossorier, S. Lin, and D. Rhee, "Bit Error Probability for Maximum Likelihood Decoding of Linear Block Codes and Related Soft Decision Decoding Methods," IEEE Trans. Inform. Theory, IT-44: 3083-90, November 1998.

Bibliography

~4!ll

"Error Bounds for Convolutional Codes and Asymptotically Algorithm," IEEE Trans. Inform. Theory, IT-13: 260-69,

"--'"'"u,,uu,F,

April 1967.

4!, G.D. 1973.

Jr., "The 'Viterbi Algorithm," Proc. IEEE, 61: 268-78, March Jr., "Generalized Minimum Distance Decoding," IEEE Trans. IT-12: 125-31, April 1966.

liii. G. Einarsson and C. E. "A Note on Soft Decision Decoding with Successive "IEEE Trans. Inform. IT-22: 88-96, 1976.

7, D.J.

and M. B. Pursley, "An Improvement to Generalized-Minimum" IEEE Trans. Inform. Theory, IT-37: 167-72,

ii!. H. T. Moorthy, S. and T. Kasami, "Soft Decision Decoding of Binary Linear Block Codes Based on an Iterative Search Algorithm," IEEE Trans. IT-43: 1030-40, May 1997. I!)),

D. "A Class of Algorithms for Decoding Block Codes with Channel Measurement Information," IEEE Trans. Inform. Theory, IT-18: 170-82, 1972.

llilt M. P. C. Fossorier and S. Lin, "Chase-Type and GMD-Type Coset Decoding," IEEE Trans. Commun., COM-48: 345-50, March 2000. H. E. J. Weldon, "Decoding Binary Block Codes on Q-ary Output Channels," IEEE Trans. Inform. Theory, IT-17: 713-18, November 1971.

ll2. W. J. M. Fossorier, and S. "Quantization Issues for Soft-Decision Decoding of Linear Block Codes," IEEE Trans. Commun., COM-47: 789-95, June 1999. B, H. Tanaka and K. Kakigahara, "Simplified Correlation Decoding by Selecting

Codewords Using Erasure Information," IEEE Trans. Inform. Theory, IT-29: 743-48, September 1983. :TI.it T. Kaneko, T. Nishijima, H. Inazumi, and S. Hirasawa, "An Efficient Maximum

Likelihood Decoding of Linear Block Codes with Algebraic Decoder," IEEE Trans. Inform. Theory, IT-40: 320-27, March 1994. :TI.§, T. Kaneko, T. Nishijima, and S. Hirasawa, "An Improvement of Soft-Decision

Maximum-Likelihood Decoding Algorithm Using Hard-Decision BoundedDistance Decoding," IEEE Trans. Inform. Theory, IT-43: 1314-19, 1997. :TI.mi, J. Snyders, "Reduced Lists of Error Patterns for Maximum Likelihood Soft

Decoding," IEEE Trans. Inform. Theory, IT-37: 1194-1200, July 1991. :Il.7. J. Snyders and Y. Be'ery, "Maximum Likelihood Soft Decoding of Binary Block

Codes and Decoders for the Golay Codes," IEEE Trans. Inform. Theory, IT-35: 963-75, September 1989.

450

Chapter 10

Reliability-Based Soft-Decision Decoding Algorithms

Ut N. J.C. Lous, P.A. H. Bours, and H. C. A. van Tilborg, "On Maximum Likeli-

hood Soft-Decision Decoding of Binary Linear Codes," IEEE Trans. Inform. Theory, IT-39: 197-203, January 1993. 19, M. Fossorier, S. Lin, and J. Snyders, "Reliability-Based Syndrome Decoding of Linear Block Codes," IEEE Trans. Inform. Theory, IT-44: 388-98, January 1998. 20. Y. S. Han, C. R. P. Hartmann, and C. C. Chen, "Efficient Priority-First Search Maximum-Likelihood Soft-Decision Decoding of Linear Block Codes," IEEE Trans. Inform. Theory, IT-39: 1514-23, September 1993. 21, Y. S. Han, C. R. P. Hartmann, and K. G. Mehrotra, "Decoding Linear Block Codes Using a Priority-First Search: Performance Analysis and Suboptimal Version," IEEE Trans. Inform. Theory, IT-44: 1233-46, May 1998. 22. B. G. Dorsch, "A Decoding Algorithm for Binary Block Codes and J-ary Output Channels". IEEE Trans. Inform. Theory, IT-20: 391-94, May 1974. 23. G. Battail and J. Fang, "Decodage pondere optimal des codes lineaires en blocs" Annales des Telecommunications, AT-41 (11-12): 580-604, November-December 1986. 24, A. Valembois and M. Fossorier, "An Improved Method to Compute Lists of Binary Vectors That Optimize a Given Weight Function with Application to Soft Decision Decoding," IEEE Commun. Lett., CL-5: 456-58, November 2001. 25. A. Valembois and M. Fossorier, "A Comparison between "Most-Reliable-Basis Reprocessing" Strategies," IEICE Trans. Fundamentals, E85-A: 1727-41, July 2002. 26. M. Fossorier and S. Lin, "Soft-Decision Decoding of Linear Block Codes based on Ordered Statistics," IEEE Trans. Inform. Theory, IT-41: 1379-96, September 1995. 27. D. Agrawal and A. Vardy, "Generalized Minimum Distance Decoding in Euclidean Space: Performance Analysis," IEEE Trans. Inform. Theory, IT-46: 60-83, January 2000. 28. M. Fossorier and S. Lin, "Error Performance Analysis for Reliability-Based Decoding Algorithms," IEEE Trans. Inform. Theory, IT-48: 287-93, January 2002. 29. M. P. C. Fossorier, T. Koumoto, T. Takata, T. Kasami, and S. Lin, "The Least Stringent Sufficient Condition on the Optimality of a Suboptimally Decoded Codeword Using the Most Reliable Basis," Proc. IEEE Int. Symp. Inform. Theory, Ulm, Germany, p. 430, June 1997. 30, D. Gazelle and J. Snyders, "Reliability-Based Code-Search Algorithm for Maximum Likelihood Decoding of Block Codes," IEEE Trans. Inform. Theory, IT-43: 239-49, January 1997.

Bibliography J]_"

451

M. P. C. Fossorier and S. Lin, "Computationally Efficient Soft-Decision Decoding, of Linear Block Codes Based on Ordered Statistics," IEEE Trans. IT-42: 738-50, May 1996.

3'.t M. P. C. Fossorier and S. Lin, "Reliability-Based Information Set Decoding of

Linear Codes," IEICE Trans.

E82-A: 2034-42, October

1999.

32t M. Fossorier, "Reliability-Based Soft-Decision Decoding with Herative Information Set Reduction," IEEE Trans. IT-48: 3101-06, December 2002. 3•\io A. Valembo:is and M. Fossorier, "Box and Match Techniques Decision Decoding," to appear in IEEE Trans. Inform. 2004.

to Soft

3~L M. P. C. Fossorier and S. Lin, "Complementary Reliability-Based Decodings

of Binary Linear Block Codes," IEEE Trans. Inform. Theory, IT-43: 1667-72, September 1997. J(/iio I Dumer, "Suboptimal Decoding of Linear Codes: Partition Technique," IEEE

Trans. Inform. Theory, IT-42: 1971-86, November 1996. '!tl t Dumer, "Ellipsoidal Coverings and Error-Free Search in Maximum Likeli-

hood Decoding," Proc. IEEE Int. Symp. Inform. Theory, Boston, Mass., p. 367, August 1998. 3~, J. L. Massey. 'Threshold Decoding,' MIT Press, Cambridge, 1963. J{!J\ V. D. Kolesnik, ''Probability Decoding of Majority Codes," Prob!. Peredachi

Inform., 7: 3-12,

1971.

4fllt G. C. Clark, Jr., and J.B. Cain, Error-Correcting Coding for Digital Communications, Plenum, New York, 1982. t!lJLo W. Meier and 0. Staffelbach, "Fast Correlation Attacks on Certain Stream Ciphers" J. Cryptology, 1: 159-76, 1989.

Lucas, M. Bossert, and M. Breitbach, "On Iterative Soft-Decision Decoding of Linear Binary Block Codes and Product Codes," IEEE J. Select. Areas Commun., JSAC-16: 276-98, February 1998.

4!2;, R.

t!l:2lo K. Yamaguchi, H. Hzuka, E. Nomura, and H. Imai, "Variable Threshold Soft Decision Decoding," IEICE Trans. Elect. Commun., 72: 65-74, September 1989.

4•1lo J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Morgan Kaufmann, San Mateo, Calif., 1988. 4!§" R.

G. Gallager, Low-Density Parity-Check Codes, MIT Press, Cambridge, 1963.

4f!fiio D. J.C. MacKay, "Good Error-Correcting Codes Based on Very Sparse Matrices," IEEE Trans. Inform. Theory, IT-45: 399-431, March 1999.

452

Chapter 1O

Reliability-Based Soft-Decision Decoding Algorithms

47. R. J. McEliece, D. J. C. MacKay, and J.-F. Cheng, "Turbo Decoding As an Instance of Pearl's "Belief Propagation" Algorithm," IEEE J. Select. Areas Commun., JSAC-16: 140-52, February 1998. 48. F. R. Kschischang and B. J. Frey, "Iterative Decoding of Compound Codes by Probability Propagation in Graphical Models," IEEE I. Select. Areas Commun., JSAC-16: 219-30, February 1998. 49. M. P. C. Fossorier, M. Mihaljevic, and H. Imai, "Reduced Complexity Iterative Decoding of Low Density Parity Check Codes Based on Belief Propagation," IEEE Trans. Commun., COM-47: 673-80, May 1999. 50. R. Lucas, M. Fossorier, Y. Kou, and S. Lin, "Iterative Decoding of One-Step Majority-Logic Decodable Codes Based on Belief Propagation," IEEE Trans. Commun., COM-48: 931-37, June 2000. 51. Y. Kou, S. Lin, and M. P. C. Fossorier, "Low Density Parity Check Codes Based on Finite Geometries: A Rediscovery," IEEE Trans. Inform. Theory, IT-47: 2711-36, November 2001.

1

,f1"'iIJ

As discussed in L convolutional codes differ from block codes in that the encoder contains memory, and the encoder at any not on the at that time but also on some numbes; of A rate R = k/11 convolutional encoder 1nemory order m can be realized as a linear circuit with memory m; that remain in the encoder for an additional m time units after and k are small k < 11, the information sequence is divided into blocks of length le and the codeword is divided into blocks of length 1L In the case when k = L the inforrnc1tion sequence is not divided into blocks and Unlike with block mininmrn distances and the fo this we describe the convolutional -,,~~~"" the notation used to We encoders can be functions for convolutional distance measures for convolutional codes. Convolutional codes were fhst ietroduced by Elias [1] in 1955 as an alternative to block codes. VIozencraft and Reiffen [2] sequential method for convolutional codes ·with large constraint studies soon began to appear. In 1963 [3] method called threshold This a number of practical applications of convolutional codes to transmission over and radio channels. These two methods are discussed in detail V""'~w, 13. maximum likelihood (ML) for soft-decision decodalgorithm that \Vas ing of convolutional codes with The Viterbi algorithm, led to the application and satellite communication in the [S] introduced a maximum a algorithm for convolutional codes for the information bits. The BCJR algowith unequal a rithm has been applied in recent years to soft-decision iterative decoding schemes in which the a priori of the information bits change from iteration to iteration. These are discussed in detail in Chapter 12, and the associated iterative decoding schemes are presented in Chapter 16. In 1976 Ungerboeck and [6] introduced the concept of trellis-coded modulation using short constraint convolutional codes of rate R = k/(k + l) along with soft-decision Viterbi decoding" a more complete introduction, see

454

Chapter 11

Convolutional Codes

Ungerboeck [7].) These schemes form the basis for achieving high-speed digital transmission over bandwidth-limited telephone channels and are discussed in detail in Chapter 18. Ungerboeck's work spawned interest in the properties of feedback encoders for convolutional codes, since high-rate codes are most efficiently represented in systematic feedback form. Another application of systematic feedback convolutional encoders, called turbo coding, was introduced by Berrou, Glavieux, and Thitimajshima [8] in 1993. (For a more complete introduction, see Berrou and Glavieux [9].) This coding scheme, which combines a parallel concatenation of two systematic feedback encoders with iterative MAP decoding, is capable of achieving moderately low BERs of around 10-5 at SNRs near the Shannon limit. It is currently being considered for adoption in a variety of applications, including data transmission on digital cellular channels, deep-space and satellite communication, and high-speed digital magnetic recording. Turbo coding is discussed in detail in Chapter 16. Convolutional codes are covered in varying amounts of detail in many books on coding theory and digital communications. Recently, three books have appeared devoted solely to convolutional codes [10, 11, 12]. The book by Johannesson and Zigangirov [12] is the most comprehensive and contains a detailed analysis of all aspects of convolutional codes. 11.1

!ENCODING Of CONVOUJT!ONAL CODIES

We present the basic elements of encoding for convolutional codes using a series of examples. Encoders for convolutional codes fall into two general categories: feedforward and feedback. Further, within each category, encoders can be either systematic or nonsystematic. We begin by considering the class of nonsystematic feedforward encoders. !EXAMPLE 11.1 Encoder

A Rate R

= 1/2 Nonsystematk Feedforward Convolutionai

A block diagram of a binary rate R = 1/2 nonsystematic feedforward convolutional encoder with memory order ,n = 3 is shown in Figure 11.1. Note that the encoder consists of k = 1 shift register with m = 3 delay elements and with n = 2 modulo-2 adders. The mod-2 adders can be implemented as EXCLUSIVE-OR gates. (We note here that any multi-input mod-2 adder. such as those in Figure 11.1, can be implemented using a sequence of binary input mod-2 adders, or EXCLUSIVE-OR gates.) Because mod-2 addition is a linear operation, the encoder is a linear system. All convolutional codes can be realized using a linear feedforward shift register encoder of this type. The information sequence Ill! = (uo, iq, u2, · · · ) enters the encoder one bit at a time. Because the encoder is a linear system, the two encoder output sequences 1 0 -..,(OJ = (v6°l. vi l. v1°l .... ) and -..,(ll = (v2). vf l, Vil),···) can be obtained as the convolution of the input sequence u with the two encoder impulse responses. The impulse responses are obtained by letting fill = (1 0 0 · · · ) and observing the two output sequences. For an encoder with memory order m, the impulse responses can last at most 111 + 1 time units and are written as g< 0l = (g60l, g) 0l, ... , g,~~l) and

Section 11.1

Encoding of Convolutional Codes

6155

~-~~--~~~~-~~~~~~~~~~~~~~>A

-----------~

-

FIGURE 11.1: A rate R = 1/2 binary nonsystematic feedforward convolutional encoder with memory order m = 3.

11; Cll

The

= (g0Cll , g1Cl) . · · ·

.

· 111 gmCl)) . F or "h t e enco d e1· o f F.1gme . , g(O)

= (1 0 1 1)

(11.la)

11;(1)

=

(11.lb)

(1 1 1 1).

responses i
= llil@ 11;(0)

(11.2a)

0

yCl) = 1lII ® 1';(1),

(11.2b)

where @ denotes discrete and all operations are modulo-2. The convolution operation implies that for all / ::,: 0, (] 1

where u1_, ~ 0 for all I < i, and all operations are modulo-2. Hence, for the encoder of Figure 11.l, (0)

v1

= U/

(1) VI

= U/

+ U/-2 + Ll/-3 + U/-1 + U,-2 + U/-3,

(11.4b)

as can easily be verified by direct inspection of the encoder diagram. After encoding, the two output sequences are multiplexed into a single sequence, called the codeword, for transmission over the channel. The codeword is given by _

Y-

(

(0)

(1)

(0)

(1)

(0)

(1)

)

vo vo , vl vl , v2 v2 , . . . .

(11.5)

456

Chapter 11

Convolutional Codes

For example, let the information sequence u sequences are

=

(1 0 1 1 1). Then, the output

= (1 0 111) ® (1 0 11) = (1 0 0 0 0 0 0 1),

(11.6a)

v< 1l = (1 0 1 11) ® (1111) = (11 0 111 0 1),

(11.6b)

v(O)

and the codeword is V

= (11, 0 1, 0 0, 0 1, 0 1, 0 1, 0 0. 11).

(11.7)

If we interlace the generator sequences g
the matrix (0)

(1)

go go

(0) (1) gl gl (0)

(1)

go go G=

(0) (1) g2 g2 (0) (1) gl gl (0)

(1)

go go

(0)

g/11

(1)

gm

(0) (1) gm_lgm-1 (0) (1)

gm-2gm-2

(0)

g/11

(1)

g/11

(0) (1) gm_lgm-1

(0)

(1)

g/11 gl/1

(11.8) where the blank areas are all zeros, we can rewrite the encoding equations in matrix form as V=l.llG, (11.9) where again all operations are modulo-2. The matrix G is called the (time domain) generator matrix of the encoder. Note that each row of G is identical to the previous row but shifted n = 2 places to the right and that G is a semi-infinite matrix, corresponding to the fact that the information sequence M is of arbitrary length. If u has finite length h, then G hash rows and 2(m + h) columns, and v has length 2(111 + h). For the encoder of Figure 11.1, if 1!ll = (l O 11 1), then V=lJIG

~

1 1 0 1 1 1 11 1 1 01 1 1 11 1 1 01 1 1 1 1 (10111) [ 11 01 11 11 11 0 1 1 1

= (11. 0 1, 0 0, 0 1, 0 1, 0 1, 0 0, 11),

J

(11.10)

which agrees with our previous calculation using discrete convolution.

EXAMPLE 11.2 Encoder

A Rate R = 2/3 Nonsystematk feedforward Convol1.11tional

As a second example of a nonsystematic feedforward convolutional encoder, consider the binary rate R = 2/3 encoder with memory order m = 1 shown in Figure 11.2. The encoder consists of k = 2 shift registers, each with m = 1

Section 11.1

Encoding of Convolutional Codes

~571

1!11(11

FIGURE 11.2: A rate R = 2/3 binary nonsystematic feedforward convolutional encoder with memory order m = 1.

delay element, along with n = 3 mod-2 adders. The information sequence enters the encoder k = 2 bits at a time and can be written as 11Il = (@o, @1 , ... ) = 2 2 1 1 ), u?lui l, ui lufl, · · ·) or as the two input sequences 11Il(l) = l, uill, 2 2 2 uil), · · ·) and @< l = (u~), ), ui ), · · ·).There are three generator sequences corre1l, · · · , /iJ) represent the l, g< sponding to each input sequence. Letting g;i) = (g(J l, 0 l, 1 l,111 generator sequence corresponding to input i and output j, we obtain the generator sequences for the encoder of Figure 11.2;

(u2\t6

(u6

ui

!J;iO)

=

(11)

!J;iO) = (0 1)

=

(0 1)

gfl

= (11),

(11.lla)

~g) = (1 0)

1J;i2)

=

(11.llb)

!J;il)

(1 0),

and we can write the encoding equations as w(O)

= u_uCl)

® !J;iO)

+ 11Il(2) ® !J;iO)

(11.12a)

w(ll

= 11Il(ll

® !J;ill

+ 11Il(2) ® lliill

(11.12b)

w<2l

= 11Il(ll ® gfl + 11Il(2l ® llii2).

(11.12c)

The convolution operation implies that (11.13a) Vz(l) (2)

Vz

= =

+ u(ll 1~1' (1) + (2) + (1) Uz Uz ul~l' i/2) I

(11.13b) (11.13c)

458

Chapter 11

Convolutional Codes

as can be seen directly from the encoder diagram. After multiplexing, the codeword is given by v = (vCO\(l)v(2) v(O)v(l)v(2) v(OJv(l\(2) ... ) (11.14) 0 0 0 ' 1 1 1 ' 2 2 2 ' ·

= (1 0 1) and u< 2l = (1 1 0), then

For example, if u< 1)

= (1 0 1) ® (11) + (1 1 0) ® (0 1) = (1 0 0 1),

(11.15a)

= (1 0 1) ® (0 1) + (11 0) ® (1 0) = (1 0 0 1), v<2l = (1 0 1) ® (11) + (1 1 0) ® (1 0) = (0 0 11),

(11.15b)

v
1

l

(11.15c)

and V

= (11 0, 0 0 0, 0 0 l, 111).

The generator matrix of a rate R (0)

(1)

(2)

g l.Og 1.0g 1.0 (0)

(1)

(2)

g2.0g2,0g2.0

G=

(0)

(1)

(2)

g 1.1 g 1.1 g 1.1 (0)

(1)

(2)

g2.1 g2.1 g2.1 (0)

(1)

(2)

gl.Ogl.Ogl.O (0)

(1)

(2)

g2.0g2.0g2.0

(11.16)

= 2/3 encoder is (0)

(1)

(2)

gl.m

gl.111

gl.m

(0)

(1)

(2)

g2.m

g2.m

g2.m

(0)

(1)

(2)

gl.m-1 gl.m-1 gl.111-l (0)

(1)

(2)

g2.m-1 g2.m-lg2.m-1

(0)

(1)

(2)

gl.mgl.mgl.m (0)

(1)

(2)

g2.mg2.mg2.m

(11.17) and the encoding equations in matrix form are again given by v = uG. Note that each set of k = 2 rows of G is identical to the previous set of rows but shifted n = 3 places to the right. For the encoder of Figure 11.2, ifu(l) = (101) and u< 2) = (11 0), then u = (uo, u,1, u2) = (11, 0 1, 1 0), and 101 011 v = uG = (11, 0 1, 1 0)

111 100 101 111 011 100 101 111 011 100

= (11 0, 0 0 0, 0 0 1, 1 11),

(11.18) which again agrees with our previous calculation using discrete convolution. In Figures 11.1 and 11.2, the connections from the k shift registers used to store the input sequences to the n modulo-2 adders used to form the output sequences correspond directly to the nonzero terms in the kn generator sequences given in (11.1) and (11.11), respectively. Similarly, any set of kn generator sequences can be used to realize a rate R = k/n feedforward convolutional encoder of the same type shown in Figures 11.1 and 11.2 with k shift registers to store the input sequences and

Section

of Convolutiona! Codes

.1

,IJ.:5!11

n modulo-2 adders to form the the k enter the left end of each shift modulo-2 adders external to the shift canonical and any feedforward convolutional encoder can be realized in this manner. will see later in this section that feedback encoders also can be realized in controller canonical Example 1L2 mustrntes that encoding, and the notation used to describe it, is more whF;n the mm1ber of sequences k > L In , the encoder contains k shift not all of which must have the same We now four definitions related to the of the shift used in convolutional encoders realized in controller canonical form.

HJ_ Let v; be the encoder with k input sequences, i

of the ith shift

lDJEJFliP>rnnor,i

For for the rate R R = 2/3 encoder of JD,JEnNnmN

JLJL2.

=

1/2 encoder of v1 = v2 = L

m is the maximum

=

max

(11.19)

v;;

l:5ci:5:J

of all k shift

For the rate R =

11.1. m = v1

encoder of Figure 11.2, m lDJJEJFITNJi'j['Wi"l

v = 3, and for the rate

The encoder memory order m is defined as m

that

H3

in a convolutional

= 1, 2, · · ·, k.

=

. v2)

=

= 3, and for the rate R = 2/3

1) = 1.

The overall constraint

v of the encoder is defined as

1J= ~ . L__, v· ,, l:5ci:5ck

that

v is the sum of the

For the rate R = 1/2 encoder of encoder of 11.2, v = v1 + v2 lDJJEJFrnnmN

straint

HA v is

of all k shift v = vi = 3, and for the rate R = 2/3 = 1

+ 1 = 2.

A, rate R = k/n convolutional encoder with overall conreferred to as an (n, k, v) encoder.

The rate R = 1/2 encoder of Figure 11.1 is a 1, 3), encoder and the rate R = 2/3 encoder of Figure 11.2 is a (3. 2, 2) encoder. Here we note that for an (n, 1. v) encoder, the overall constraint length v equals the memory order m; that is, v = m when k = 1; however, for k > 1, in general, m :s v, and in the case when each shift register has the same length, v = km. D{AMWJ>ILIE 'H ,3

IE111rncol®r A (4, 3, 3) binary nonsystematic feedforward convolutional encoder in which the shift register lengths are v1 = 0, v2 = 1. and v3 = 2 is shown in Figure 11.3. The

460

Chapter 11

Convolutional Codes

FIGURE 11.3: A (4, 3, 3) binary nonsystematic feedforward convolutional encoder.

memory order is m = 2, and the overall constraint length is v = 3. Following the notation of Example 11.2, the generator sequences are given by

giO) = (100)

gil) = (100)

gi2) = (100)

g?) = (100),

(11.21a)

giO) = (000)

gil) = (110)

gi2) = (010)

gfl = (100),

(11.21b)

giO) = (000)

gil) = (010)

gi2) = (101)

gfl = (101).

(11.21c)

In the general case of an (n, k, v) feedforward encoder with memory order m, the generator matrix is given by Gm G 111 _1

Gm

Gm-2

Gm-1

(11.22)

where each G1 is a k x n submatrix whose entries are (0)

gu

(0)

G1=

g2,/

(1)

gl,l (1)

g2,l

(0)

(1)

gk,l

gk,l

(11-l)

gu

(n-1)

g2,l

(11.23)

(n-1)

gk,l

Again note that each set of k rows of G is identical to the previous set of rows but shifted n places to the right. For an information (2) (k) h d d sequence l11l = (uo, an1, · · ·) = (u 0(1) u 0(2) · · · u 0(k) , u (1) u 1 · · · u 1 , · · · ), Le co ewor 1

Section 11.1

1 0 w = (wo, w1, · · ·) = ( v6°l v6 ) · · · vg'-ll, vf l

Encoding of Convolutional Codes

d!}/Ei

v?) ···

vi'7-ll, · · ·) is given w = ll!lG. 'vile are now in a position to give a formal definition of a convolutional code. 1L§ An (n. k, v) convolutional code is the set of an output sequences (codewords) by an (n, k, v) convolutional encoder; that it is the row space of the encoder matrix G.

lI})iEJFrnnmN

Because the codeword 'Vis a linear combination of rows of the generator matrix G, an k, v) convolutional code is a linear code.

The set of all codewords the (4. 3, 3) convolutional encoder of Figure 11.3 is a (4, 3, 3) convolutional code. fa any linear time-domain operations involving convolution can be replaced by more convenient transform-domain operations involving Because a convolutional encoder is a linear system, each sequence in the encoding equations can be replaced by a corresponding polynomial, and the convolution operation replaced by polynomial multiplication. In the polynomial representation of a binary sequence, the sequence itself is represented by the coefficients of the polynomial. For example, for a (2, 1, v) encoder, the encoding equations become yCO\D) = llll(D)~(Ol(D), (11.24a)

i where lill(D)

1l(D) = llll(D)~C 1\D),

(11.24b)

= uo + iqD + u2D 2 + · · ·

(11.25a)

is the information sequence, (11.25b) and

w0 \D)

= V6l) + v?) D + v;l) D 2 + ...

(11.25c)

= g(Ol + 01 a(O) D + ... + o
(11.25d)

are the encoded sequences, qr,(Ol(D) a

and (11.25e) are the generator polynomials of the code, and all operations are modulo-2. We can then write the codeword as (11.26a) or, after multiplexing, as (11.26b)

462

Chapter 11

Convolutional Codes

The indeterminate D can be interpreted as a delay operator, where the power of D denotes the number of time units a bit is delayed with respect to the initial bit in the sequence. (Co1nrl:i1111U1ed)

EXAMPLE 11.1

For the (2, 1, 3) encoder of Figure 11.1, the generator polynomials are i{0\D) = 1 + D 2 + D 3 and g(ll (D) = 1 + D + D 2 + D 3 . For the information sequence irn(D) = 1 + D 2 + D 3 + D 4 , the encoding equations are (11.27a) v(

11

(D)

2

4

3

2

= (1 + D + D + D )(1 + D + D + D

3

)

= 1 + D + D + D + D + D7 , 3

4

5

(11.27b) and we can write the codeword as V(D)

=

[1

+ D 7 , 1 + D + D 3 + D 4 + D 5 + D 7 ],

(11.28a)

or v(D)

= 1 + D + D 3 + D 7 + D 9 + D 11 + D 14 + D 15 .

(11.28b)

Note that in each case the result is the same as previously computed using convolution and matrix multiplication. The generator polynomials of an encoder realized in controller canonical form can be determined directly from the encoder block diagram, as noted previously for generator sequences. The sequence of connections ( a 1 representing a connection and a Ono connection) from the delay elements of a shift register to an output (mod-2 adder) is the sequence of coefficients in the associated generator polynomial; that is, it is the generator sequence. The lowest-degree (constant) terms in the generator polynomials correspond to the connections at the left ends of the shift registers, whereas the highest-degree terms correspond to the connections at the right ends of the shift registers. For example, in Figure 11.1, the sequence of connections from the shift register to output vCOl is gCOJ = (1 0 1 1), and the corresponding generator polynomial is g< 0 l(D) = 1 + D 2 + D 3 . Because the right end of the shift register in an (n, 1, v) encoder realized in controller canonical form must be connected to at least one output, the degree of at least one generator polynomial must be equal to the shift register length v1 = v = m; that is, m = max [deg g
In an (n, k, v) encoder where k > 1, there are n generator polynomials for each of the k inputs. Each set of n generators represents the sequences of connections from one of the shift registers to the n outputs, and hence v;

= max [deg O:<:j:<:11-l

g;.i\D)],

1::: i ::: k,

(11.30)

where g)il (D) is the generator polynomial relating the ith input to the }th output.

Section 11.1

Encoding of Convolutional Codes

453:

Because the encoder is a linear system, and llll(il(D) represents the ith sequence, and ,/il (D) represents the jth output sequence, the generator polynomial ~fil (D) can be interpreted as the encoder transfer function relating input i to output j. As with any n-output linear system, there are a total of kn transfer generator functions. These can be represented by the k >< n (transform matrix

(11.31)

Using the generator matrix, we can express the (transform domain) encoding equations for an (n, k, v) feedforward encoder as V(D)

= llJ(D)iG(D),

(11.32)

Ull< 2l(D), ... , llll(kl(D)] is the k-tuple of input sequences, and where llJ (D) ~ [ unC 1l 0 V(D) ~ [w< 1(D), w(ll(D), ... , w< 11 - 1l(D)] is then-tuple of output sequences, or the codeword. After multiplexing, the codeword can also be expressed as

(11.33)

For the (3, 2, 2) encoder of Figure 11.2, (11.34) For the input sequences un< 1l (D) equations give the codeword

= 1 + D 2 and

llll( 2l (D)

1 + D, the encoding

(11.35a) which also can be written as v(D)

= 1 + D + D 8 + D 9 + D 10 + D 11 .

(11.35b)

Again, these results are the same as those calculated using convolution and matrix multiplication.

464

Chapter 11

Convolutional Codes

We can rewrite Eqs. (11.31), (11.32), and (11.33) to provide a means of representing the multiplexed codeword v(D) directly in terms of the input sequences. A little algebraic manipulation yields k

v(D)

L 1i\D1 )g; (D), 1

=

(11.36)

i=l

where (11.37)

is a composite generator polynomial relating the ith input sequence to v(D). {Continued)

IEXAMPllE 11.1

For the (2, 1, 3) encoder of Figure 11.1, the composite generator polynomial is g(D)

= g<0 \D 2) + Dg
and for u(D) = 1 + D 2 v(D)

(11.38)

+ D 3 + D4, the codeword is

= u(D 2 )g(D) =

(1

+ D 4 + D 6 + D 8 )(1 + D + D 3 + D 4 + D 5 + D 6 + D 7 ) (11.39)

which again agrees with previous calculations. An important subclass of convolutional encoders is the class of systematic encoders. In a systematic encoder, k output sequences, called systematic output sequences, are exact replicas of the k input sequences. Although, in general, the k systematic output sequences can be any set of k output sequences, we will follow the convention that the first k output sequences are the systematic sequences; that is,

ull, i = l, 2, · · ·, k,

..,U-l) =

.l

(11.40)

and the generator sequences satisfy g;j)

=

lifj=i-1 i = 1, 2, · · · , k. 0 if j

I-

(11.41)

i - 1

In the case of systematic feedforward encoders, the (time-domain) generator matrix is given by

I

IPo

G=

l

OP1 I.Po

OP2 OP1 IPo

OPm OPm-1 OPm-2

OPm OPm-1

OPm

lJ

(11.42)

Section 11.1

vvhere ll is the k >< k ,n[c,nc,,1-, k x (n - k) matrix whose entries are -

(k)

is the k x k all-zero (n-l)

(k+l)

gl.l k

IP1 =

ii])

of Convolutional Codes

g2J

(k)

gk.l

gu

gu

(k+lJ g2.l

g2J

and

TID1

~lii5

is the

·-

(11-l)

(11.43)

(11-l)

(k+l)

gk.l

gk.l

Similarly, the k >< n

matrix becomes

,'.G(D) =

_Ck\D)

ll;f'-1\D)

(k)(D)

l!;g1-l) (D)

1 0

0

1/;1

0

1

0

1/;2

0

0

1

1/;k

'

(11.44)

(kl

(11-ll (D)

'lf,k

Because the first k sequences are sequences, they are also called output sequences are caHed kn generator must nonsystematic feedforward encoder, only k(n - k) to define a systematic feedforward encoder. encoders a subclass of the set of all possible encoders. encoder whose ( transform-donrnin) generator matrix lG(D) does not contain an matrix in some set of k columns is said to be For codes represented feedforward encoders with generator matrices in the form of (11.42) or (11.44), we can corresponding systematic manner. The (time-domain) nc,nr11 -r matrices in a matrix is given IPT

][

IPf

ii])

JP'[

IT

ii])

wf

ii])

l[D

]P'~;-1

JP;f,

0

JH[=

IP';f,

jU>T

][

ii])

JIT'T

ii])

JP'[

ll

ii])

IPT 111-l

I[/)

]ll)T H l

ii])

I[/)

l[DT

I!])

-' 0

'm-2

]PT 117

2

IPT

0 l[DT

1

ll I[))

rr»[

][

(11.45)

where, in this case, ll is the (n -k) >< (n -k) and ilJl is the (n -k) >< (n -k) all-zero matrix. Similarly, the (n - k) x n (transform-domain) parity-check matrix is

466

Chapter 11

Convolutional Codes

given by

H(D)

1t(D)

gt>(D)

~tlCD)

1 0

0

gik+l\D)

gik+l\D)

g
0 1

0

le

= gf1-l) (D)

gf1-l) (D)

(11.46a)

gg7-1\D)

0 0

1

where the last (n - k) columns of H(D) form the (n - k) x (n - k) identity matrix. The parity check matrix in (11.46a) can be rewritten as

H(D)

=

hiO) (D)

hill (D)

hilc-1) (D)

1 0

0

hiO)(D)

h?)(D)

1 hf'- \D)

0 1

0

h(k-ll (D)

0 0

1

(0)

h11-1c(D)

1l (D) ru/n-k

11-k

(11.46b)

where we have defined the (j - k + l)st parity-check polynomial associated with the (i - l)st output information sequence as h?:;!1 (D) = g;i) (D), j = k, k+ 1, · · · , n 1, and i = 1, 2, · · · , k. Simplifying notation, we can write the (n k) x k parity check polynomials as h/l (D), i = 1, 2, · ·. , n -k, and j = 0, 1, ... , k-1. (In the important special case of rate R = (n - 1)/n codes, i.e., (n - k) = 1, we denote the k = n - 1 parity-check polynomials without subscripts as h(j) (D). j = 0, 1, ... , k -1.) Finally, we associate the parity-check sequence hjil with the coefficients of the parity-check polynomial hj1) (D). Any codeword v (or V(D)) must then satisfy the parity-check equations (11.47) where the matrix Ohas semi-infinite dimensions in this case, or (11.48)

V(D)HT (D) = O(D),

where O(D) represents the 1 x (n - k) matrix of all-zero sequences. In other words, the set of all codewords v ( or V(D)) is the null space of the parity-check matrix H (or H(D)).

IEXAMPllE H.4 Encoder

Consider the (2, 1, 3) systematic feedforward encoder shown in Figure 11.4. The generator sequences are g< 0) = (1 0 0 0) and g< 1l = (1 1 0 1), and the generator matrices are given by 11

01 11

00 01 11

01 00 01

01 00

l

01

,

J

(11.49)

Section 11.1

!Encoding of Convolutional Codes

ii}t61

systematic feedforward convolutional encoder.

FIGURE 11.4: A (2, 1, 3)

and G(D)

For an input sequence 11.Il(D)

=

1+D

[1

+ D 3 ].

(11.50)

= 1 + D 2 + D 3 , the output information sequence is (11.51)

the output

sequence is 1 w< \D)

=

1 llll(D)g< 1(D)

=

(1

+

= 1 + D + D2 + D3 +

+ D 3)(1 + D + D 3) + D5 + D6 ,

(11.52)

and the codeword is given by V(D)

=

[1

+ D2 +

(11.53a)

or w(D)

= y(O) (D 2) + DY(l) (D 2) = 1 + D + D 3 + D 4 + D 5 + D 6 + D 7 + D 9 + D 11 + D 13 ,

(11.53b)

which we can also write as Y=

(ll,01,11,ll,01,01,Ql).

(11.53c)

Using (11.45) and (11.46b ), we see that the parity-check matrices are given by 11 10 IHI=

11

00 10 11 10 00 10 11 10 00 10 10 00

11 10

(11.54) 11

468

Chapter 11

Convolutional Codes

and 1]=[1+D+D 3

H(D) = [lhl(Ol (D)

11J•

(11.55)

~

For the codeword of (11.53) we then have 1 1 0 1 0 0 1 1 1 0 1 1

1 0 0 0 1 0 1 1

vHT = [11, 01, 11, 11, 01, 01, 01]

1 0 0 0 1 0 1 1

1 0 0 0 1 0 1 1

1 0 0 0 1 0 1 1

=0,

(11.56)

or V(D)l-lf (D) = [1 + D 2 + D 3

1 + D + D 2 + D 3 + D 4 + D 5 + D6][ l + Dl + D

= O(D).

EXAMPLE 11.5 IEnrnder

3

J

(11.57)

A !Rate R

= 2/3

Systematic: feedforward Cm1rvoh..1tooB1a!

Now, consider the rate R = 2/3 systematic feedforward encoder with generator matrix G(D)

=[

1

0

0 1

1+D

+D

2

l+D

J.

(11.58)

Using (11.46b), we can write the parity-check matrix as H(D)

= [Jh/Ol (D)

h(ll (D)

(11.59)

The controller canonical form realization of the encoder requires a total of v = v1 + v2 = 3 delay elements and is shown in Figure 11.S(a). This realization results in a (3, 2, 3) encoder. Moreover, since the output information sequences are given by v(O\D) = u(l\D) and vC 1l (D) = uC 21(D), and the output parity sequence is given by v<2\D)

= lll!< 11(D)gi2\D) + rui<2JcD)gi2l(D), = (1 + D + D 2 )u< 1JcD) + (1 + D)u<2l (D),

(11.60)

the encoder can also be realized as shown in Figure 11.S(b). Note that this realization requires only tvvo delay elements rather than three.

'.Section 11. 1

Encoding of Convolutional Codes

£Jltlii9

llll(I)

(a)

_ l l l l l _ l J ~ - - - - - - 0 - - - - - - - < : ; - - - - . . , o - yiOI

(b)

FIGURE 11 .5: (a) Controller canonical form and (b) observer canonical form realizations of a rate R = 2/3 binary systematic feedforward convolutional encoder.

The encoder realization of Figure lLS(b) is called observer canonical form. fo general, for observer canonical form encoder realizations, there is one shift register corresponding to each of the 11 output sequences, the k input sequences enter modulo-2 adders internal to the shift registers, and the outputs at the right end of each shift register form the 11 output sequences. Also, the lowest-degree (constant) terms in the generator (parity-check) polynomials represent the connections at the right ends of the shift registers, whereas the highest-degree terms represent the connections at the left ends of the shift registers. It is important to note that this is exactly the opposite of the correspondence between polynomial coefficients and delay elements in the case of a controller canonical form realization. For this reason, when an encoder is realized in observer canonical form, it is common to write the generator (parity-check) polynomials in the opposite of the usual order, that is, from highest degree to lowest degree. In the case of systematic encoders like the one in Figure 1L5(b), there are only (n - k) shift registers, and the k input sequences appear directly as the first k encoder output sequences. Definitions related to the lengths of the shift registers used in convolutional encoders realized in observer canonical form can be given that are analogous to Definitions 1Ll-1L3. The only difference is that in this case there are, in general, n shift registers rather then k,

470

Chapter 11

Convolutional Codes

and vi is defined over the range j = 0, 1,- · · , n - 1. Any feedforward convolutional encoder can be realized in observer canonical form, and we will see shortly that feedback encoders also can be realized in this manner. Thus, controller canonical form and observer canonical form provide two general realization methods that can be applied to all convolutional encoders.

EXAMPLE 11.5

(Conti1111UJed)

For an observer canonical form realization, we rewrite the generator matrix of (11.58) as G(D) = [ 1 0 0 1 and the parity-check matrix of (11.59) as H(D)

D

= [D 2 + D + 1

2

+D +1

D+l

J

(11.61)

(11.62)

D+l

In this way we can clearly identify the coefficients of the polynomials in (11.61) and (11.62) with the connections in Figure 11.S(b). For this realization, we see that vo = 0, v1 = 0, v2 = 2, m = max(O.'.':J.'.':n-1) VJ = 2, and v = °E(O.'.':J.'.':n-1) VJ = 2. Thus, the observer canonical form realization results in a (3, 2, 2) encoder. Example 11.5 raises an interesting question. Both encoder realizations generate exactly the same code, that is, the row space of G(D), but since there are two distinct encoder realizations with different memory requirements, is this a (3, 2, 3) code or a (3, 2, 2) code? If we realize the encoder in controller canonical form, the overall constraint length is v = 3, resulting in a (3, 2, 3) encoder; however, if we realize the encoder in observer canonical form, then v = 2, resulting in a (3, 2, 2) encoder. We will see in the next section that since a convolutional encoder can be realized as a linear sequential circuit, it can be described using a state diagram with 2v states. Also, we will see in the next chapter that maximum likelihood decoding and maximum a posteriori probability decoding of convolutional codes require a decoding complexity that is proportional to the number of states in the encoder state diagram. Hence, we always seek an encoder realization with a minimal number of states, that is, with a minimal overall constraint length v. We will continue our discussion of minimal encoder realizations after introducing another class of systematic encoders. Systematic feedback encoders generate the same codes as corresponding nonsystematic feedforward encoders but exhibit a different mapping between information sequences and codewords. We again illustrate the basic concepts using several examples.

EXAMPllE 11.6 Em::oder Consider the rate R generator matrix G(D)

= [g

A Rate R

= 1/3

Systematk feedback Convoh.rticmal

= 1/3 nonsystematic feedforward convolutional encoder with g
1+ D2

1 +DJ. (11.63a)

Section 11.1

Encoding of Convolutional Codes

11.6(a). Revers-

The (3, 1, 2) controller canonical form realization is shown in the order of the m vve have G(D)

= [D 2 + D + 1

4n

+1

from which we obtain the 1, 5) observer canonical form encoder realization shown in Figure 11.6(b ). Note that in this case the controller canonical form realization has overall constraint length v = 2, that is, 4 states, whereas the observer canonical form realization 32 states! Now, we divide each in by the gCO) (D) = 1 + D + to obtain the modified generator matrix G' (D)

=

[1

14;Cll(D)/g(O)(D)

=

[1

(l+D )/(l+D+D )

2

(D)/g(O)(D)] 2

(11.64) (l+D)/(l+D+

in feedback form. We note in this case that the response of the encoder represented by G' (D) has infinite that the feedback shift register realization of G' (D) is an infinite impulse response (HR) linear system. For this reason, the (time-domain) generator matrix G' corresponding to G' (D) contains sequences of infinite length. For example, the ratio of generator polynomials g< 1l(D)/g<0 l(D) in (11.64) produces the power series (1 + D 2 )/(1 + D + D 2 ) = 1 + D + D2 + + D 5 + D 7 + D 8 + D 10 + ... , whose time-domain equivalent is the infinite-length sequence (11101101101 · · · ). The that is, the set of encoder output sequences generated by G' (D) is exactly the same as the code generated by G(D). This can be seen by noting that if the information sequence illl(D) produces the codeword 'Y(D), that V(D) = llll(D)G(D), in the code generated by G(D), then the information sequence llli' (D) = (1 + D + D 2)llll(D) produces the same codeword V(D), that is, V(D) = llll, (D)G' (D), in the code by G' (D). fo other words, the row spaces of G(D) and G' (D) are the same, and thus generate the same code. His important to note, that there is a different mapping between information sequences and codewords in the two cases. The 4-state, (3, l, 2) encoder diagram, in controller canonical to the systematic feedback encoder of (11.64), is shown in Figure 11.6(c). fo contrast, the observer canonical form realization of (G' (D) requires 16 states (see Problem 11.8). Finally, since G' (D) is systematic, we can use (11.46a) to obtain the parity-check matrix 2 2 n-JI' (D) = [ (1 + D )/(1 + D + D ) 1 0 (11.65) 2 0 1 ' (l+D)/(l+D+D )

J

where IHI' (D) is a valid parity-check matrix for both G(D) and G' (D); that is, all codewords in the row spaces of G(D) and G' (D) are also in the null space of IHI' (D), and as noted for generator sequences, the systematic (time-domain) parity-check matrix IHI' corresponding to G' contains sequences of infinite length.

In general, if the k x 11 polynomial matrix G(D) is the generator matrix of a rate R = k/n nonsystematic convolutional encoder, then elementary (polynomial) row operations can be performed on G(D) to convert it to a systematic k >< n

412

Chapter 11

Convolutional Codes

+ >-------------v(2) (a)

(b)

(c)

FIGURE 11.6: (a) The controller canonical form and (b) the observer canonical form realizations of a nonsystematic feedforward encoder, and (c) the controller canonical form realization of an equivalent systematic feedback encoder.

Section 11. 'l

of Convolutional Codes

~.13

in the

11.6 exhibi,t HR

entries comain sequences of infinite nonzero in a feedback encoder has infinite feedback encoders are called recursive co11vo!11tfonaf COdeSo r,
codewords as me nonrecursive convolotional code feedforw,wd eacoder: however, as noted in ,,~,,cuur,wj be'cween information sequences and codewords is different in the two cases. In information sequences 1 finiteinformation sequences of weight 1 ·~a·,m·h, of recursive codes turns out to be a ,foctor achieved the class of concate,uited convolutional to be discussed in ~

0

11.2 was

To

that the observer canonical form realization of (11.34) feedback

1L Row 1::::} + D)][Row 1]. Row2::::} Row2+ 1]. :t Rmv 2::::} [(l + D)/(1 + D + ~t Row 1 ::::} Row 1 + [D / (1 +

lo

The result is the modified

2].

matrix (see Problem 11.7) 0

1

2

1/(1+D+D ) (1 + D 2 )/(1 + D + D 2 )

J

(11.66)

414

Chapter 11

Convolutional Codes

in systematic feedback form. Now, we can use (11.46b) to form the systematic parity-check matrix H' (D) = [hC0\D)/h< 2\D)

h(ll (D) /h< 2l (D)

1]

(11.67a)

(1 + D 2 )/(1 + D + D 2 )

=[1/(l+D+D 2 )

1]

or the equivalent nonsystematic polynomial parity-check matrix H(D) = [hCOJ (D)

li2l (D)] = [1

h(l) (D)

1 + D2

1+D

+ D 2 ],

(11.67b)

where iJi.Ul (D) represents the parity-check polynomial associated with the jth output sequence. Again, the recursive code generated by G' (D) is exactly the same as the nonrecursive code generated by G(D), and the encoder diagram, in controller canonical form, corresponding to the systematic feedback encoder of (11.66), is shown in Figure 11.7(a). We note in this case that Figure 11.7(a) represents a 16-state, (3, 2, 4) encoder; that is, two length-2 shift registers are needed in the controller canonical form realization. Reversing the order of the polynomials in (11.66) and (11.67a), we obtain

G' (D) =

[ 1 0

0

1

2

l/(D + D + 1) (D 2 +1)/(D 2 +D+l)

J

(11.68)

and H' (D)

= [l/(D 2 + D + 1)

(D 2 + 1)/(D 2 + D + 1)

1],

(11.69)

which leads to the observer canonical form systematic feedback encoder realization shown in Figure 11.7(b). We note that this represents a 4-state, (3, 2, 2) encoder; that is, for the systematic feedback encoder, the observer canonical form realization requires fewer states than the controller canonical form realization. The rate R = 2/3 code of Example 11.2 has a 4-state nonsystematic feedforward encoder realization in controller canonical form (see Figure 11.2). It also has a 4-state systematic feedback encoder realization in observer canonical form (see Figure ll.7(b)). Thus, in this case, both encoder realizations have an overall constraint length v = 2; that is, we obtain a minimal encoder realization in both controller canonical form and observer canonical form. We now give an example of a rate R = 2/3 code in which a minimal encoder realization is obtained only in observer canonical form. EXAMPLE 11.7 Encoder

A Rate R

Now, consider the rate R given by

=

= 2/3

Systematic Feedback Convolutionai

2/3 nonsystematic feedforward generator matrix

G(D) - [ 1 D 0 1 + D + D2 + D3

1+ D 1+ D2 + D3

J.

(11.70)

A 16-state, (3, 2. 4) controller canonical form encoder realization is shown in Figure 11.S(a). (Note that the observer canonical form realization of (11.70) would

Section 11.1

Encoding of Convolutional Codes

dil1!'li

(a)

(b)

FIGURE 11.7: (a) A (3. 2, 4) systematic feedback encoder in controller canonical form and (b) an equivalent (3, 2, 2) systematic feedback encoder in observer canonical

form. require 64 states.) To convert
416

Chapter 11

Convolutional Codes

,---------------------------l>--V(O)

~----------.J

+

i--+-----<>-V(I)

(a)

-'-11(_11_ _ _ _ _ _ _ __,.__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __.V(O)

wP>

----------1--------1---
(b)

FIGURE 11.8: (a) A (3, 2, 4) nonsystematic feedforward encoder in controller canonical form and (b) an equivalent (3, 2, 3) systematic feedback encoder in observer canonical form.

in systematic feedback form, and we can use (11.46b) to form the systematic parity-check matrix H' (D) = [h(O) (D) /hi<2) (D) hC 1\D) ;niP\D) 1] =[(l+D+D 2 )/(l+D+D 2 +D 3 )

(l+D+D 3 )/(l+D+D 2 +D 3 )

1]

(11.72a) or the equivalent nonsystematic polynomial parity-check matrix H(D) = [1

+ D + D2

1+D

+ D3

(11.72b)

Encoding of Convolutional Codes

Section 11. 1

411

fo this case, the encoder diagram, in controller canonical to the feedback encoder of (1L71), represents a (3, 2. 6) encoder; that is, tvvo length-3 shift are needed in the controller canonical form realization. Reversing the order of the m and (1L72a), we obtain

+ D + 1)/(D3 + + D + 1) + D + 1)/(D3 + D 2 + D + 1)

J

73)

and (D) = [(D

2

+ D + 1)/(D3 +

+ D + 1)

(D

3

+ D + l)/(D 3 + D 2 + D + 1)

l],

(1L74) which leads to the observer canonical form feedback encoder realization an 8-state, (3, 2, 3) ,,,,,,~,,,.,, shown in Figure 11.8(b ). Vl/e note that this that is, the observer canonical form feedback encoder realization fewer states than the controller canonical form feedforward encoder realization. Example 11.6 shows that for a rate R = code, a minimal encoder realization is obtained in controller canonical form. fa general, for any rate R = l/n encoder described by a 1 x n polynomial generator matrix G(D) or an equivalent systematic rational function generator matrix G (D), the controller canonical form realization of either G(D) or d (D) is minimal, provided all common factors have been removed from G(D). (Removing common factors from a 1 x n generator matrix does not change its row space.) For example, consider the rate R = 1/2 nonsystematic feedforward encoder described by the generator matrix 1

ii;(l)(D)]

= [1 +

(11.75)

The controller canonical form realization of this encoder has overall constraint length \! = 3; that it has 2v = 8 states; however, if we remove the common factor (1 + D) from the generator gCOl (D) and gCll (D), we obtain the generator matrix G(D)

=

[g

g(l)(D)]

= [l + D

(11.76a)

or its systematic feedback equivalent,

d (D) = [1

(1

+ D + D 2 )/(1 + D)].

(11.76b)

1

The controller canonical form realization of both G(D) and
1

418

Chapter 11

Convolutional Codes

factors have been removed from the 1 x n polynomial parity-check matrix H(D) corresponding to the systematic rational function parity-check matrix H' (D) derived from G' (D). (Removing common factors from a 1 x n polynomial parity check matrix does not change its null space.) An exception occurs when one or more of the rational functions in G' (D) is not realizable; that is, the delay of its denominator polynomial exceeds the delay of its numerator polynomial, where the delay of a polynomial is defined as its lowest-degree nonzero term. In this case, a realizable minimal realization must be found, which may not correspond to observer canonical form. An example is given in Problem 11.9. In general, the rate R = k / n convolutional code produced by a k x n generator matrix G(D) can also be produced by any k x n matrix G' (D) with the same row space as G(D). If the encoder contains feedback, the code is recursive; otherwise, it is nonrecursive. Any realization of one of these equivalent generator matrices that has minimal overall constraint length, that is, a minimal number of delay elements, results in a minimal encoder. The foregoing summary of the requirements for determining minimal encoder realizations applies to all code rates normally used in practice, that is, rates R = l/n and R = (n 1)/n. For code rates with k > 1 and (n - k) > 1, such as rates 2/5, 2/4, and 3/5, determining minimal encoder realizations is more complex. In these cases, a minimal encoder may require a realization that is neither in controller canonical form nor in observer canonical form. The subject of minimal encoder realizations was first treated in a seminal paper by Forney [13]. More recently, the book by Johannesson and Zigangirov [12] provides a comprehensive treatment of this topic. Throughout the remainder of the text, when we refer to an (n, k, v) convolutional code, we assume a minimal encoder realization; that is, v is the minimal overall constraint length of any encoder realization. Also, when we use the term systematic (or nonsystematic) code, we mean the code produced by a systematic (or nonsystematic) encoder. Finally, we observe that for rate R = 1/ n encoders in controller canonical form and rate R = (n - 1)/n encoders in observer canonical form, the overall constraint length v equals the memory order m. Thus, for rates R = l/n and R = (n - 1)/n, we can refer to an (n, k, v) code or, equivalently, an (11, k, m) code. In general, however, if the form of the encoder realization is not specified, m :::_: v. To this point we have discussed nonsystematic feedforward encoders, systematic feedforward encoders, and systematic feedback encoders. A fourth general type of encoder is the class of nonsystematic feedback encoders. IEXAMPLIE 11.3 !Encoder

Consider the rate R = 2/3 nonsystematic recursive convolutional code (NSRCC) generated by the matrix G(D)

=[

2

l/(l+D+D D 2 /(1 + D 3 )

)

+ D3) 1/(1 + D 3 )

D/(1

+ D 3) 1/(1 + D)

1/(1

J

(11.77)

in nonsystematic feedback form. To determine the controller (observer) canonical form realization of (11.77), the rational functions in each row (column) of G(D) must

of Convolutional Codes

6}.79

that we must find the least have the same denominator con1r11on denor11inator for the rational fiD1ncdons in each rovv In this case~ since 1/(1 + D + (1 + D)/(1 + and 1/(l + D) = (1 + D + D 2 )/(l + D 3 ),

1/(1 +

(1

+D +

j.

+

(11.78)

dete:nI1iI1e controller and observer canonical for1n realizations of

These realizations are sho·vvI1 <;;'~'-D (

V

vlab.,,

(1J

=

==

/'\ 0)

9) realization. h, Problem 11.10

is shown that a minimal 8--state ( 1J = 3) observer canonical forrn rerdizatlor1 is achlle 1,1ed to systematic in a

feedback

2. 3) SRCC

1.8 and Problem 11.10 illustrate that the same used to find rninirnal realizations fer feedback encodeTs. To summarize our discussion of encoder for 11.10 the controller and realizable rational function

(fo+fiD+ ...

g(D)

+

(11.79)

a nor1zero constartt terrn to to the notation that vve will use to describe the remainder of the text for the most and rate R = l/n

is ')Ve

con1Iol1Jtio1Jca.i feedfor\Nard enco{ier \A.Tith

or its rational func,tion

feedback

= [1

(11.80b)

feedforward encoder.) The code is ,Hc'"""c"c

1,;(0)(D), \l);(l)(D),

0• 0 ,

,v;(n-1)

sequences 'G convolutional code can be described (see feedback encoder vvith rational function n~.~·n,c,~n

1\D)

any rate a minimal

L.,;,u,H~Hv

CD)/1ti
1_1;< 11 -

matrix

1]

(11.81a)

(11.81b)

480

Chapter 11

Convolutional Codes

+

+

1--------~V(O)

-------------+-__,,._.

+

>------<>V(l)

(a)

D

(b)

FIGURE 11.9: (a) The controller canonical form and (b) the observer canonical form realization of a nonsystematic feedback encoder.

Section 11.1

Encoding of Convolutional Codes

dilliH

FIGURE 11.10: (a) The controller canonical form and (b) the observer canonical form realization of an arbitrary realizable rational function. (If 11n<11 - 1l(D) = 1, (11.81) describes a systematic feedforward The code is described by specifying the n parity-check polynomials nn< 0l (D), lln(ll(D), ... , nnCn-ll (D) or the n equivalent sequences Jln(O), lln(ll, ... , JlnCn-1).

In the notation of (11.81), for rate R = (n - 1)/n codes, the sequence w
482

Chapter 11

Convolutional Codes

output parity sequence v(n-l) is at the bottom of the diagram, and the (n - 1) output information sequences, in the order vCO), ... , v< 11 - 2>, are on top. (See, for example, the labeling of output sequences in Figure 11.8(b)). For reasons related to the design of trellis-coded modulation systems, which are described in Chapter 18 and use only rate R = (n - 1)/n codes, it has become conventional in the literature to reverse the order of the labeling of input and output sequences when describing rate R = (n 1)/n codes. In other words, instead of being written in the notation of (11.81), the parity-check matrices used to describe a rate R = (n - 1)/n code are given as lhl(l) (D) /lll(OJ (D)

1]

(11.82a)

and (11.82b) that 1ru<0>(D) =

(For systematic feedforward encoders, the new labeling implies 1, and = g~~-/\D), j = 1, 2, ... , n - 1.) In this case, the output parity sequence vCO) is associated with the feedback parity-check sequence 1rn
lhl.(}l(D)

1112)

llll(l)

vl2J

viii

villi

FIGURE 11.11: A relabeled version of the (3, 2, 3) systematic feedback encoder of Figure 11.8(b ). 1Exceptions mp. mfldP. in S1c,tions 13.5-13 7 3nd in Ch3ptpr ?.1 whe,re,, to conform to the, literntnrn on systematic feedforward encoders, the labeling of Figure 11.5 is used to describe rate R = (n - 1) / n codes.

Section 11.1

Encoding of Convolutional Codes

4~3

is malfunctioning, or when the channel SNR is high enough that decoding is not necessary. In general, this process requires an inverter to recover the information sequence from the codeword. For systematic encoders the inverter is trivial, since the information sequence is available. For nonsystematic encoders. on the other an n x k inverse matrix rr;;- 1 (D) must exist such that (11.83) for some l ~ 0, where TI is the k x k identity matrix. Equations (11.32) and (11.83) then that (11.84)

and the information sequence can be recovered with an /-time unit delay from the codeword by letting V(D) be the input to the n-input, k-output linear sequential circuit whose transfer function matrix is rr;;- 1 (D). For an (n, 1, v) convolutional encoder. a generator matrix G(D) has a feedforward inverse rr;;- 1 (D) of delay l if and only if (11.85)

for some l ~ 0, ,vhere GCD denotes greatest common divisor. For an (n, k, v) encoder with k > 1, let

t,,; (D).

i

=

1, 2. · · · , ( ; ), be the determinants of the

( ; ) distinct k x k submatrices of the generator matrix G(D). Then, a feedforward inverse of delay l exists if and only if

J= D

GCD [ t,,; (D): i = 1, 2, ... , ( ; )

1

(11.86)

for some l ~ 0. Procedures for constructing feedforward inverses rr;;-l (D) are given in Massey and Sain [14].

m, For the (2, 1, 3) encoder of Figure 11.1, the generator matrix is given by G(D)

= [gCOJ (D)

1l1l(D)] = [1 + D 2 + D 3

1+ D + D

2

+ D 3 J, (11.87a)

GCD[gCOJ(D), g(l\D)]

= GCD[l+D 2 +D3,

l+D+D 2 +D 3 ]

= 1,

(11.87b)

and the transfer function matrix rr;;-l(D) = [ 1 + D + D D+D 2

2

J

(11.88)

provides the required feedforward inverse of delay O; that is, G(D)G- 1 (D) = 1. A realization of the encoder inverse is shown in Figure 11.12(a).

484

Chapter 11

Convolutional Codes

(a)

(b)

FIGURE 11.12: Feedforward encoder inverses for (a) a (3, 1, 2) encoder and (b) a

(3, 2, 2) encoder. b. For the (3, 2, 2) encoder of Figure 11.2 with generator matrix G(D) given in (11.34), the 2 x 2 submatrices of G(D) yield determinants 1 + D + D 2 , 1 + D 2 , and 1. Because (11.89) there exists a feedforward inverse of delay 0. The required transfer function matrix is given by

G- (D)~[: l{D l 1

(11.90)

and its realization is shown in Figure ll.12(b). To understand what happens when a feedforward inverse does not exist, it is best to consider another example. !EXAMPLE 11 .10

A Catastrophic Encoder

Consider the (2, 1, 2) encoder whose generator matrix is given by (11.91a) In this case we see that (11.91b)

Section 11.1

Encoding of Convolutional Codes

4~5

and a feedforward inverse does not exist. If the information sequence is un(D) = 0 1 2 1JD = 1 + D + D +···,then the output sequences are vC l(D) = 1 and v( l(D) = 1 + D; that is, the codeword has weight 3 even though the information sequence has infinite H this codeword is transmitted over a BSC, and the three nonzero bits are to zeros the channel then the received sequence wiH be aH zero. A maximum likelihood decoder wiH then the all-zero codeword as its since this is a valid codeword and it agrees exactly with the received the estimated information sequence win be l!Il(D) = il])(D), an infinite number of errors caused by a finite number three in this of channel errors. Clearly, this is very undesirable, and the encoder is said to to error propagation; that is, it is a catastrophic encoder. m encoders for use in an error control coding system, it is to avoid the selection of catastrophic encoders. Massey and Sain [14] have shovvn that (11.85) and (11.86) are necessary and sufficient conditions for an encoder to be noncatastrophic. Hence, any encoder for which a feedforward inverse exists is noncatastrophic. H follows that systematic encoders are since a trivial feedforward inverse exists. Minimal nonsystematic encoders are also noncatastrophic, but nonminimal encoders can be catastrophic [12]. Note, for example, that the nonminimal encoder of (11.75) does not satisfy (11.85), and thus it is catastrophic. Only noncatastrophic encoders are listed in the tables of good convolutional encoders given later in the text in Chapters 12, 13, 16, and 21. An (n, k, v) convolutional encoder can generate infinitely long encoded sequences (codewords). Because n encoded bits are produced for each k information bits, R = k/n is called the rate of the convolutional encoder. For a finite-length information sequence of h time units, or K* = kh bits, it is customary to begin encoding in the all-zero state; that the initial contents of the shift registers are all zeros. Also, to facilitate decoding, it is necessary to return the encoder to the all-zero state after the last information block has entered the encoder, as will be seen in Chapters 12 and 13. This requires that an additional m input blocks enter the encoder to force it to return to the all-zero state. For feedforward encoders, it can be seen from the encoder diagram that these m termination blocks must be all zeros, whereas for feedback encoders, the termination blocks depend on the information sequence (see Problem 11.12). These termination blocks are not part of the information sequence, since they are fixed and cannot be chosen arbitrarily. (We note here that for controller canonical form encoder realizations with v < km, only v bits are required for termination. Thus, km - v bits in the m termination blocks can be information bits. For example, for the (4, 3, 3) encoder realization shown in Figure 11.3, only v = 3 bits are needed for termination, and the remaining 3 bits in the 2 termination blocks can be information bits. A similar situation holds in the case of observer canonical form realizations (see Problem 11.13). For simplicity; however, we ignore these special cases in the succeeding development.) The corresponding codeword has length N = n(h + m), including them output blocks, or nm output bits, resulting from termination. Such a terminated convolutional code can be viewed as an (N, K*) = [n(h + m), kh] linear block code with a K* x N generator matrix G. The block (terminated convolutional) code rate is given by

486

Chapter 11

Convolutional Codes

nu7!m),

Rt = lfv'' = the ratio of the number of information bits to the length of the codeword. Ifh >> m, thenh/(h +m) ~ 1, and the block (terminated convolutional) code rate Rt and the convolutional encoder rate R are approximately equal. This represents the normal mode of operation for convolutional codes, and we usually do not distinguish between the rate R of a convolutional encoder and the rate Rt of the associated block (terminated convolutional) code; that is, we assume R = R 1 • If his small, however, the ratio which is the rate Rt of the block (terminated convolutional) code, is reduced below the convolutional encoder rate R by a fractional amount kh m nk - niJz+m5 (11.92) k/n h+m called the fractional rate loss. For example, for the (2, 1, 3) encoder and information sequence in Example 11.1, h = 5 and the fractional rate loss is = 37.5%; however, for an information sequence of length h = 1000, the fractional rate loss is only 1503 = o. 3 %. For short information sequences, the fractional rate loss can be eliminated by simply not transmitting the nm termination bits and considering the first N* = nh encoder output bits as the codeword; however, as will be seen in Chapters 12 and 13, if the termination bits are not transmitted, the performance of the code is degraded. One way to combat this problem is to allow the encoder to start in an arbitrary state-that is, the initial contents of the shift registers are not necessarily all zeros-and then force it to return to the same state at the end of the information sequence. It turns out that if the starting state is chosen properly, no extra bits are required to force the encoder back to the same state, so no fractional rate loss is incurred. This technique results in so-called tail-biting codes, a subject that is discussed in more detail in Section 12.7.

nu7!m),

i

11.2

STRUCTURAL PROPERTIES Of CONVOLUTIONAL CODES

Because a convolutional encoder is a linear sequential circuit, its operation can be described by a state diagram. The state of an encoder is defined as its shift register contents. For an (n, k. v) encoder in controller canonical form, the ith shift register at time unit l ( when u jl), u u are the encoder inputs) contains v; bits, (i) (i) (i) (i) denoted as s1_ 1 , s1_ 2 , · · · , sl-v;, where s1_ 1 represent the contents of the leftmost

fl, ··· , ?)

delay element, and 1 ::= i ::= k. DEJFINHION

si~v; represents the contents of the rightmost delay element,

11.6 The encoder state
Thus, we see from (11.20) that there are a total of 2v different possible states. For an (n, 1, v) encoder, the encoder state at time unit l is simply
=

(s1-1 S/-2 · · · S/-v),

(11.94)

In the case of feedforward encoders, we can see from the encoder diagram (see, for example, Figure 11.1) that the contents of the ith shift register at time unit

Section 11.2

. previous . l are its v; at time unit l is given by

Structural Properties 01' Convolutional Codes

. u (i)_ , u (i)_ , t hat 1s, 1 1 1 2

··· ,

gj,i857

u1(i)_v,, an d h ence t he enco d er state

(11.95) For an (n, 1, v) encoder, (11.93) becrnnes (f

=

(U/-1

U/-2

U/-v).

(11.96)

For observer canonical form encoder the encoder state fF/ at time unit l can also be defined as in (11.93), that in this case there are, in a total of n rather than k shift registers. Also, feedforward encoders in observer canonical form do not have the property that the contents of a shift are its v; previous that is, (11.95) and (11.96) apply to feedforward encoder realizations in controller canonical form. Each new block of k bits causes the encoder to that there is a transition to a new state. Hence, there are 2k branches each state in the state diagram, one to each different input block. For an (n, 1, v) encoder, therefore, there are two branches leaving each state. Each branch in the state diagram is labeled with the k inputs (u;1), u; 2>, · · · , u;k)) the tran1 01 sition and the n corresponding outputs (v[ , v[1>, · · · , v?7- \ The state for the feedforward controller canonical form realizations of the (2, L 3) encoder of Figure 11.1 and the (3, 2, 2) encoder of Figure 11.2 are shown in Figures and 11.13(b ), respectively. The states are labeled So, S1, · · · , S2, -1, where convention S; represents the state whose binary v-tuple representation bo, b1, · · · , b,,_ 1 is equivalent to the integer i = bo2° + b12 1 + · · · + bv-12v-l _For for the (2, 1, 3) encoder of Figure 11.1, if the shift register contents at time l are u1 = (s1-1 s1-2 s1_3) = (u1-1 u1-2 u1_3), the state at time l is denoted as S;, where i

= u1-1 + 2u1-2 + 4u1_3.

We note here that for time-invariant convolutional encoders, that encoders whose generator matrix and associated encoder diagram do not with the encoder state diagram is also time-invariant. Thus the same state diagram e.g., those in Figure is valid for aH time units l. Time-varying convolutional encoders are not treated explicitly in this text, but we note here that the class of turbo codes, to be covered in Chapter 16, is generated by a special kind of convolutional encoder. Assuming that the encoder is initially in state So (the all-zero obtain the codeword corresponding to any given information sequence by the path through the state diagram determined by the information sequence and noting the corresponding outputs on the branch labels. Following the last information block, a terminated encoder is returned to state So by a sequence of m blocks appended to the information sequence. For in Figure 11.13(a), if 1!.Il = (1 1 1 0 1), encoding is terminated by appending m = 3 zeros to 1!.Il, and the resulting codeword is given by v = (11, 10,01, 0 1, 11, 1 0, 1 l, 11). We now return briefly to the subject of catastrophic encoders introduced in the previous section. An encoder is catastrophic if and only if the state diagram contains a cycle with zero output weight other than the zero-weight cycle around the state So. The state diagram for the catastrophic encoder of Example 11.10 is

488

Chapter 11

Convolutional Codes

1/10

0111

0/01 0/00

(a) 00/000

10/010

01/111

11/101 (b)

Encoder state diagrams for (a) the (2, 1, 3) encoder of Figure 11.1 and (b) the (3, 2, 2) encoder of Figure 11.2.

FIGURE 11.13:

shown in Figure 11.14. Note that the single branch cycle around the state S3 has zero output weight. The state diagram can be modified to provide a complete description of the Hamming weights of all nonzero codewords, that is, a codeword weight enumerating function (WEF) for the code. We begin by considering the case of unterminated encoders; that is, the information sequence can be of arbitrary length. State So is split into an initial state and a final state, the zero-weight branch around state So is deleted, and each branch is labeled with a branch gain Xe/, where d is the weight of the n encoded bits on that branch. Each path connecting the initial state to the final state represents a nonzero codeword that diverges from and remerges with state So exactly once. (We do not enumerate codewords that never remerge with state So, since, for noncatastrophic encoders, they must have infinite weight. Also, codewords that diverge from and remerge with state So more than once are not enumerated, since they can be considered as a sequence of shorter codewords.) The path gain is the product of the branch gains along a path, and the weight of the associated codeword is the power of X in the path gain. The modified state diagrams for the encoders of Figures 11.1 and 11.2 are shown in Figures 11.lS(a) and 11.lS(b), respectively.

Structural Properties of Convolutional Codes

Section 11.2

~8!lJ

0/00

0/01

S2 ~

--.~~~~~---r~ 1/10

0/11

1/00

FIGURE 11.14:

State diagram of a (2, l, 2) catastrophic encoder.

x'

~ >6

x2 (b)

FIGURE 11.15: Modified encoder state diagrams for (a) the (2, 1, 3) encoder of Figure 11.1 and (b) the (3, 2, 2) encoder of Figure 11.2.

490

Convolutional Codes

Chapter 11

Computing Path Gains in a State Diagram

EXAMPLE 11.11

a. The path representing the state sequence SoSiS3S7S6SsS2S4So in Figure 11.lS(a) has path gain x 2 . xi . xi . xi . x2 . xi . x2 . x2 = xi2, and the corresponding codeword has weight 12. b. The path representing the state sequence SoSiS3S2So in Figure 11.lS(b) has path gain X2 . xi. x 0 . xi = x4, and the corresponding codeword has weight 4. We can determine the codeword WEF of a code by considering the modified state diagram of the encoder as a signal flow graph and applying Mason's gain formula [15) to compute its transfer function, (11.97) where Ad is the number of codewords of weight d. In a signal flow graph, a path connecting the initial state to the final state that does not go through any state twice is called a forward path. Let Fi be the gain of the ith forward path. A closed path starting at any state and returning to that state without going through any other state twice is called a cycle. Let Ci be the gain of the ith cycle. A set of cycles is nontouching if no state belongs to more than one cycle in the set. Let {i} be the set of all cycles, {i', }'} be the set of all pairs of nontouching cycles, {i", j", !"} be the set of all triples of nontouching cycles, and so on. Then, we define (11.98) i 11 ,j",l"

i 1 ,j1

where .L; C is the sum of the cycle gains, Li',J' C;,CJ' is the product of the cycle gains of two nontouching cycles summed over all pairs of nontouching cycles, L;",J"J" C;"C}"Cz is the product of the cycle gains of three nontouching cycles summed over all triples of nontouching cycles, and so on. Finally, ,6,.; is defined exactly like ,6,. but only for that portion of the graph not touching the ith forward path; that is, all states along the ith forward path, along with all branches connected to those states, are removed from the graph when computing ,6,.;. We can now state Mason's formula for computing the transfer function A(X) of a graph as 11

A(X)

=

~ F,6,.·

Li ,6,.,

l '

(11.99)

where the sum in the numerator is over all forward paths. We give several examples to clarify this procedure. EXAMl?lE 11.12

Computing the WEf of a (2, 1, 3) Code

Consider the code produced by the (2, 1, 3) nonsystematic feedforward encoder of Figure 11.1, whose modified state diagram is shown in Figure 11.lS(a). There are 11 cycles in the graph of Figure ll.15(a):

Section 11.2

1: Cycle 2: Cycle 3: 4: Cycle 5: 6: 7: 8:

Cycle 9: 10: Cycle 11:

S1S3S7S6SsS2S4S1 S1S3S7S6S4S1 S1S3S6SsS2S4S1 S1S3S6S4S1 S1S2S5S3S7S6S4S1 S1S2S5S3S6S4S1 S1S2S4S1 S2SsS2 S3S7S6SsS3

S3S6SsS3 S7S7

of

There are 10 Cycle Pair 1: Cycle Pair 2: Pair 3: Cycle Pair 4: Cycle Pair 5: Cycle Pair 6: Cycle Pair 7: Cycle Pair 8: Cycle Pair 9: Cycle Pair 10:

Structural Properties of Convolutional Codes

~!li1

(C1 = X 8 ) (C2 = X 3) (C3 = X 7) (C4 = X 2) (Cs = X 4) (C6 = X 3) (C7 = X 3) (Cs= X) (C9 = X 5 ) (C10 = X 4) (C11 = X)

cycles:

(Cycle 2, Cycle 8) (Cycle 3, Cycle 11) 4, Cycle 8) (Cycle 4, Cycle 11) ( Cycle 6, Cycle 11) (Cycle 7, Cycle 9) 7, Cycle 10) (Cycle 7, 11) ( Cycle 8, 11) (Cycle 10, 11)

(C2Cs = X 4) (C3C11 = X 8 ) (C4Cs = X3) (C4C11 = X3) (C6C11 = X 4) (C7C9 = X 8 ) (C7C10 = X 7) (C7C11 = X 4) (CsC11 = X 2 ) (C10C11 = X 5 )

There are 2 triples of nontouching cycles: Cycle Triple 1: (Cycle 4, Cycle 8, Cycle Triple 2: (Cycle 7, Cycle 10, Cycle 11)

(C4CsC11 = X 4) (C7C10C11 = X 8 )

There are no other sets of nontouching cycles. Therefore, Ci

= 1-

+ + x 2 + x 4 + x 3 + x 3 + x + x 5 + + X) xs + x3 + x3 + x4 + xs + x7 + x4 + x2 + xs) - cx4 +

cx 8 + x

+cx4 +

= 1- 2X -

3

X3 .

(11.100)

There are 7 forward paths in the graph of Figure 11.15(a): Forward Path 1: Forward Path 2: Forward Path 3: Forward Path 4: Forward Path 5: Forward Path 6: Forward Path 7:

S0S1S3S7S6SsS2S4So S0S1S3S7S6S4So

S0S1S3S6SsS2S4So S0S1S3S6S4So S0S1S2SsS3S7S6S4So SoS1 S2Ss S3S6S4So S0S1S2S4So

xs)

(F1 = x12) (F2 = X 7 ) (F3 = X 11 ) (F4 = X 6 ) (F5

= X 8)

(F6 = X 7 ) (F7 = X 7 )

492

Chapter 11

Convolutional Codes

Forward paths 1 and 5 touch all states in the graph, and hence the subgraph not touching these paths contains no states. Therefore, fl.1

=

fl.s

= 1.

(11.101)

The subgraph not touching forward paths 3 and 6 is shown in Figure ll.16(a), and hence, (11.102) fl.3 = fl.6 = 1 - X. The subgraph not touching forward path 2 is shown in Figure ll.16(b ), and hence, (11.103) The subgraph not touching forward path 4 is shown in Figure ll.16(c), and hence, fl.4 = 1 -- (X + X) + (X 2) = 1 - 2X + X 2 .

(11.104)

The subgraph not touching forward path 7 is shown in Figure 11.16( d), and hence, fl.7 = 1 - (X + X 4 +XS)+ (XS)= 1 - X - X 4 .

(11.105)

The transfer function for this graph, that is, the WEF of the associated (2, 1, 3) code, is then given by x 12 · 1 +

x 7 (1 x

8 ·

X) + X 11 (1 X) + X6 (1 - 2X + X2 ) + 1 + X 7 (1 - X) + x 7 (1 - X - X4 )

A(X)=~~~~~~~-1--~2X~--X~~~~~~~~

3

(11.106)

x6 + x1 - xs 1-2X=

x 6 + 3X7 + sx 8 + 11x9 + 2sx 10 +

...

(a)

0

X

X

X

fiJ

1

X

-

(b)

fiJ

(c)

FIGURE 11.16:

(d)

Subgraphs for computing fl.; in Example 11.12.

Section 11 .2

o-f Convolutional

The codeword WEF A(X) of the of all nonzero codewords that from and remerge with state In this case there is one such codeword of 6, three of 8, and so on. ,~~~--·----------------

--------

---•--

---~-,-~·--

Consider the code the Figure whose modified state shown that (see Problem 11.17) there are 8 1 of nontouching in the £1

= 1-

()(2

+

:rnd

+x+

+

-~- ){ -1-

+(X6+

= 1-2X -

L F;

f:,,;

=

and

in the

There are 15 forward

+

(1 - X -

+

-1

+

+ x 8 .1 + (1 -

+ = 2X

3

+

+

(1- X 'II

-_/!_

(1 -

·1+

+ +

X6(1 -

+

-

+

x 6 .1 + (1 -

(1- X)

-\-

+ x 6 .1 + xs + x6 -

Hence, the codeword WEF is given

A(X)=

~~~~~~~~~~~

1 - 2x - 2x 2

+

-

x3

+ x~ + xs

sx 4 + 15X5 + ....

and this code contains tvvo nonzero codewords of of weight 5, and so on. Additional information about the weight of a code can be obtained using essentially the same H the modified state labeling each branch to a nonzero information block w is the weight of the k information bits on that with L, then the codeword input-output the code is given by A(W.X.

=

~ L w.d.l

'

-vw

Aw.d.f\l

494

Chapter 11

FIGURE 11.17:

Convolutional Codes

Augmented modified state diagram for the (2, 1, 3) encoder of

Figure 11.1. The coefficient Aw.cl.I denotes the number of codewords with weight d, whose associated information weight is w, and whose length isl branches. The augmented modified state diagram for the encoder of Figure 11.1 is shown in Figure 11.17. IEXAMf'ILIE

n .12

For the graph of Figure 11.17, it can be shown that (see Problem 11.18) 6.

+ X3W3L5 + x7w3L6 + x2w2L4 + x4w4L7 + X3W3L6 + X 3WL 3 + XWL 2 + x 5 w 3L 4 + x 4 w 2L 3 + XWL) + (X 4W 4 L 7 + x8w4L7 + x3w3L6 + X3W3L5 + x4w4L7 + X8W4L7 + x7w3L6 + x 4w 2L 4 + x 2 w 2L 3 + x 5 w 3L 4) - (X 4W 4L 7 + x 8 w 4L 7) = 1 - XW(L + L 2) - X2 W2 (L 4 - L 3 ) - x 3 w L 3 - x 4 w 2(L 3 - L 4 ) =1_

(X8W4L7

(11.111) and

L Fit..i

=

x 12 W 4 L 8 .1

+ x 7 w 3L 6(l

+x 6 w 2L 5 [l - XW(L +X7 W 3 L 7 (l - XWL)

- xw L 2) + x 11 w 3L 7(1- xw L)

+ L 2 ) + X 2W 2 L 3 ] + X 8 W 4 L 8 • l + X 7 WL 4(l - XWL - X 4W 2L 3) (11.112)

Section 11.2

Structural Properties of Convolutional Codes

Hence, the codeword IOWEF is

by

+ x1wL4 -

x6w2Ls A(W,

L) = 1-

+ L2) _

x8w2Ls

x2w2(L4 _ L3) _ )(3WL3 _ )(4W2(L3 _ L4)

+

=X6W2L5+

tlil!il:Si

L6+W3L7)

(11.113)

This implies that the codeword of weight 6 has length-5 branches and an information weight of 2, one codeword of 7 has branches and information 1, another has length-6 branches and information weight 3, the third has length-7 branches and information weight 3, and so on. The term W p+l in A(W, L), corresponding to information weight 1, always represents the shortest the augmented modified state diagram. H is interesting to note, however, that this may not be the minimum-weight path. Kn Example 11.12, for instance, the term X 7 W L 4 represents the shortest path, whereas the term x 6 w2 L 5 represents the minimum-weight path. Also, it is important to note that the codeword WEF A(X) is a property of the code; that is, it is invariant to the encoder representation. The codeword IOWEF A(W, X, L), on the other hand, is a property of the encoder, since it depends on the mapping between input sequences and codewords (see Problem 11.21). An alternative version of the IOWEF, which contains only information about the and output weights but not the lengths of each can be obtained by simply setting the variable Lin A(W, L) to unity; that A(W, X)

=

L Aw,d ww xd = A(W,

L)IL=l,

(11.114)

w,d

where Aw,d = Lt Aw,d,t represents the number of codewords with weight d and information weight w. Similarly, the WEF A(X) is related to the IOWEF as A(X)

=

L Ac1Xc1 = A(W, X)lw=l = A(W,

(11.115)

d

where Ac1 = Lw Aw,d· In Chapter 16 we will see that an important tool needed to compute the WEF of turbo codes is the codeword conditional weight enumerating function (CWEF), which enumerates the weights of all codewords associated with particular information weights. For an information weight of w, the CWEF is given by (11.116) Aw (X) = Auul xd.

L d

H follows directly from (11.114) that the IOWEF can be expressed as A(W, X)

=

L wwAw(X). w

Clearly, the CWEF is also a property of the encoder.

(11.117)

496

Chapter 11

Convolutional Codes

EXAMPLE 11.12

(Conrtirmedl)

The simplified IOWEF of (11.113) becomes A(W, X)

=

A(W, X, L)IL=l

=

2

W X

6

+ (W + 2W 3 )X7 + (W 2 + 4W 4 )X 8 + · ·., (11.118)

indicating one codeword of weight 6 with information weight 2, three codewords of weight 7, one with information weight 1 and two with information weight 3, and so on, and we can obtain the WEF as A(X)

=

A(W, X)lw=l

= x6 +

3X 7 + 5X 8 + · · · ,

(11.119)

which agrees with the A(X) computed previously in (11.106). To compute the CWEFs for this encoder we must find the path weights associated with the information sequences of a given weight that traverse the modified state diagram from the initial state to the final state. There is one such information sequence of weight 1, u = (1 0 0 0), three such sequences of weight 2, u = (1100 0), (101000), and (100 100 0), nine such sequences of weight 3, u = (111 0 0 0), (11 0 100 0), (1 0 11 0 0 0), (1 0 1 0 1 0 0 0), (11 0 0 1 0 0 0), (1 0 0 11 0 0 0), (1 0 1 0 0 1 0 0 0), (1 0 0 1 0 1 0 0 0), and (1 0 0 1 0 0 1 0 0 0), and so on. (The last three O's in each sequence represent the termination bits.) Thus, using Figure 11.17, we see that A1(X) = X 7 , (11.120a) A 2 (X) =

x 6 + x 8 + x 10 ,

(11.120b)

and A3(X)

= 2X 7 +

3X 9 + 3X 11 + x 13 .

(11.120c)

Finally, using (11.117), we can write A(W, X) =

L Ww Aw(X) w

= wx

7

+

w cx 2

6

8

+ x + x

10

)

(11.121) 3 7 9 11 13 + w c2x + 3X + 3X + x ) + ....

Because of the different ways in which they were calculated, it is not immediately obvious that (11.118) and (11.121) give the same result; however, if we rewrite A(W, X, L) in (11.113) listing the numerator and denominator terms in increasing powers of W, and then set L = 1, we obtain A(W. X)

=

wx7 + w2cx6 -

xs)

-l---W-(2_X_+_X_3_)_

the same result as in (11.121 ).

(11.122)

Section 11.2

Structural Properties of Convolutional Codes

42l7

vv•"~'""' V"'A"''-''v illustrates that if we wish to express the IOWEF in terms of codeword weight, we should list the numerator and denominator terms in A(VV, X) in increasing powers of X to however, if we wish to express the IOWEF in terms of information weight, we should list them m powe~ofVV. consider the case of terminated encoders whose sequences are limited to Jc = h + m blocks, where >c the total length of the sequence, which h information blocks and m termination blocks. the for WEFs to terminated encoders, we write the numerator and denominator terms of the IOWEF A(W. X. L) in increasing powers of L. Then, we perform division and drop terms of degree larger than LA from the power series.

(CtJJlfiltamJJte«ll} VVe

by

(11.113) as

A(W,X,

(11.123) let the encoder be terminated after >c = 8 blocks; that is, consider only sequences with h = 5 information blocks and m = 3 termination blocks. Then, division and truncation of terms with degree larger than L 8 gives (see Problem 11.22) A(W,X.

(11.124) as the IOV\/EF of this terminated encoder. There are exactly 15 terms in this IOWEF, because we consider only information sequences whose first bit is a 1, and we do not consider the information sequence 1ll1 = (1 0 0 0 1 0 0 0). since it diverges from and remerges with state So twice. (Note that W5 is the largest input weight represented in (11.124), since Jc - m = 5.) We can now compute the simplified IOWEF, the CWEFs, and the WEF of this terminated encoder as A(W, X) = A(W, X, L)IL=l = WX

7

+ w2(x 6 + (11.125a)

A1 (X) A 2 (X)

A3(X)

=

(11.125b) 6

= x +x +x • = 2X 7 + 3X 9 + x 11 ,

(11.125d)

+ xrn + x12.

(11.125e)

8

A.i(X) = 2xs As(X) =

x

9

,

10

(11.125c)

(ll.125f)

498

Chapter 11

Convolutional Codes

and A(X)

=

A(W,

X)lw=l

=

X

6

+ 3X7 + 3X8 + 4X 9 + 2X 10 + xli + xi 2 ,

(11.125g)

respectively. Note that the CWEF A3(X) given in (11.125d) for the terminated encoder differs from the expression given in (11.120c) for the unterminated encoder. This is because some of the weight w = 3 information sequences in the unterminated case are more than h = 5 bits long, and thus they are not valid input sequences to the terminated encoder. It is important to note here that the foregoing method for computing codeword WEFs considers only nonzero codewords that diverge from the all-zero state at the initial time unit and remerge exactly once. In other words, delayed versions of the same codeword or codewords that diverge and remerge more than once are not counted. (The reason for enumerating the weights of nonzero codewords in this fashion is related to the method used to calculate the bit-error probability of convolutional codes with maximum likelihood decoding, a subject that will be discussed in Chapter 12.) Thus, in the case of a terminated encoder with an information sequence of length h = 5 considered in the preceding example, only 15 nonzero codewords are counted, rather than the 25 -1 = 31 nonzero codewords that exist in the equivalent (N, K*) = [n(h + m), kh] = (16, 5) block code. Methods for calculating the complete WEF of the equivalent block code obtained by terminating a convolutional encoder are needed to compute the WEFs of turbo codes. These methods are discussed in more detail in Chapter 16. To determine the information bit-error probability of convolutional codes with maximum likelihood decoding, a subject to be discussed in the next chapter, it is convenient to modify the codeword WEF expressions introduced previously to represent the number of nonzero information bits associated with codewords of a given weight. These modified expressions are referred to as bit WEFs. For an information weight of w, the bit CWEF is given by Bw(X)

=

L

Bw,dxd,

(11.126)

d

where Bw,d = (w/k)Aw,d· Hence, Bw,d can be interpreted as the total number of nonzero information bits associated with codewords of weight d produced by information sequences of weight w, divided by the number of information bits k per unit time. It follows directly that the bit IOWEF can be expressed as (11.127) w.d

w

and the bit WEF is given by B(X)

=

L

BdXd

=

B(W,

X)lw=l,

(11.128)

d

where Bd = Lw Bw,d is the total number of nonzero information bits associated with codewords of weight d, divided by the number of information bits k per unit

of

Section 11.2

time. fo this case, all three bit the encoder. We now note that the definitions of any wand d,

01

fo~·

here

w

w

Equation can then be used to establish the between the codevvord IOV,/EF m1d the B(X)

I )A = ~( Le,\U) /(

=

I

w.d

d

a -'--'--"-----a-v_v___ k1 -----' .

that

-;7(]

W.dA

I W=l

--~1 l

1(

F=l

'

fron1 the co,devvord I0:· 1.}/JEF

can be calculated

the bit VVEF

, ... -

cJ VV

A(W, X).

As will be seen in the next , the decisions rnade a~1 I\/ll.,D for convolutional codes at each time unit result in the ·u,"v,.,·u,u Thus, to determine the per include factor of (1/ k) in the bit WEFs defined here. For terminated convolutior,aJ we will also consider maximum a This method of 1s this case, decisions are made block of information bits c1J one and the WEFs are defined fo , the J[OVJEF A(W, X) is modified to include all 2K - 1 = 2"1, - 1 nonzero code·01ords that exist in the equivalent (JV. = [11 (h + kh] block code, veTsions of a codeword and codewords that and remerge with the c1J1 . . ze.w state more than once, and the codeword VVEFs are modified if K = k(h + 111) bits to the Bw.d = (w/ K)Aw.d, and Then, Bw.d the total number of nonzern infom12tion bits a::sociated vvith codewords of information sequences of the total number of input bits K in a block. since K = kh of the K bits are information but since h is with m, we usually ignore this These modified V:\/EFs are discussed in more detail in Chapter 16.

Applying (11.130) to the codeword J[OWEF in this example, we obtain the bit WEF

for the

1 cJA(W,

B(X)

= k

aw

a[w 2 x 6 +cw+ 2w 3 )X 7 +

aw

+

+ ... ] I

L 3) encoder

SOll[J

Chapter 11

Convolutional Codes

The coefficients of B(X) in (11.131) represent the total number of nonzero information bits associated with codewords of each weight for this encoder. We will see in the next chapter how the bit WEF B(X) can be used to determine the bit-error probability associated with maximum likelihood decoding of convolutional codes. For systematic encoders, since k information sequences and (n - k) parity sequences comprise a codeword, the weight of a codeword is the information weight w plus the weight of the parity sequences. In this case, it is convenient to express the codeword CWEF as (11.132) where Aw.z is the number of codewords with information weight w and parity weight z. (We note here that the number of codewords Aw.z with information weight w and parity weight z is the same as the number of codewords Aw,d with information weight wand total weight d = z + w.) Then, the codeword input redundancy weight enumerating functions (IRWEFs) are given by A(W. Z, L)

=

L

(11.133)

Aw.z.lwwzzLI,

w.z.l

where Aw.z.l is the number of codewords of length l with information weight w and parity weight z, and A(W, Z)

=

L Aw,z Ww zz = A(W, Z, L)IL=l = L Ww Aw(Z).

(11.134)

w.z

w,::.:

H follows that the codeword WEF can be expressed as A(X)

=

L Ac1Xc1 = A(W, Z)lw=Z=X,

(11.135)

d

where Ac1 = Lw+z=d Aw.z is the total number of codewords of weight d. (The notation Lw+z=d Aw.z implies that we are summing over all coefficients Aw,z such that w + z = d.) The bit WEFs for systematic encoders are determined in the same way as in (11.126)-(11.128), with Bw.z = (w/ k)Aw.z (or, for terminated encoders, with Bw.z = (w/ K)Aw.J, For an information weight of w, the bit CWEF is given by (11.136)

and Bw.z can be interpreted as the total number of nonzero information bits associated with codewords of parity weight z and information weight w, divided by the number of information bits k per unit time. It follows directly that the bit IRWEF can be expressed as D/T1!"7

'7\

DlVV, L,J

=

~

Jrb

"!;F1JJ r,?

LDw.z•V w,::::

L,"

=

~

L

w

Tv1J) r.

v

/'"""7\

Du,lL,),

(11.137)

Section 11.2

Structural Properties of Convolutional Codes

!Si(Q)i

and the bit WEF is B(X)

=

LBc1Xc1

=

(11.138)

B(W. Z)lw=Z=X,

d

where Bc1 = Lw+:=d Bw,: is the total number of nonzero information bits associated with codewords of weight d, divided by the number of information bits k per unit time. Finally, (11.137) and (11.138) can be used to develop the formal relationship betvveen the codeword IRWEF A(W, and the bit WEF B(X): B(X)

=

lw=Z=X = L V,

B(W,

w.:::

=

! w a[Lw.:

-WW z:]

aw

k

w . .: :

I

=

W=Z=X

(11.139)

! w aA(W, Z) I aw

k

;

W=Z=X

that is, the bit WEF B(X) can be calculated directly from the codeword KRWEF A(W.

Again, for systematic encoders, the codeword WEF is a property of the code and the codeword CWEFs and IRWEFs, and the bit CWEFs, KRWEFs, and WEF are an properties of the encoder. To determine the IRWEF of a systematic encoder using the transfer function method introduced here, we replace the labels Xd representing codeword weights in the augmented modified state diagram with labels z: representing parity weights. We now illustrate the techniques for finding WEFs for systematic encoders with an example using a systematic feedback encoder. we note that for feedback encoders, the bits needed to terminate the encoder are, in general, nonzero. in this case, we refer to the powers of W in the WEFs as nonzero input bits rather than nonzero information bits, since termination bits are not considered information bits; however, this distinction is important only for low-weight sequences. lE}{AMrllE H.1dJJ lE&'1!«:1Q11DJ<err

Consider the (2, 1, 2) systematic feedback convolutional encoder whose generator matrix is given by rG(D)

=

(11.140)

[1

The controller canonical form realization and augmented modified state diagram for this encoder are shown in Figure 11.18. Following the same procedure as in Examples 11.12 and 11.13, we see that there are three cycles in the graph of Figure 11.18(b): Cycle 1: S1S3S2S1 Cycle 2: S1S2S1 Cycle 3: S3S3

(C1 (C2 (C3

= z2L3) = WL 2 ) = WL)

We also see that there is one pair of non touching cycles: Cycle Pair 1: (Cycle 2, Cycle 3)

(C 2 C 3

=

W 2 L 3).

502

Chapter 11

Convolutional Codes ~----------------+-VIOJ

~----------i.._, +

v(ll

(a)

®

WZL L (b)

FIGURE 11.18: (a) The controller canonical form realization and (b) the augmented modified state diagram for a (2, 1, 2) systematic feedback encoder.

In this case there are no other sets of nontouching cycles. Therefore, ~= 1

2 3

(Z L

+ WL 2 + WL) + (W 2L 3) =

1- Z 2 L 3

W(L +L 2 ) + W 2L 3 . (11.141)

We now see that there are two forward paths in the graph of Figure 11.lS(b): Forward Path 1: s0 s1 s2 s0 Forward Path 2: SoS1S3S2So

= (F2 =

(F1

W 3 z2L 3) W2 Z 4L 4)

The subgraph not touching forward path 1 is simply the single branch cycle around state S3, that is, cycle 3. Therefore ~1

= 1- WL.

(11.142)

Forward path 2 touches all states in the graph, and hence, ~2

= 1.

(11.143)

Finally, applying (11.99), we obtain the codeword IRWEF: A(W, Z, L)

=

w3 z 2 L 3 (1- WL) + w2 z 4 L 4 .1 1 - z2L3 - W(L

+ L 2) + W 2L 3

w2z4L4 + w3z2L3 - w4z2L4 1 - z2L 3 - W(L + L 2) + W 2 L3 - w2z4L4 ..L w3 - Fez. L) ' "

I

z2L3 ..L z4(Ls + L6) F(Z, L) ' p2cz, L)

L

(11.144)

l

..1- •.•

J'

Section 11 ,2

Convolutional Codes

Structural

5(Q)3

,Nhere we have defined , L) = 1 , In this case we see that there are an infinite number of codewords corresponding to input sequences of 2, (In the state diagram, 1 has zero weight associated with it, so that an infinite number of codewords with input 2 can be traversing this cycle an number of times,) We can represent these codewords expanding the first term in (1L144) as follows: 10

F(Z,L)

+

0

0

0

(1L145)

),

that is, corresponding to weight-2 sequences, there are codewords with parity 4 + 2j and 4 + 3j, j = 0, L 2, ... , owing to the gain C1 = Z 2 L 3 of 1. Similarly, there are an infinite number of codewords corresponding to input sequences of weight 3, and so on. We can now obtain the IRWEF without length information from (11.144) as follows:

w2z4 + w3z2 -

= A(W Z L)IL 1 = - - ~ - - - - ~

' · '

·

z2 -

1

=

2w

w2z4

+ w2

J

[

= - - + w3 - - + - - + ... F(Z)

where

=

p2

1 - Z 2 , and vve can use (11.134) to obtain the A2(Z)

= - -2 ,

(11.146) '

CWEFs: (11.147a)

1- Z

+ 1-2z2 + z 4 '

(11.147b)

and so on.His

to note here that (11.147a) implies that all input weight-2 of at least 6 weight 2 plus parity of at least 4), whereas that there is one weight-3 codeword with a total weight of the minimum-weight codeword in this case is a weight-3 input sequence. Finally, the codeword WEF is obtained by setting W = Z = X in (11.146), giving the expression A(X)

=

A(W, Z)lw=Z=X 175

=

_L"_,-

1-2X

=

x6 + xs - x6 ,

1- X 2 -2,,C

+ X2

(11.148)

= X5 + 2X 6 + 4X 7 + 8X 8 + ....

(Note that the substitution W = Z = X is made directly into the rational function representation of (11.146) rather than into the series representation in order of increasing weight, since this results in a simplified expression for A(X).) We see from (11.148) that this code contains one codeword of weight 5, two of weight 6. four of weight 7. and so on. In Problem 11.23 it is shown that the equivalent nonsystematic feedforward encoder, which produces the same code as

504

Chapter 11

Convolutional Codes

the systematic feedback encoder of (11.140), has the same codeword WEF A(X) as the one in (11.148). But its IOWEFs and CWEFs represent different input-output relationships than the IRWEFs and CWEFs computed previously, because these functions are properties of the encoder. Finally, following (11.136)-(11.138), the bit CWEFs are given by 2z 4

B2(Z) 3Z 2

B3(Z)

(11.149a)

= 1 - z2,

= 1z2 +

6Z 4 1 - 2z2 + z4,

(11.149b)

the bit IRWEF is given by B(W, Z)

=

L Ww Bw(Z) w

2w 2 z 4 = 1 - z2

+

3w 3 z 2 1 - z2

+

(11.150)

6W 3 Z 4 1 - 2z2 + z 4 + · · · ·

and the bit WEF is given by B(X)

=

B(W, Z)lw=Z=X

3X 5

= 1X2 +

2x 6 1 - x2

+

6X 7 1 - 2x2 + x 4 + · · ·

(11.151)

·

We can also use (11.139) to calculate B(X) as follows: B(X)

=

! w aA cw. Z) I aw

k

=w

a[(w2z4

W=Z=X

+ w3z2 -

w4z2)/(l - z2 - 2w + w2)] I

aw

(11.152) W=Z=X

I

-

N(W, Z) D(W, Z) W=Z=X'

where N(W, Z)

= 2w 2(z 4 - z 6 ) + w 3 (3Z 2 - 5Z 4) - 2W 4(Z 2 - 2z 4 + 3Z 2) +7w 5 z 2

2w 6 z 2

(11.153a)

and D(W, Z) = 1 - 2Z 2 + Z 4 - 4W(l - Z 2) + 2W 2(3 - Z 2) - 4W 3

+ W4.

(11.153b)

Now, carrying out the division we obtain (see Problem 11.24) B(X)

= 3X5 + 6X 6 + 14X7 + 32X 8 + · · · ;

(11.154)

that is, the weight-5 codeword has information weight 3, the two weight-6 codewords have total information weight 6, and so on. (Because k = 1 in this case,

Section 11.2

Convolutionai Cocles

Structural

5(115

the coefficients of B(X) the total information (11.151) and codewords of a given three terms to give the same on input then the coefficients of B()() in the ers of x 8 . In calculate 11.14 we saw that there was h1 the state an infinite number of codevvords with that allowed the encoder to are characteristic of all feedback weight 2. These 1 drives the encoder away from the all-2/:;ro state, encoders, in which a is traversed some number of and after the zero-input another input 1 returns the encoder to the all-zero state. VVe vvill see in ~,",,,.;c,n that the codevvords these sequences a critical role in the of turbo codes. vVe now introduce an the state volutional encoders. The diagram of the encoder as a visualization of the set of all the all-zero state it becomes difficult to apply once the overall constraint length v exceeds 3, that once the state diagram contains more than 8 states. A more efficient method the transfer function of an encoder is to use a state the encoding equations. We illustrate the technique example.

In the augmented modified state that represents the weights of an 2, 3. Then, we can write the

of Figure let be a state variable from the initial state So to the state S;, i =

+L :E2 = VVL:E1 + ZL

I:1 = VVZL

and the overall transfer function of the A(VV, Z,

1s

=

VVZLI:2.

in the state variables Z:::1,

Equations (11.155) are a set of linear :E3. We can rewrite them in matrix form as 1

-L

-WL

1 0

-ZL

0 -ZL

1-WL

and

(1

51Jl6

Chapter 11

Convolutional Codes

Using Cramer's rule to solve (11.157) for the state variable I:2, we obtain WZL -ZL I:2 =

0 ] -ZL 1-WL

0 0

dct [ - ~L

1 dct [ -WL

-L

0

1 0

-ZL l-WL

-ZL

]

(11.158)

wz 3L 3 + w 2 zL 2 - w 3zL 3 1 - Z 2 L 3 - W(L + L2) + W2L3.

Now, we apply (11.156) to find the IRWEF: A(W, Z, L)

=

WZL:E2 w2z4L4 + w3z2L3 - w4z2L4 1 - z 2L 3 - W (L + L2) + wz L3'

(11.159)

which is the same result obtained using the signal flow graph approach. H should be clear from the preceding examples that the transfer function approach to finding the WEFs of a convolutional encoder quickly becomes impractical as the overall constraint length v becomes large. As will be seen in the next section, however, the important distance properties of a code necessary to estimate its performance at moderate-to-large SNRs can be computed without the aid of the transfer function. Nevertheless, the transfer function remains an important conceptual tool, and it will be used in Chapters 12, 16, and 18 to obtain performance bounds for convolutional codes used with maximum likelihood decoding. 11.3

lDl!STAN(IE IPIROIPIElrmlES OIF (ONVOll!Jl!ONAL CODES

The performance of a convolutional code depends on the decoding algorithm employed and the distance properties of the code. In this section several distance measures for convolutional codes are introduced. Their relation to code performance will be discussed in later chapters. The most important distance measure for convolutional codes is the minimum free distance dtree· JD)JEnNnnoN

ll.1.7

The minimum free distance of a convolutional code is

defined as

.6_

•

I

II

dfree = ~l~{d(w, W ) : 11R nn .nn

I

fl

=/=

l!ll },

(11.160)

where w' and w" are the codewords corresponding to the information sequences un". respectively. 0

11R and

In (11.160) it is assumed that the codewords w' and w" have finite length and start and end in the all-zero state So: that is. m termination blocks are appended to the information sequences l!R and uu". H llll and llll have different lengths, zeros are added 0

0

0

Section 11.3

Distance Properties of Convolutional Codes

51(H'

to the shorter sequence so that the corresponding codewords have Hence, dtree is the minimum distance between any two finite-length codewords in the code. Because a convolutional code is a linear code,

(,/ + v")

= , ,,

dtree

: un' =fa un"}

1l11 ,1l11

= min{w(v):

llil

1l11

(11.161)

=fa QJJ}

= min{w(un
QJJ},

where ,'f is the codeword to the information sequence 11.Il. Hence, dtree is the minimum-weight codeword produced by any nonzero information sequence. Also, it is the minimum weight of all finite-length in the state that diverge from and remerge with the all-zero state So, and hence it is the lowest power of X in the WEF A(X). For example, dtree = 6 for the (2, L 3) code of Example 11.12, and = 3 for the (3, 2. 2) code of Example 11.13. Another important distance measure for convolutional codes is the column distance function (CDF). Let [v] 1

=

(v

(0)

0

(1)

v0

(n-1)

· · · v0

,

(0)

(1)

v1 v1

(11-l)

· · · v1

(0)

• · · · , v1

(1)

v1

(11-l))

· · • v1

(11.162)

denote the /th truncation of the codeword v, and let (11.163)

denote the /th truncation of the information sequence un. [l)JEnNnmN HJ~

The column distance function of order l. d1, is defined as ,6,_

d1

I

=

II

I

!!

min {d([Y ]1. [Y ]1): [llll lo f- [un ]o} [llll']1.[1l11"]1

= min{w[Y]1): [un]o f-

(11.164) @),

[1l11]/

where Y is the codeword corresponding to the information sequence un. Hence, d1 is the minimum-weight codeword over the first (/ + 1) time units whose initial information block is nonzero. fo terms of the generator matrix of the code. (11.165) [Y]1 = [UlJ ]1 [(G ]1, where [
[

= [


J

•

I::: m.

+ 1) columns of
(11.166a)

Chapter 11

508

I [G]1

=

Go

G1 Go

Convolutional Codes

Gm-1

Gm

Gm-2

Gm-1

G,,,

G1 Go

G2 G1 Go

G3 G2 G1

l Gm G 111 _1

Gm

Gm-2

Gm-l

, l > m.

Go

(11.166b) Then, di= min{w([llll]1[G]1): [llll]o -IO} (u];

(11.167)

is seen to depend only on the first n(l + 1) columns of G. This accounts for the name "column distance function." We note here that the form of the generator matrix G used in (11.166) (and in (11.22)) assumes a feedforward encoder realization, that is, a polynomial generator matrix with memory order m. For feedback encoders with rational function generator matrices, the associated generator sequences have infinite duration, and the k x n submatrices G1 extend infinitely to the right in (11.166). In this case, the CDF can still be expressed using (11.167). We also note that the free distance dtree is independent of the encoder realization, that is, it is a code property, whereas the column distance function dz is an encoder property; however, if we restrict our attention to encoders for which the k x n submatrix Go has full rank, then the CDF d1 is independent of the encoder realization. (This results in the sensible property that a codeword is not delayed with respect to its information sequence.) Thus, we will refer to the CDF dz as a code property. Definition 11.8 implies that d1 cannot decrease with increasing l; that is, it is a monotonically nondecreasing function of l. The most efficient way of computing the CDF of a code is to modify one of the sequential decoding algorithms to be introduced in Chapter 13 (see Section 13.4). The complete CDF of the (2, 1, 16) nonsystematic code with G(D) = [1 + D + D 2 + Ds + D6 + D 8 + D 13 + D 16, 1 + D 3 + D 4 + D 7 + D 9 + D 10 + D 11 + D 12 + D 14 + D 15 + D 16 ] is shown in Figure 11.19. Two cases are of specific interest: l = m and l ---+ oo. For l = m, d111 is called the minimum distance of a convolutional code. Hence, d 111 is also denoted by dmin· From (11.167) we see that dmin represents the minimum-weight codeword over the first (m + 1) time units whose initial information block is nonzero. For the code of Figure 11.19, d111 ; 11 = d16 = 9. Much of the early work in convolutional codes treated d 111 ; 11 as the distance parameter of most interest, because the principal decoding techniques at that time had a decoding memory of (m + 1) time units. (See Section 13.5 on majority-logic decoding of convolutional codes.) More recently, as maximum

Section 11.3

of Convolutional Codes

5,(Q)!9J

d/,ee = 18 ---------------------------------------

18

16 14 12

8 6

4 2 0'--~~~--"~~~~--'~~~~--"-~~~~-'-~~~~-"---~'--~~~

0

10

20

30

40

50

60

FIGURE 11.19: The column distance function of a (2, L 16) code.

maximum a (MAP) decoding, and sequential likelihood (ML) dmin as decoding have become more prominent, dtree and the CDF have the distance of since the decoding memory of these is unlimited. A thorough discussion of the between distance measures and is included in Chapters 12 and 13. For I -> co, cit is the minimum-weight codeword of any length whose first information block is nonzero. Comparing the definitions of lim1---,. 00 d1 and dfree, we can shovv that (see Problem 11.31) for noncatastrophic encoders · lim di

/-+CO

= djree.

(11.168)

Hence, d1 reaches and then it stays constant. This usually happens when l reaches about 3v. For code of Figure 11.19 with v = 16, d1 = 18 for l c:".. 53, and hence, dti'ee = 18. Because djree is defined for finite-length codewords, however, (11.168) is not necessarily true in the case of catastrophic encoders.

encoder whose state diagram is shown in Figure 11.14. Consider again the For this encoder, do = 2, and d1 = d2 = · · · = lim1---,. 00 d1 = 3, since the truncated informationsequence[illl]1 = (1, 1, l, .. ·, l)alwaysproducesthetruncatedcodeword

510

Chapter 11

Convolutional Codes

[v]1 = (11, 01, 00, 00, · · · , 00), even in the limit as l -+ oo. Note, however, that all finite-length paths in the state diagram that diverge from and remerge with the all-zero state So have a weight of at least 4, and hence, dtree = 4. In this case we have a situation in which lim1--+oo d1 = 3 :/=- dtree = 4; that is, (11.168) is not satisfied. It is characteristic of catastrophic encoders that an infinite-weight information sequence produces a finite-weight codeword. In some cases, as in the preceding example, this codeword can have a weight less than the free distance of the code, owing to the zero-output weight cycle in the state diagram. In other words, an information sequence that traverses this zero-output weight cycle forever will itself pick up infinite weight without adding to the weight of the codeword. In a noncatastrophic encoder, which contains no zero-output weight cycle other than the zero-weight cycle around the state So, all infinite-weight information sequences must generate infinite-weight codewords, and the minimum weight codeword always has finite length. Unfortunately, the information sequence that produces the minimumweight codeword may be quite long in some cases, and hence the calculation of dfree can be a difficult task. The best achievable dfree for a convolutional code with a given rate R and overall constraint length v has not been determined in general; however, upper and lower bounds on dtree have been obtained using a random coding approach. These bounds are thoroughly discussed in References [16], [17], and [18]. A comparison of the bounds for nonsystematic encoders with the bounds for systematic encoders implies that more free distance is available with nonsystematic feedforward encoders of a given rate and constraint length than with systematic feedforward encoders. This observation is verified by the code construction results presented in the next two chapters and has important consequences when a code with large dtree must be selected for use with ML, MAP, or sequential decoding. Thus, if a systematic encoder realization is desired, it is usually better to select a nonsystematic feedforward encoder with large dfree and then convert it to an equivalent systematic feedback encoder.

PROBLEMS 11,1 Consider the (3, 1, 2) nonsystematic feedforward encoder with g
= (110),

g(l)

=

(101),

g< 2)

=

(111).

a. Draw the encoder block diagram. b, Find the time-domain generator matrix G. c, Find the codeword v corresponding to the information sequence 1lll = (11101). 11,2 Consider the (4, 3, 3) nonsystematic feedforward encoder shown in Figure 11.3. a. Find the generator sequences of this encoder. b, Find the time-domain generator matrix G. c, Find the codeword v corresponding to the information sequence 11!1 (llO, Oll, 101).

Problerns

5"1 i

H3 Consider the (3, 1, 2) encoder of Problem 11.1. :Bio Find the transform-domain matrix lbio Find the set of output sequences and the codevvord v(D) to the information sequence illl(D) = 1 + + + HA Consider the 2, 2) feedforward encoder shown in 11.2. ;!i;l ( D) and ~-.,-,,,,h to the set of information sequeni::es H~HH~>u

;!i;(l) 2

~( )

=

(101101).

= (110011).

find the time-domain 111a trix cG. sequences and sequence un = (1 l O 1). ]L!Di Consider the (3, 2. 3) feeclforward encoder vvith talo

!bi, Find the

(D)

to the iDfornwtion

= l +D +

:mo Draw the controller canonical form realization of this encodeL How manJ

delay elements are required in this realization? lbio Draw the simpler observer canonical form realization that three delay elements. 1JL'1 Verify the sequence of frorn the nonsysternatic feedforward realizations of the feedback realizations of and (11.71). llolli Draw the observer canonical form realization of the generator matrix (D) (11.64) and determine its overall constraint v. 1JL'1Ji Consider the rate R = feedforward encoder ,7,;ith generator matrix CG(D)

=[

D 1

l

+ Dl + D 2

J

.

Draw the controller canonical form encoder realization for vVhat is the overall constraint length v? lbio Find the generator matrix cc;' (D) of the feedbadc encoder. Is (D) realizable? If not, find an equivalent realizable generator matrix and draw the corresponding minimal encoder realization. Js this minimal realization in controller canonical form or observer canonical form? What is the minimal overall constraint length v? lloll[]) Use elementary row operations to convert the rate R = matrix of (11.77) to systematic feedback form, and dravv the minimal observer canonical form encoder realization. Find and draw a feedbar::lr controlie;canonical form encoder realization with the same number of states. HJ_]_ Redraw the observer canonical form realization of the 2. 2) feedback encoder in Figure 11.7(b) using the notation of (11.82) and the relabeling scheme of Figure 11.11. talo

512

Chapter 11

Convolutional Codes

JLloU Consider the (3, 1, 2) systematic feedback encoder shown in Figure 11.6(c). Determine the v = 2 termination bits required to return this encoder to the all-zero state when the information sequence 1lll = (10111). UoB Consider the (4, 3, 3) nonsystematic feedforward encoder realization in controller canonical form shown in Figure 11.3. taL Draw the equivalent nonsystematic feedforward encoder realization in observer canonical form, and determine the number of termination bits required to return this encoder to the all-zero state. What is the overall constraint length of this encoder realization? lbio Now, determine the equivalent systematic feedback encoder realization in observer canonical form, and find the number of termination bits required to return this encoder to the all-zero state. What is the overall constraint length of this encoder realization? UoJL4l Consider the (2, 1, 2) nonsystematic feedforward encoder with G(D) = [1 + D 2 1 + D + D 2 ]. !!lo Find the GCD of its generator polynomials. lbio Find the transfer function matrix G- 1 (D) of its minimum-delay feedforward inverse. UoJL5 Consider the (2, 1, 3) nonsystematic feedforward encoder with G(D) = [l + D2 1 + D

+ D 2 + D 3 ]. Find the GCD of its generator polynomials. lbio Draw the encoder state diagram.
JLJLJL((ii

Uo17 Uol8 HoJLil))

Uo:Z@

UJ;JL

JLlo:22 Uo23

Bibliography

SB

rue Draw the augmented modified state diagram for this encoder. !bi" Find the IOWEF A(W, X, L), the three lowest input weight CWEFs, and the

WEF A(X) for this encoder. Compare the results obtained in (b) with the IRWEF, CWEFs, and WEF computed for the equivalent systematic feedback encoder in Example 11.14. In Example 11.14, verify all steps leading to the calculation of the bit WEF in (11.154). Consider the (2, 1, 2) systematic feedforward encoder with G'(D) = [I 1 + fillo Draw the augmented modified state diagram for this encoder. !bi" Find the IRWEF A(W, Z. the three lowest input weight CWEFs, and the WEF A(X) for this encoder. Recalculate the IOWEF A(W, X, L) in Example 11.12 using the state variable approach of Example 11.14. Recalculate the WEF A(X) in Example 11.13 using the state variable approach of Example 11.14. Consider the (3, 1, 2) code generated by the encoder of Problem 11.l. mo Find the free distance dtree· lbio Plot the complete CDF. 1\'.o Find the minimum distance d 111 ; 11 • Repeat Problem 11.28 for the code generated by the encoder of Problem 11.15. mo Prove that the free distance dtree is independent of the encoder realization, i.e., it is a code property. !bi" Prove that the CDF d1 is independent of the encoder realization; that it is a code property. (Assume that the k x n submatrix G'o has full rank.) Prove that for noncatastrophic encoders !Co

lLJLZ
JLJLZl[ii

UJ\7 lLlLoZa'l

lLJLoZ~

Uo:?lij

]_JL"3]_

lim d1

/-s,oo

= dtree.

18!18UO
'.
~" A. J. Viterbi, "Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm," IEEE Trans. Inform. Theory, IT-13: 260-69, April 1967. !!t L. R. Bahl, J. Cocke, F. Jelinek, and J. Raviv, "Optimal Decoding of Linear Codes for Minimal Symbol Error Rate," IEEE Trans. Inform. Theory, IT20: 284-87, March 1974. 16io

G. Ungerboeck and I Csajka, "On Improving Data-Link Performance by Increasing the Channel Alphabet and Introducing Sequence Coding," in IEEE

International Symposium on Information Theory (!SIT 1976) Book of Abstracts, p. 53, Ronneby, Sweden, June 1976.

514

Chapter 11

Convolutional Codes

7. G. Ungerboeck, "Channel Coding with Multilevel/Phase Signals," IEEE Trans. Inform. Theory, IT-28: 55-67, January 1982. 3. C. Berrou, A. Glavieux, and P. Thitimajshima, "Near Shannon Limit ErrorCorrecting Coding and Decoding: Turbo Codes," in Proc. IEEE International Conference on Communications (ICC 93), 1064-70, Geneva, Switzerland, May 1993. 9. C. Berrou and A. Glavieux, "Near Optimum Error Correcting and Decoding: Turbo-codes," IEEE Trans. Commun., COM-44: 1261-71, October 1996. 10. A. Dholakia, Introduction to Convolutional Codes. Kluwer Academic, Dordrecht, The Netherlands, 1994. 11. L. H. C. Lee, Convolutional Coding: Fundamentals and Applications. Norwood, Mass., Artech House, 1997. 12. R. Johannesson and K. S. Zigangirov, Fundamentals of Convolutional Coding. IEEE Press, Piscataway, N.J., 1999. 13. G.D. Forney Jr., "Convolutional Codes I: Algebraic Structure," IEEE Trans. Inform. Theory, IT-16: 720-38, Nov. 1970. 14. J. L. Massey and M. K. Sain, "Inverses of Linear Sequential Circuits," IEEE Trans. Computers, C-17: 330-37, April 1968. 15. S. Mason and H. Zimmermann, Electronic Circuits, Signals, and Systems. John Wiley, New York, N.Y., 1960. 16. R. Johannesson and K. S. Zigangirov, "Distances and Distance Bounds for Convolutional Codes," in Topics in Coding Theory-In Honour of Lars H. Zetterberg, G. Einarsson et al., eds., 109-36, Springer-Verlag, Berlin, 1989. 17. G.D. Forney Jr., "Convolutional Codes II: Maximum Likelihood Decoding," Inform. and Control, 25: 222-66, July 1974.

:rn.

D. J. Costello Jr., "Free Distance Bounds for Convolutional Codes," in IEEE Trans. Inform. Theory, IT-20: 356-65, May 1974.

(Q) ~CTJ lt om [lJJ ITTtu [D) ®~@ ©~ oc11 gi @v ((QJC1Jw©~ [lJJC~o@ou©J~ (©[)~®~ we show that convolutional encoders have a natural trellis structure, two decoding based on this structure. Both of these to a certain criterion. In 1967 Viterbi [1] introduced a for convolutional codes that has since become known as the Viterbi algorithm. Later, Omura [2] showed that the Viterbi was programming solution to the problem of the shortest a Then, Forney [3, 4] recognized that it was in fact a maximum likelihood (ML) decoding algorithm for convolutional codes; that is, the decoder output selected is the codeword that maximizes the conditional of the received sequence. The Cocke, Jelinek, and Raviv (BCJR) algorithm [5] was introduced in 1974 as a maximum a posteriori probability (MAP) decoding method for convolutional codes or block codes with a trellis structure. The optimality condition for the BCJR algorithm is slightly different than for the Viterbi algorithm: in MAP decoding, the probability of information bit error is minimized, whereas in ML decoding the probability of codeword error is minimized (alternative versions of MAP decoding were introduced McAdam, Welch. and Weber [6], Lee [7], and Hartmann and Rudolph [8]); but the of these two in decoding convolutional codes is essentially identical. Because the Viterbi algorithm is simpler to implement, it is in practice; however, for iterative decoding applications, such as the Shannon limit-approaching turbo codes to be discussed in Chapter 16, the BCJR algorithm is preferred. ·we also present a version of the Viterbi algorithm that produces soft outputs, the Viterbi algorithm (SOVA) introduced in 1989 Hagenauer and Hoeher [9], for iterative decoding applications. Union bounds on the performance of convolutional codes with ML decoding are developed in this , and tables of optimal codes based on these bounds are given. The bounds make use of the weight-enumerating functions, or transfer functions, derived in Chapter 11. The application of transfer function bounds to the of convolutional codes was also introduced Viterbi [10). This work laid the foundation for the evaluation of upper bounds on the performance of any convolutionally encoded system. The technique is used again to develop performance bounds in Chapter 16 on turbo coding and in Chapter 18 on trelliscoded modulation. (Tightened versions of the transfer function bound are given in [11, 13].) Finally, the practically important code modification techniques of tail-biting and puncturing are introduced. "''"~'

0

"'"'

516

12.1

Chapter 12

Optimum Decoding of Convolutional Codes

THIE ViTIERIBi ALGORITHM

To understand the 'literbi decoding algorithm, it is convenient to expand the state diagram of the encoder in time, that is, to represent each time unit with a separate state diagram. The resulting structure is a trellis diagram, first introduced for linear block codes in Chapter 9. An example is shown in Figure 12.1 for the (3, 1, 2) nonsystematic feedforward encoder with G(D) = [1

+D

1 + D2

1+D

+ D 2]

(12.1)

and an information sequence of length h = 5. The trellis diagram contains h+m+ 1 = 8 time units or levels, and these are labeled from Oto h + m = 7 in Figure 12.l. For a terminated code, assuming that the encoder always starts in state So and returns to state So, the first m = 2 time units correspond to the encoder's departure from state So, and the last m = 2 time units correspond to the encoder's return to state So. It follows that not all states can be reached in the first m or the last m time units; however, in the center portion of the trellis, all states are possible, and each time unit contains a replica of the state diagram. There are 2" = 2 branches leaving and entering each state. The upper branch leaving each state at time unit i represents the input bit u; = 1, and the lower branch represents u; = 0. Each branch is labeled with the n = 3 corresponding outputs v;, and each of the 2h = 32 codewords of length N = n(h + m) = 21 is represented by a unique path through the trellis. For example, the codeword corresponding to the information sequence u = (1 1 1 0 1)

0

3

2

-

4

5

6

Time units - .

FIGURE 12.1: Trellis diagram for a (3, 1, 2) encoder with h

= 5.

7

l"he Viterbi

Section 12.1

is shown highlighted in Figure 12.1. fo the general case of an (11, k, v) encoder and an information sequence of length K* = kh, there are 2" branches and entering each state, and 2rc distinct paths through the trellis corresponding to the codewords. Now, assume that an information sequence !ill = (lillo, · · · , DJJ 11 _1) of K* = kh is encoded into a codeword v = (vo, v1, · · · . vh+m-1) of N = n(h + 111) and that a Q-ary sequence ir = (iro, ir1. · · · , irh+m-1) is received over a binary-input, Q-ary output discrete memoryless channel (DMC). Alternatively, we can write these sequences as !ill= (uo, iq, · · · , uK''-1), v = (vo, v1, · · · , VJV-1), and ir = (ro, r1, · · · , rN_i), where the subscripts now simply the of the in each sequence. As discussed in Section 1.4, the decoder must an estimate w of the codeword v based on the received sequence ir. A maximum likelihood (ML) decoder for a DMC chooses was the codeword w that maximizes the log-likelihood function log P(irlw). Because for a DMC

n

h+m-1 P(rrlw)

=

n

N-1 P(rr1lw1)

=

1=0

(12.2)

P(nlv1),

1=0

it follows that N-1

h+m-l

logP(rrlw)

=

L

logP(ir1lw1)

=

L

logP(rilv1),

1=0

1=0

where P (ri Iv1) is a channel transition probability. This is a minimum error decoding rule when all codewords are equally likely. The log-likelihood function log(rrlv), denoted by M(rrlv), is called the metric associated with the path ( codeword) w. The terms log P (rr1 lv1) in the sum of (12.3) are called branch metrics and are denoted by M(rr1lv1), whereas the terms logP(rilv1) are called bit metrics and are denoted by M (r1 Iv1). Hence, we can write the metric M (rrjw) as h+m-l

M(rrjv) =

L

N-1

h+m-l

M(ir1lw1) =

1=0

L

1=0

logP(irtlw1) =

L

1=0

N-1

M(rtlv1) =

L

logP(rtlv1).

l=O

We can now express a partial path metric for the first t branches of a path as t-l

t-l

111-l

L

111-l

L

M([!rlw]1) = LM(!r1lw1) = :z=1ogP(rrtlv1) = M(nlv1) = logP(nlv1). 1=0 1=0 1=0 1=0 (12.5) The following algorithm, when applied to the received sequence rr from a

DMC, finds the path through the trellis with the largest metric, that is, the maximum likelihood path (codeword). The algorithm processes ir in a recursive manner. At each time unit it adds 2" branch metrics to each previously stored path metric

518

Chapter 12

Optimum Decoding of Convolutional Codes

(the add operation), it compares the metrics of all 2k paths entering each state (the compare operation), and it selects the path with the largest metric, called the survivor (the select operation). The survivor at each state is then stored along with its metric. The Viterbi Algorithm

Step 1. Beginning at time unit t = m, compute the partial metric for the single path entering each state. Store the path (the survivor) and its metric for each state. Step 2, Increase t by 1. Compute the partial metric for all 2k paths entering a state by adding the branch metric entering that state to the metric of the connecting survivor at the previous time unit. For each state, compare the metrics of all 2k paths entering that state, select the path with the largest metric (the survivor), store it along with its metric, and eliminate all other paths. Stie)lll 3. Ht < h + m, repeat step 2; otherwise, stop. The basic computation performed by the Viterbi algorithm is the add, compare, select (ACS) operation of step 2. As a practical matter, the information sequence corresponding to the surviving path at each state, rather than the surviving path itself, is stored in steps 1 and 2, thus eliminating the need to invert the estimated codeword vto recover the estimated information sequence ii when the algorithm finishes. There are 2v survivors from time unit m through the time unit h, one for each of the 2v states. After time unit h there are fewer survivors, since there are fewer states while the encoder is returning to the all-zero state. Finally, at time unit h + m, there is only one state, the all-zero state, and hence only one survivor, and the algorithm terminates. We now prove that this final survivor is the maximum likelihood path. TIH!JEORJEM 12.1 The final survivor likelihood path; that is,

vin the Viterbi algorithm is the maximum

M(rlv) :0: M(irlv), all

V

-::fa V.

(12.6)

Proof. Assume that the maximum likelihood path is eliminated by the algorithm at time unit t, as illustrated in Figure 12.2. This implies that the partial path metric of the survivor exceeds that of the maximum likelihood path at this point. Now, if the remaining portion of the maximum likelihood path is appended onto the survivor at time unit t, the total metric of this path will exceed the total metric of the maximum likelihood path; but this contradicts the definition of the maximum likelihood path as the path with the largest metric. Hence, the maximum likelihood path cannot be eliminated by the algorithm; that is, it must be the final survivor. Q,E.D. Theorem 12.1 shows that the Viterbi algorithm is an optimum decoding algorithm in the sense that it always finds the maximum likelihood path through the trellis. From an implementation point of view, it is more convenient to use positive integers as metrics rather than the actual bit metrics. The bit metric M (rz Iv1) = IogP(rzlvz) can be replaced by c2[IogP(rzlvz) + ci], where ci is any real number

Section 12.1

The Viterbi Algorithm

519

Time~--> t- 1

Maximum likelihood path

~

"'

FIGURE 12.2: Elimination of the maximum likelihood path.

and c2 is any positive real number. H can be shown (see Problem 12.2) that a 1 log P (r1 IV/) also maximizes path 'V that maximizes M (rrlY) = I:!".c-;;1 M (r1 IV/) = 1 I:i".c0 c2[logP(r1lv1) +c 1), and hence the modified metrics can be used without affecting the performance of the Viterbi algorithm. H c1 is chosen to make the smallest metric 0, c2 can then be chosen so that all metrics can be approximated by integers. There are many sets of integer metrics possible for a given DMC, depending on the choice of c2 (see Problem 12.3). The performance of the Viterbi algorithm is now slightly suboptimum owing to the metric approximation by integers, but q and c2 can always be chosen such that the degradation is very slight. We now give two examples illustrating the operation of the Viterbi algorithm.

L~o

IEXAMPLIE 1;u OiJJil:ffi>iJJii:DMC

Consider the binary-input, quaternary-output (Q = 4) DMC shown in Figure 12.3. Using logarithms to the base 10, we display the bit metrics for this channel in a metric table in Figure 12.4(a). Choosing q = 1 and c2 = 17.3, we obtain the integer metric table shown in Figure 12.4(b ). Now, assume that a codeword from the trellis

520

Chapter 12

Optimum Decoding of Convolutional Codes

0

FIGURE 12.3: A binary-input, quaternary-output DMC.

01

02

Ii

1i

~

01

02

12

1i

0

-0.4

-0.52

-0.7

-1.0

0

10

8

5

0

1

-1.0

-0.7

-0.52

-0.4

1

0

5

8

10

X l

l

(a)

(b)

FIGURE 12.4: Metric tables for the channel of Figure 12.3.

diagram of Figure 12.1 is transmitted over the DMC of Figure 12.3 and that the quaternary received sequence is given by (12.7)

The application of the Viterbi algorithm to this received sequence is shown in Figure 12.5. The numbers above each state represent the metric of the survivor for that state, and the paths eliminated at each state are shown crossed out on the trellis diagram. The final survivor,

v= (111, 010, 110, 011, 000, 000, 000),

(12.8)

is shown as the highlighted path This surviving path corresponds to the clecoded information sequence 1lll = (1100 0). Note that the final m = 2 branches in any trellis

Section 12.1

The Viterbi Algorithm

521

0

-

°'

er,

'0,\_\.

~

0 0 0

9,

0 0 0

1:

0 0 0

c5

"T

~

0 -,

'0,\_\. \.'0,\_

\_\_\J '>'

-

c;·

s

-

0 0

0

ofo

-

9,

\_\\J

c;

0

r---

~r

-

....,

~

0 0

O/o

0

'° 0 0 0

[[[

'°,,,

re,

N

"'

0 0 0

o;o'

lr)

~

0

~~

[[[

0

c:5' ~

0

-9,

0 0 0

~

'"

FIGURE 12.5: The Viterbi algorithm for a DMC.

522

Chapter 12

Optimum Decoding of Convolutional Codes

path correspond to O inputs and hence are not considered part of the information sequence. In the special case of a binary symmetric channel (BSC) with trans1t10n probability p < 1/2, 1 the received sequencer is binary ( Q = 2) and the log-likelihood function becomes (see (1.11)) log P(rlv)

= d(r, v) log _P_ + N log(l 1-p

p),

(12.9)

where d(r, v) is the Hamming distance between rand v. Because log l~p < 0 and N log(l - p) is a constant for all v, an MLD for a BSC chooses v as the codeword v that minimizes the Hamming distance h+m-1

d(r, v)

=

L

1=0

N-1

d(r1, v1)

=

L d(ri, v1).

(12.10)

1=0

Hence, when we apply the Viterbi algorithm to the BSC, d (r1. v1) becomes the branch metric, d (rz, vz) becomes the bit metric, and the algorithm must find the path through the trellis with the smallest metric, that is, the path closest to r in Hamming distance. The operation of the algorithm is exactly the same, except that the Hamming distance replaces the log-likelihood function as the metric, and the survivor at each state is the path with the smallest metric. EXAMPLE 12.2

The Viterbi Algorithm for a !BS(

An example of the application of the Viterbi algorithm to a BSC is shown in Figure 12.6. Assume that a codeword from the trellis diagram of Figure 12.1 is transmitted over a BSC and that the received sequence is given by

(110,110,110,111,010,101,101).

(12.11)

v= (111,010,110,011,111,101,011),

(12.12)

r=

The final survivor,

is shown as the highlighted path in the figure, and the decoded information sequence is iii = (11001). That the final survivor has a metric of 7 means that no other path through the trellis differs from r in fewer than seven positions. Note that at some states neither path is crossed out. This indicates a tie in the metric values of the two paths entering that state. If the final survivor goes through any of these states, then there is more than one maximum likelihood path, that is, more than one path whose distance from r is minimum. From an implementation point of view, whenever a tie in metric values occurs, one path is arbitrarily selected as the survivor, owing to the impracticality of storing a variable number of paths. This arbitrary resolution of ties has no effect on the decoding error probability. 1 A BSC vvith p > 1/2 can nhvnys be converted to a BSC with p labels.

<

1/2 simply by reversing the output

Section 12.1

The Viterbi

0 0 0

80

8 S.

0

O

0

0 0

,--<

g

C

,--<

0

,--<

0

FIGURE 12.6:

The Viterbi algorithm for a BSC.

Now, consider a binary-input, additive white Gaussian noise (AWGN) channel with no demodulator output quantization, that is, a binary-input, channel. Assume the channel inputs O and 1 are represented by the BPSK signals (see (1.1))

±[fi;cos(2nfot), where we use the mapping 1 ---+ +'1fs and O -> -'1£,.

524

Chapter 12

Optimum Decoding of Convolutional Codes

Normalizing by ./Es, we consider the codeword v = (vo, v1, · · · , VN-1) to take on values ±1 according to the mapping 1 ---+ + 1 and O ---+ -1, and the (normalized) received sequence ir = (ro, r1, · · · , rN-1) to be real-valued (unquantized). The conditional probability density function (pdf) of the (normalized) received symbol ri given the transmitted bit vi is (12.13) where No/ Es is the (normalized) one-sided noise power spectral density. If the channel is memoryless, the log-likelihood function of the received sequence ir given the transmitted codeword v is

n

N-l

M(irlv) = lnp(ll'IV) = ln

N-l

p(r1lv1) =

= - -E.,

No

N-l

~

L, (r1 - v1) 2

+ -N 2

1=0

Es In - nNo

N-l

Es~ No

L lnp(r1lv1) l=O

1=0

N 2

2

(12.14)

Es nNo

= - - L , ( r1 -2riv1+l)+-ln-l=O

N-l

~

Es

No l=O

No

E, = (2-')

2

N

Es

2

nN0

L, (r1vi) - -(lirl + N) + - ln -

= C1 (ir-v) +

C2,

where C1 = (2Es/No) and C2 = [(E5 /No)(lirl 2 + N) - (N/2)ln(E 5 /nNo)] are constants independent of the codeword v and ll'·V represents the inner product (correlation) of the received vector I!' and the codeword v. Because C1 is positive, the trellis path (codeword) that maximizes the correlation r-v also maximizes the log-likelihood function ln p(rlv). It follows that the path metric corresponding to the codeword vis given by M(rlv) = ll'·V, the branch metrics are M(r1lv1) = r1·V1, l = 0, 1, · · · , h + m - 1, the bit metrics are M(nlv1) = riv1, l = 0, 1, · · · , N - 1, and the Viterbi algorithm finds the path through the trellis that maximizes the correlation with the received sequence. (It is important to remember in this case that the (realvalued) received sequence r is correlated with trellis paths (codewords) v labeled according to the mapping 1 ---+ + 1 and O ---+ -1.) An interesting analogy with the binary symmetric channel is obtained by representing the (real-valued) received sequence and the codewords as vectors in N -dimensional Euclidean space. (In this case, the N -bit codewords with components ±1 all lie on the vertices of an N-dimensional hypersphere.) Then, for a continuousoutput A WGN channel, maximizing the log-likelihood function is equivalent to finding the codeword v that is closest to the received sequence r in Euclidean distance (see, e.g., [12] or [14]). In the BSC case, on the other hand, maximizing the log-likelihood function is equivalent to finding the (binary) codeword v that is closest to the (binary) received sequencer in Hamming distance (see (12.9)). An

Section 12.2

Performance Bounds for Convolutional Codes

525

example of the application of the Viterbi algorithm to a continuous-output A WGN channel is given in Problem 12.7. In Section 1.3 we noted that making soft demodulator decisions (Q > 2) results in a performance advantage over making hard decisions ( Q = 2). The preceding two examples of the application of the Viterbi algorithm serve to illustrate this point. H the 01 and 02 are converted into a 0, and 11 and h are converted into a single 1, the soft-decision DMC is converted to a hard-decision BSC with transition probability p = 0.3. In the preceding the sequence rr in the hard-decision case is the same as in the soft-decision case with 01 and 02 converted to Oand 11 and 12 converted to 1; but the Viterbi algorithm yields different results in the two cases. In the soft-decision case ( Q = 4), the information sequence 1ll1 = (1 1 0 0 0) produces the maximum likelihood path, which has a final metric of 139. In the hard-decision case (Q = 2), however, the maximum likelihood path is 1ll! = (1 1 0 0 1). The metric of this on the quaternary output channel is 135, and so it is not the maximum likelihood path in the soft-decision case; however, since hard decisions mask the distinction between certain soft-decision outputs-for example, outputs 01 and 02 are treated as equivalent outputs in the hard-decision case- the hard-decision decoder makes an estimate that would not be made if more information about the channel were available, that is, if soft decisions were made. (For another example of the difference between soft-decision and hard-decision decoding, see Problems 12.4-12.6.) As a final comment, both of the preceding channels can be classified as "very noisy" channels. The code rate R = 1/2 exceeds the channel capacity C in both cases. Hence, we would not expect the performance of this code to be very good with either channel, as reflected the relatively low value (139) of the final metric of the maximum likelihood path in the DMC case, as compared with a maximum possible metric of 210 for a path that "agrees" completely with rr. Also, in the BSC case, the final Hamming distance of 7 for the maximum likelihood path is large for paths only 21 bits long. Lower code rates would be needed to achieve good performance over these channels. The performance of convolutional codes with Viterbi decoding is the subject of the next section. 12.2

flERfORMANCIE ~OIUINIDi§ fO~ CONVOWT!ONAIL COIDllES

We begin this section by analyzing the performance of maximum likelihood (Viterbi) decoding for a specific code on a BSC. We will discuss more general channel models later. First, assume, without loss of generality, that the all-zero codeword w = l{ll is transmitted from the (3, 1, 2) encoder of (12.1). The IOWEF of this encoder (see Problem 11.19) is given by x 7 wL 3 A(W, X, L) = 1- XWL(l + X 2 L)

= x 7 wL 3 [1 + XWL(l + X 2 L) + x 2 w 2 L 2(l + X 2L 2) + .. ·] = x7WL3 + x8w2L4 + x9w3L5 + X10(W2L5 + VV4L6) + ... ;

(12.15)

that is, the code contains one codeword of weight 7 and length 3 branches generated by an information sequence of weight 1. one codeword of weight 8 and length 4 branches generated by an information sequence of weight 2, and so on.

526

Chapter 12

Optimum Decoding of Convolutional Codes

~

\

S0

S0

t-3

,----------,

t-2

S0

, - - - - , - " - -,*.-_--;

t-l

~

Time -

Correct path v

FIGURE 12.7: A first event error at time unit t.

We say that a first event error is made at an arbitrary time unit t if the all-zero path (the correct path) is eliminated for the first time at time unit t in favor of a competitor (the incorrect path). This situation is illustrated in Figure 12.7, where the correct path vis eliminated by the incorrect path v' at time unit t. The incorrect path must be some path that previously diverged from the all-zero state and is now remerging for the first time at time t; that is, it must be one of the paths enumerated by the codeword WEF A(X) of the code. Assuming it is the weight-7 path, a first event error will be made if, in the seven positions in which the correct and incorrect paths differ, the binary received sequencer agrees with the incorrect path in four or more of these positions, that is, if r contains four or more 1'sin these seven positions. If the BSC transition probability is p, this probability is P7

=

P[four or more 1'sin seven positions]

(12.16)

Section 12.2

Performance Bounds for Convolutional Codes

521

Assuming that the weight-8 path is the incorrect path, a first event error is made with probability Pg=~ (

!)

p4(1- p)4

+

t (! )

pe(l - p)8-e,

(12.17)

e=S

since if the metrics of the correct and incorrect paths are tied, an error is made with 1/2. In general, the incorrect path has weight d, a first event error is made with probability

t (:

)pe(l-pi-e.

dodd

e=d!!

I !2 ( d/2d )

pd/2(1 -

ri/2 +

-

~ ( ~

d ) pe(l - p)d-e e '

d even.

e=~+l

(12.18) Because all incorrect of length t branches or less can cause a first event error at time unit t, the first event error probability at time unit t, Pf (E, t), can be overbounded, using a union bound, by the sum of the error probabilities of each of these paths. Hall incorrect paths of length greater than t branches are also included, ( E, t) is overbounded by 00

(£, t) <

L

Ac1Pc1,

(12.19)

d=d1;ee

where Art is the number of codewords of weight d (i.e., it is the coefficient of the weight-d term in the codeword WEF A(X) of the code). Because this bound is of t, it holds for all time units, and the first event error probability at P1(E), is bounded by 00

(£) <

L

Ac1Pc1.

(12.20)

d=d1;ee

The bound of (12.20) can be further simplified by noting that ford odd. Pc1

=

L d

(

de ) pe(l - p)d-e

a-fi l O

c-

(12.21)

528

Chapter 12

Optimum Decoding of Convolutional Codes

It can also be shown (see Problem 12.9) that (12.21) is an upper bound on

Pd

ford

even. Hence, d

00

P1(E) <

L

Ad

[2Jp(l - p)] ,

(12.22)

d=drree

and for any convolutional code with codeword WEF A(X) follows by comparing (12.22) with the expression for A(X) that (12.23) The final decoded path can diverge from and remerge with the correct path any number of times; that is, it can contain any number of event errors, as illustrated in Figure 12.8. After one or more event errors have occurred, the two paths compared at the all-zero state will both be incorrect paths, one of which contains at least one previous event error. This situation is illustrated in Figure 12.9, where it is assumed Decoded path

/

I

Correct pa th

FIGURE 12.8: Multiple error events.

FIGURE 12.9: Comparison of two incorrect paths.

Section 12.2

Performance Bounds for Convolutional Codes

52!91

that the correct path w has been eliminated for the first time at time unit t - l incorrect path w', and that at time unit t incorrect paths w' and w" are H the partial metric for w" exceeds the partial metric for w' at time unit t, then it must also exceed the partial metric for w, since w' has a better metric than 'ii at time t to the first event error at time t - l. Hence, if w" were with wat time unit t, a first event error would be made. We say that an event e;.-ror occurs at time unit t if w" survives over w', and the event-error probability at time unit t, is bounded by (E, t), P(E, t) :S: (12.24) since if V11 survives over 'o/1 at time unit I, then it would also survive if -v"'"'~" with w. fo other the event that w" has a better metric than w' at time t probability P(E, t)) is contained in the event that w" has a better metric than v1 at time t (with probability P1(E, t)). The situation illustrated in Figure 12.9 is not the only way in which an event error at time unit t can follow a first event error made earlier. Two other are shown in Figure 12.10. fo these cases, the event error made at time unit t either totally or partially replaces the event error made previously. Using the same arguments, it follows that (12.24) holds for these cases also, and hence it is a valid bound for any event error occurring at time unit t. 2 The bound of (12.19) can be applied to (12.24). Because it is oft, it holds for all time units, and the event-error probability at any time unit, P(E), is bounded by 00

P(E) <

L

00

Ac1Pc1 <

L

just as in (12.23). For sman p, the bound is dominated by its first term, that is, the free distance term, and the event-error probability can be approximated as P(E)

IE}(AMr?'llE U,3

~

d/in· I I /? [2J p(l - p) J ~ Ac1/iec2G /ice J//iee - .

(12.26)

IEvailLllzrtta1r1gJ ii:i1ite IEvteO'ilii:-IEu"u"~W r?'m10Ja10Ja!aii:y

For the (3, 1, 2) encoder of (12.1), obtain

dtree =

7 and

Ac1/iee = 1.

P(E) ~ 27 p 712 = 1.28 x 10- 5 _

Thus for p = 10-2 • we (12.27)

The event-error probability bound of (12.25) can be modified to a bound on the bit-error probability, Pb(E), that the number of information bit errors per decoded information bit. Each event error causes a number of information bit errors equal to the number of nonzero information bits 2In the two cases shown in Figure 12.10, the event error at time unit t replaces at least a portion of a previous error event. The net effect may be a decrease in the total number of decoding errors; that is. the number of positions in which the decoded path differs from the correct path. Hence, using the first event error probability as a bound may be conservative in some cases.

530

Chapter 12

Optimum Decoding of Convolutional Codes

Time--

Time--

FIGURE 12.10: Other error event configurations.

on the incorrect path. Hence, if each event error probability term Pd is weighted by the number of nonzero information bits on the weight-d path, or if there is more than one weight-d path, by the total number of nonzero information bits on all weight-d paths, a bound on the expected number of information bit errors made at any time unit results. This bound can then be divided by k, the number of information bits

Performance Bounds for Convolutional Codes

Section 12.2

per unit time, to obtain a bound on Pb(E). In other is bounded by

:5311

the bit-error probability

00

(E) <

L

(12.28)

BdPd,

d=djiee

where Bc1 is the total number of nonzero information bits on all weight-d paths, divided the number of information bits k per unit time it is the coefficient of the weight-d term in the bit WEF B(X) = of the encoder). Now, (12.21) in (12.28) and the for we see that 00

Pb(E) <

L

00

Bc1Pr1 <

L

(12.29)

for any convolutional encoder with bit WEF B(X). (We note here that the bound of (12.25) on the event-error probability P(E) depends only on the whereas the bound of (12.29) on the bit-error probability (£) depends on the encoder.) For small p. the bound of (12.29) is dominated by its first term, so that Pb (£). ~ Bd/rcc . [2,/p(l - p)

IEXAMu:»LIE 12.4

dfiee

J

I

I

~ B dfree . 2GJiecpGficc

/2

.

(12.30)

IEvaiviaitirruigi 'it~iie r8u1t~1Erru\Qlu rmtiait0Jm1ty

For the (3, 1. 2) encoder of (12.1). Br11;e,· obtain

= 1. and dfree = 7. Thus, for

p

= 10- 2 , we (12.31)

when pis small, the most the same as for the event-error probability. In other likely error event is that the weight-7 is decoded instead of the all-zero path, thereby causing one information bit error. Typically, then, each event error causes one bit error, which is reflected in the approximate expressions for P(E) and Pb(E). Slightly tighter versions of the bounds on P(E) and Pb(E) given in (12.25) and (12.29), respectively, have been derived in [11] and [13]. If the BSC is derived from an A WGN channel with BPSK modulation, optimum coherent detection, and binary-output quantization (hard decisions), then from (1.4) we have p=Q

{li;) · (VNo

(12.32)

Using the bound of (1.5) as an approximation yields (12.33) and for a convolutional code with free distance

dtree,

(12.34)

532

Chapter 12

Optimum Decoding of Convolutional Codes

when pis small, that is, when the channel SNR Es/ No is large. Defining the energy per information bit, E1,, as

(12.35) and noting that } is the number of transmitted symbols per information bit, we can write (12.36) for large bit SNR E1,/ No. On the other hand, if no coding is used, that is, R = 1 and Es transition probability p is the bit-error probability P1, (E) and Pb(E)

~ Q(

flf)

"~e-E;/No

=

(without coding).

E1,, the BSC

(12.37)

Comparing (12.36) and (12.37), we see that for a fixed £1,/ No ratio, the (negative) exponent with coding is larger by a factor of Rdtree/2 than the exponent without coding. Because the exponential term dominates the error probability expressions for large £1,/ No, the factor Rdtree/2, in decibels, is called the asymptotic coding gain y: y =L1

dB 10log10 (Rdfree) 2

(12.38)

in the hard-decision case. It is worth noting that coding gains become smaller as E1,/ No becomes smaller. In fact, if E1,/ No is reduced to the point at which the code rate R is greater than the channel capacity C, reliable communication with coding is no longer possible, and an uncoded system will outperform a coded system. This point is illustrated in Problem 12.12. Bounds similar to (12.25) and (12.29) can also be obtained for more general channel models than the BSC. For a binary-input A WGN channel with finite ( Q-ary) output quantization (a DMC), the bounds become P(E) < A(X)ix=Do

(12.39a)

and (12.39b) where Do ~ L(O.'.:).'.:Q~l) )PCJIO)P(Jll) is a function of the channel transition probabilities called the Bhattacharyya parameter. Note that when Q = 2, a BSC results, and Do = 2) p(l - p). Complete derivations of slightly tighter versions of these bounds can be found in [12]. In the case of a binary-input A WGN channel with no output quantization, that is, a continuous-output A WGN channel, we again assume that the all-zero codeword vis transmitted. In other words, v = (-1, -1, · · ·, -1) according to the mapping O _,.. -1 and 1 _,.. +1. Now, we calculate the probability Pd that the correct path vis eliminated for the first time at time unit t by an incorrect path w' that differs from v in d positions. Because the Viterbi algorithm always selects the path with

Section 12.2

Performance Bounds for Convolutional Codes

53:l

the largest value of the log-likelihood function log p(ll"lv) (see (12.14)), a first event error at time unit t is made with probability Pc1 = Pr{M([ll"lv'] 1) > M([ll"lv] 1)} = Pr{[ll"·v']1 > [ll"·v]i}

111-1

=

Pr

(12.40)

l

111-l

L riv; - L riv1 > 0 j ,

1

1=0

J

1=0

where [ll"·v] 1 represents the inner of ll" and v over their first t branches. (Note that since we are now dealing with real numbers, the event M([ll"lv'] 1) = M([ll"lw] 1) has probability the difference in the sums in (12.40) is nonzero only in the d where v; f. v1. Without loss of generality, let l = 1, 2, · · · , d represent Then, recalling that in the d positions where v; f. v1, v; = +1 and v1 = we can express (12.40) as Pc1

=

Pr

(t(+n) -

t(-r1) >

l=l

=

Pr

o) = Pr (2

1=1

tr1 > l=l

!En> l

o) (12.41)

0 .

cl

l=l

(12.41) can be interpreted to mean that a first event error at time unit t occurs if the sum of the received symbol values in the d positions where v; f. v1 is fo other words, if the received sequence ll" in these d positions looks more like the incorrect path w' than the correct path v, a first event error is made. Because the channel is memoryless, and the transmitted codeword is assumed to be v = (-1, -1, · · ·, -1), p ~ Y:}= 1 r1 is a sum of d independent Gaussian random variables, each with mean -1 and variance No/2Es (see (12.13)); that p is a Gaussian random variable with mean -d and variance dNo/2Es (see, e.g., [11] or [14]). Thus, we can write (12.41) as Pr1

=

Pr{p > O}

and with the substitution y

=

(p

=

(~) 1 - -5 -

ndNo

+ d)J2Es/dNo,

00

o

-£,ip+d)"

e

dNo

dp,

(12.42)

(12.42) becomes

(12.43)

~ Q (/¥,) ~ Q (/u~) ,

534

Chapter 12

Optimum Decoding of Convolutional Codes

where Q(x) is the familiar complementary error function of Gaussian statistics. Now, substituting (12.43) into the bounds of (12.25) and (12.29), we obtain the following expressions for the event- and bit-error probabilities of a binary-input, continuous-output A WGN channel: (12.44a)

P1,(E) < d~" BdQ

(j2d:oEb).

(12.44b)

Using the (slightly weaker than (1.5)) bound Q(x) < exp(-x 2/2) in (12.44), we obtain the expressions 00

OO

Pb(E) <

dRE1,

Ac1e -------«o = A(X)lx=exp(-RE1,/No)

P(E) <

L

(12.45a)

dRE1,

Bc1e -------«o

= B(X)lx=exp(-RE1,/No)·

(12.45b)

d=dticc

Comparing (12.45) with (12.39), we see that in the case of a binary-input A WGN channel with no output quantization, that is, a continuous-output A WGN channel, the Bhattacharyya parameter is given by Do= exp(-REb/No). It is instructive to compare the approximate expression for P1,(E) given in (12.36) for a BSC with a similar expression obtained for a binary-input, continuousoutput A WGN channel from (12.45b ). For large Eb/ No, the first term in the bit WEF dominates the bound of (12.45b), and we can approximate Pb(E) as (12.46) Comparing the exponent of (12.46) with that of (12.36), we see that the exponent of (12.46) is larger by a factor of 2. This difference is equivalent to a 3-dB energy (or power) advantage for the continuous-output A WGN channel over the BSC, since to achieve the same error probability on the BSC, the transmitter must generate an additional 3 dB of signal energy (or power). This energy advantage illustrates the benefits of allowing soft decisions, that is, an unquantized demodulator output, instead of making hard decisions. The asymptotic coding gain in the soft-decision case is given by (12.47) y ,6. 10 loglO (Rdfree) dB, an increase of 3 dB over the hard-decision case. The decoder complexity increases, however, owing to the need to accept real-valued inputs. The foregoing analysis is based on performance bounds for specific codes and is valid only for large values of E1,/ No. A similar comparison of soft decisions with finite-output quantization ( Q > 2) and hard decisions can be made by computing the approximate expression for (12.39b) for a particular binary-input DMC and comparing with (12.30) for ;ci BSC Generally, it is found that Q = 8 allows one to achieve a performance within about 0.25 dB of the optimum performance achievable

Section 12.2

Performance Bounds for Convolutional Codes

535

with an unquantized demodulator output while avoiding the need for a decoder that accepts real-valued A random coding analysis has also been used to demonstrate the of soft decisions over hard decisions [12, 14]. For small values of Eb/ No, this shows that there is about a 2-dB penalty in signal power attached to the use of hard decisions; that is, to achieve the same error 2 dB more signal power must be generated at the transmitter when the demodulator output is hard rather than unquantized. Over the entire range of Eb/ No the decibel loss associated with hard decisions is between 2 dB and 3 dB. the use of soft decisions is preferred in many applications as a means of regaining the 2-3-dB loss to hard quantization, at a cost of some increase in decoding Somewhat tighter versions of the bounds in the inequality Q(,jy+z):::: Q(,jy)e-2 (y > 0, z 2:. 0). Setting y = 2djreeREb/ No and z = 2(d - djree)REb/ No, we can use (12.43) as Pc1

=Q

(V~) ~

to write

= Q ( Jy + z)

2dfree REb e) No

id-d/ieei/?£1,

v

IQ

Now, defining the function f(x)

we can write

p d

:S

=

(12.50)

Q(Jh)eX,

f (dfreeRE1,) - ":£1, No

e

o .

Finally, substituting (12.51) into the bounds of (12.25) and (12.29), we obtain the following expressions for the event-- and bit-error probabilities of a continuous-output A WGN channel: 00

P(E) <

L

A,tf

(d

RE ) dRE1, fre~O b e -No

d=d/iee

=

f (

(12.52a)

dfreeRE1,) No A(X)lx=exp(-RE1,/No)

(12.52b) =

f (

dfreeREb)

No

B(X)lx=exp(-RE1,/No)·

536

Chapter 12

Optimum Decoding of Convolutional Codes

We note that the tightened bounds of (12.52) differ from the bounds of (12.45) by the scaling factor f (dfreeR Eb/ No), which depends only on the free distance of the code. We now give an example illustrating the application of the bit-error probability bounds in (12.45b) and (12.52b ). IEXAMPLIE 12.5

Bit-lEnrorr Probability 1801..m1ds for an AWGN Channel

Consider the (3, 1, 2) encoder of (12.1) whose IOWEF A(W, X, L) is given in (12.15). The bit WEF of this encoder is given by B(X)

= Cl/ k)

aA(W, X) I

aw

w=1

7

= a[x w;c1- xw - x 3 w)] I

aw

W=l

(12.53)

x1 (1 - 2X + X 2

-

2X 3 + 2X4 + X6)

= x 7 + 2x 8 + 3X 9 + 6X 10 + .... The free distance of this code is dtree = 7, and we can use (12.53) directly to evaluate the bounds of (12.45b) and (12.52b ). In Figure 12.11 we plot these two bounds as functions of the bit SNR Eb/ No. Also plotted for comparison is the performance of uncoded BPSK and the result of a computer simulation showing the Viterbi decoding performance of this code on a continuous-output A WGN channel. Note that the tightened bound of (12.52b) agrees almost exactly with the simulation for SNRs higher than about 4 dB, whereas (12.45b) tracks the simulation closely but is not as tight. For lower SNRs, however, both bounds diverge from the simulation result. This is typical behavior for union bounds; namely, at SNRs near capacity (about -0.5 dB for rate R = 1/3 and BPSK modulation), they do not give good estimates of performance. The (soft-decision) asymptotic coding gain of this code is given by y

= lOlog10(Rdfree) = 10log10 (7 /3) = 3.68 dB.

(12.54)

This is the coding gain, compared with uncoded BPSK, that is achieved in the limit of high SNR. We see from Figure 12.11, however, that the real coding gain at a biterror probability of Pb(E) = 10-4 is only about 3.3 dB, illustrating that real coding gains are always somewhat less than asymptotic coding gains. In general, larger real coding gains are achieved by codes with fewer nearest-neighbor codewords, that is, smaller values of Ac11;c,· and Bc11;"". The performance bounds derived in this section are valid for unterminated convolutional encoders; that is, they are independent of the length N of the codeword. The event-error probability P(E) is the probability that at any given time unit the Viterbi algorithm will select a path that introduces errors into the

Section 12.2

Performance Bounds for Convolutional Codes

537

Bound (12.45b) Bound (12.52b) Uncoded BPSK

le-1

Viterbi decoding

le-2

le-3

Gi"' le-4

~

~

le-5

le-6 _

\

le-7

'

le-8'--~~~~-'-~~~~--"-~~~~~'--~~~~----'-~~~~--' 2 4 6 10 0 8

FIGURE 12.11:

Performance bounds for convolutional codes.

decoded sequence, whereas the bit-error probability Pi,(E) is the average number of bit errors per unit time. Thus the bit-error probability P1,(E) represents the bit-error rate (BER) of an unterminated encoder. The word-error probability, or word-error rate (WER), of an unterminated encoder, on the other hand, is essentially unity, since if a long enough sequence is encoded, at least one error event must occur. To determine bounds on the WER and BER of terminated encoders, we must modify the codeword and bit WEFs to include delayed versions of codewords and codewords that diverge and remerge with the all-zero state more than once, as noted previously in Section 11.2. In other words, terminated convolutional codes are block codes, and the WEFs must account for all possible codewords. Techniques for finding these modified WEFs and evaluating the performance of terminated convolutional codes are presented in Chapter 16 on turbo coding.

538

12.3

Chapter 12

Optimum Decoding of Convolutional Codes

CONSTRUCTION Of GOOD CONVOlUTIONAl CODES

We can now address the problem of constructing good codes for use with maximum likelihood (Viterbi) decoding. Once a desired code rate has been selected, the performance bounds presented in the previous section can be used as guidance in the construction of good codes. For example, the bounds of (12.25), (12.29), (12.39), and (12.45) all indicate that the most significant term for both P(E) and Pb(E), that is, the free distance term, decays exponentially with increasing dfree and increases linearly with increasing Ac1/iee and Bc1/iee· This relationship suggests that the most important criterion should be maximizing the free distance dj,ee· Then, as secondary criteria, Ac1fiee' the number of (nearest-neighbor) codewords with weight djree, and Bc1/iee' the total information sequence weight of all weight-dfree codewords, divided by k, should be minimized. (Because of the close relationship between Ac11;ee and Bc1fiee (see (11.129)), it is sufficient to simply minimize Ac1/iee·) Generally speaking, djree is of primary importance in determining performance at high SNRs, but as the SNR decreases, the influence of the number of nearest-neighbors Ac11;ee increases, and for very low SNRs the entire weight spectrum plays a role. Finally, the use of catastrophic encoders should be avoided under all circumstances. Most code constructions for convolutional codes have been done by computer search. Algebraic structures that guarantee good distance properties, similar to the BCH construction for block codes, have proved difficult to find for convolutional codes. This has prevented the construction of good long codes, since most computer search techniques are time-consuming and limited to relatively short constraint lengths. An efficient search procedure for finding djree and Ac1r,ee based on the Viterbi algorithm has been developed by Bahl, Cullum, Frazer, and Jelinek [15] and modified by Larsen [16]. The algorithm assumes the received sequence is all zeros, confines the search through the trellis to only those paths starting with a nonzero information block, uses a metric of O for an agreement and +1 for a disagreement, and searches for the path with the minimum metric. As soon as the metric of the survivor at the all-zero state is less than or equal to the metric of the other survivors, the algorithm can be terminated. The metric at the all-zero state then equals dfree, since none of the other survivors can ever remerge to the all-zero state with a smaller metric. (A straightforward modification of this procedure can also be used to compute Ac11m .. ) A trellis depth of several constraint lengths is typically required to find dfree for a noncatastrophic encoder. For a catastrophic encoder, the survivor at the all-zero state may never achieve the minimum metric, owing to the zero loop in the state diagram. (This can be used as an indicator of a catastrophic encoder.) This algorithm is capable of computing djree and Ac11;""for values of v up to about 20. For larger values of v, the number of storage locations required by the algorithm, 2", becomes unacceptably large and other means of finding djree must be tried. No general solution to the problem of finding dtree for large values of v has yet been discovered. We shall see in Section 13.4, however, that some sequential decoding algorithms for convolutional codes can, with proper modification, be used to compute the free distance of codes for values of v up to about 30. Lists of optimum codes for rates R = 1/4, 1/3, 1/2, 2/3, and 3/4 are given in Table 12.1, where the optimality criterion first maximizes dfree and then minimizes Adrree. In the table, vie list the overall constraint length v, the free distance djree, the number of nearest-neighbor codewords Ac1/iee' and the soft-decision asymptotic

Section 12.3

Construction of Good Convolutional Codes

53l~

coding gain y, given (12.47), of each code. fo Tables 12.l(a), 12.l(b ), and 12.l(c), for the (low-rate) codes with k = 1, we list the coefficients of the polynomials in the generator matrix CG(D) = [g(Ol(D) g< 1l(D) ... g(n-ll(D)] (see (1L80a)), that the generator sequences, in octal form, for the minimal controller canonical form encoder realization. In Tables and 12.1(e ), for the (high-rate) codes with k > 1, we list the coefficients of the polynomials in the parity-check matrix IHI(D) = [1mC11 -ll(D) ... ]m(ll(D) lm(O\D)] (see (1L82b) ), that is, the parity-check sequences, in octal for the minimal observer canonical form encoder realization. To be consistent in the octal of the generator and parity-check sequences for convolutional encoders listed throughout this text, we adopt the convention. We first write a binary polynomial if(D) of degree v from highest order to lowest order as follows: lf(D)

=

fuDv

+ fv-1Du-l + ... + fiD + fo.

TABLE 12.1(a) 3 : Optimum rate R

=

(12.55)

1/4 convolutional

codes. g(I[))

I!;(]_)

g<2)

1 5 13 25 45 117 257 533 1173

1 5 13 27 53 127 311 575 1325

3 7 15 33 67 155 337 647 1467

II

1 2 3 4 5 6 7 8 9

g<3)

3 7 17 37 77

171 355 711 1751

Aafree

y ( cd!Iffi)

1 1 2 4 3 2 1 1 3

L76 3.98 5.12 6.02 6.53 6.99 7AO 7.78

6 10 13 16 18 20 22 24 27

TABLE 12.1(b): Optimum rate R

=

829

1/3 convolu-

tional codes.

3 Tables

II

g
g(Jl)

g<2)

1 2 3 4 5 6 7 8 9 10 11 12

1 5 13 25 47 117 225 575 1167 2325 5745 2371

3 7 15 33 53 127 331 623 1375 2731 6471 13725

3 7 17 37 75 155 367 727 1545 3747 7553 14733

12Ja-e adapted from [17].

dfree

Aapw

y (cd!Iffi)

5 8 10 12 13 15 16 18 20 22 24 24

1 2 3 5 1 3 1 1 3 7 13 5

222 426 522 6.02 6.36 6.99 727 7.78 823 8.65 9.03 9.03

540

Optimum Decoding of Convolutional Codes

Chapter 12

= 1/2 con-

TABLE 12.1(c): Optimum rate R

volutional codes. g
3 5 13 27 53 117 247 561 1131 2473 4325 10627 27251

1 7 17 31 75 155 371 753 1537 3217 6747 16765 37363

djree

Adfree

y (dB)

3 5 6 7 8 10 10 12 12 14 15 16 16

1 1 1 2 1 11 1 11 1 14 14 14 1

1.76 3.98 4.77 5.44 6.02 6.99 6.99 7.78 7.78 8.45 8.75 9.03 9.03

TABLE 12.1(d): Optimum rate R = 2/3 convolu-

tional codes. V

h(2)

h(l)

Jh.(O)

2 3 4 5 6 7 8 9 10

3 17 23 71 123 313 555 1051 2621

5 15 31 57 147 227 631 1423 2137

7 13 27 73 121 241 477 1327 3013

djree

Adfree

y (dB)

3 4 5 6 7 8 8 9 10

1 1 3 7 17 43 6 17 69

3.01 4.26 5.23 6.02 6.69 7.27 7.27 7.78 8.24

I TABLE 12.1(e): Optimum rate R = 3/4 convolutional

codes. V

h(3)

h_(2)

11<1)

)h_(O)

2 3 4 5 6 7 8 9

2 11 33 47 107 211 535 1475

5 13 25 73 135 341 757 1723

7 15 37 57 133 315 733 1157

6 12 31 75 141 267 661 1371

djree

Adfi-cc

y (dB)

3 4 4 5 6 6 7 8

6 10 2 7 27 5 27 136

3.52 4.77 4.77 5.74 6.53 6.53 7.20 7.78

I I

Section 12.3

with the octal

= [1 + the best

D

2

o-r Good Convolutional Codes

!j;gj\1

bit fo. we group the coefficients in threes and with O's on the left to make the total

+

l+D+

+

1+

+

+

sequences are listed as 11;< 0 l 2, 3) encoder in Table

IHI(D) = [!ht(2)

cm +

J

Construction

-:and

nPl CD) n,< 0 )

+1

+D+

and its sequences are listed as Ilu(2 l = (We note that the controller canonical form in reverse order to the octal rp·,nr;,c~·n canonical form in (12.57) are the search are been obtained decision of some of the codes are the (2, 1, 18) code with dfree = 22 achieves an with soft-decision however, since the number of nearest uv«~-.,~.,, code is a rather large A22 = the real will exceed 10 clB low BERs SNRs). The codes listed in Table 12.1 eithe:c a nonsystematic feedforward encoder or an 0-~-,u~ .. ~ feedback encoder. This is because for a more free distance is available with nonsystematic feedforward encoders than with feedforward encoders. as was pointed out in 11; however, that no inverting circuit is needed to recover a version of the information sequence from the codeword ,vithout decoding. This allows the user to look" at the received information sequence vvithout to invert a take a "nonsystematic codeword." This feature can be ing is done off-line, or where the decoder is or where the channel is known to be "noiseless" becomes unnecessary. In these cases, the of encoder realization is preferred. In some applications a feedforward encoder realization may be example, for feedforward m blocks of all-zero input bits can used to terminate the encoder rather than the termination bits u~~'~"~' information sequence. In this case it is desirable to to combine the property of systematic feedforward encoders with the free distance available with nonsystematic feedforward encoders. To this and Costello [18] developed a class of rate R = 1/2 nonsystematic feedforward encoders called

542

Chapter 12

Optimum Decoding of Convolutional Codes

quick-look-in encoders that have a property similar to the quick-look capability of systematic encoders. They are defined by

(12.58) and g~O) = g~l) = g1l;!l = g1~;) = 1; that is, the two generator sequences differ in only a single position. These encoders are always noncatastrophic (see Problem 12.15), and their feedforward inverse has the trivial transfer function matrix (12.59) Because (12.60) the information sequence 1lll(D) can be recovered from the codeword V(D) with a one time unit delay. The recovery equation is (12.61) and we see that if p is the probability that a bit in the codeword V(D) is in error, then the probability of a bit error in recovering mi(D) is roughly 2p, 4 because an error in recovering u1 can be caused by an error in either 1 or 1. For any noncatastrophic rate R = 1/2 encoder with feedforward inverse

v;~ v;~

(12.62) the recovery equation is (12.63) for some l, and an error in recovering u1 can be caused by an error in any of w[g01(D)] positions in vC 0l(D) or any of w[g 11(D)] positions in v<1l(D). Hence, the

probability of a bit error in recovering mi(D) is A~ w[g01 (D)] + w[g11 (D)] times the probability of a bit error in the codeword. A is called the error probability amplification factor of the encoder. A = 2 for quick-look-in encoders, and this is the minimum value of A for any R = 1/2 nonsystematic encoder. For R = 1/2 systematic (feedforward or feedback) encoders, (12.64) and A = 1 for systematic encoders. Hence, quick-look-in encoders are "almost systematic," in the sense that they have the minimum value of A for any nonsystematic encoder. Catastrophic encoders, at the other extreme, have no feedforward inverse, and their error probability amplification factor is infinite. 4 We are ignoring here the unlikely event that two errors in V(DJ will cancel, causmg no error in recovering 1111(DJ.

Section 12.3

Construction of Good Convolutional Codes

543

rate R = quick-look-in convolutional codes.

TABLE 12.2:

J)

2 3 4 5 6 7 8 9 10 11

12 13 14 15 16 17 18 19 20 21 22 23 24

i,,;({P) 5 15 31 55 151 215 455 1335 3055 6055 14135 34731 60545 171045 341225 613151 1422255 3007451 6153605 14565371 32720445 63347465 147373045

5 6 7 8 9 9 10 11 12 13 14 14 15 16 16 17 18 18 19 20 20 21 22

Adfree

1' (tl!IIB}

1 1 2 2 4 1 1 3 3 8 10 3 6 11 2 5 6 2 4 7 1 3 7

3.98 4.77 5.44 6.02 6.53 6.53 6.99 7.40 7.78 8.13 8.45 8.45 8.75 9.03 9.03 9.29 9.54 9.54 9.78 10.00 10.00 10.21 10.41

The capability of quick-look-in encoders to an immediate estimate (prior to decoding) of the information sequence from a version of the codeword with an error probability amplification factor of only 2 makes them desirable in some applications. The free distances and number of nearest neighbors for the optimum rate R = 1/2 codes generated by quick-look-in encoders are listed in Table 12.2. Note that dtree for the best quick-look-in R = 1/2 codes is a little less than dtree for the best overall R = 1/2 codes listed in Table 12.l(c). Thus, the "almost systematic" property of quick-look-in codes results in a small performance penalty compared with the best codes (see Problem 12.16). Their free distances are superior, however, to what can be achieved with the codes generated by systematic feedforward encoders (see Problem 12.17). As noted previously, there are few algebraic constructions available for convolutional codes. One exception is the construction of orthogonal codes for use with majority-logic decoding. These constructions are covered in Chapter 13. Another approach, initiated by Massey, Costello, and Justesen [19], uses the minimum distance properties of a cyclic block code to provide a lower bound on the free distance of an associated convolutional code. H g(X) is the generator polynomial of any (n, k) cyclic code of odd length n with minimum distance dg, and lln(X) = (X 11 - 1)/i,,;(X) is

544

Chapter 12

Optimum Decoding of Convolutional Codes

the generator polynomial of the (n, n - k) dual code with minimum distance dh, the following construction for rate R = 1/2 codes results.

Construction 12.1 The rate R = 1/2 convolutional encoder with composite generator polynomial g(D) is noncatastrophic and has djree ::,: min(dg, 2d1i). The cyclic codes should be selected so that dg ~ 2d1,. This suggests trying cyclic codes with rates in the range 1/3 :::: R :::: 1/2 (see Problem 12.18). EXAMIP'llE 12.6

Co!flstn.u::ting Commh..lltional Codes from Block Codes

Consider the (15, 5) BCH code with generator polynomial g(X) = 1 + X + X 2 + x 4 + x 5 + X 8 + X 10 . This code has minimum distance dg = 7. The generator polynomial of the dual code is h(X) = (X 15 - 1)/g(X) = x 5 + X 3 + X + 1, and d1, = 4. The rate R = 1/2 convolutional encoder with composite generator polynomial g(D) = 1 + D + D 2 + D 4 + D 5 + D 8 + D 10 then has djree > min(7, 8) = 7. The polynomial generator matrix is given by G(D)

= [1 + D + D 2 + D 4 + D 5 1 + D 2 ],

(12.65)

and we see that the encoder is noncatastrophic and has constraint length v = 5. If un(D) = 1, the codeword (12.66) has weight 7, and hence dfree

= 7 for this code.

A similar construction can be used to produce codes with rate R

= 1/4.

Construction 12.2 The rate R = 1/4 convolutional code with composite generator polynomial g(D 2 ) + Dh(D 2 ) is noncatastrophic and has dfree ::,: min(dg + d11, 3dg, 3d1,).

The cyclic codes should be selected so that dg ~ d1,. This suggests trying cyclic codes with rates near R = 1/2 (see Problem 12.18). Two difficulties prevent these constructions from yielding good long convolutional codes. The first is the problem of finding long cyclic codes with large minimum distances. The second is the dependence of the bound on the minimum distance of the duals of cyclic codes. The second difficulty was circumvented in a subsequent paper by Justesen [20]. Justesen's construction yields the bound dfree ::,: dg, but it involves a rather complicated condition on the roots of g(X) and, in the binary case, can be used only to construct convolutional codes with odd values of n. Another paper by Tanner [21] broadens the class of block codes to include quasi-cyclic codes. Tanner's construction yields the bound djree ::,: dmin, where dmin is the minimum distance of an associated quasi-cyclic code, and it provides an interesting link between the theories of block and convolutional codes.

12.4 iMPLEMIENTATiON AND IP'IERFORMANCIE OF THE ViTIERBI ALGORITHM The basic operation of the Viterbi algorithm was presented in Section 12.1. In a practical implementation of the algorithm, several additional factors must be

Section 1

Implementation and Performance o'f the Viterbi Algorithm

545

considered. In this section we discuss some of these factors and how they affect decoder Decr[})deu Jlf!e,,fflr[})ll'J, Because there are 2" states in the state of the encoder, the decoder must reserve 2" words of for the survivors. Each vvord must be of the along with its metric. Since the increase with the constraint \!, in 1t 1s not feasible to use codes ·with large v (although special-purpose Viterbi decoders with constraint as \! = 14 have been This limits the available free gains of around 7 dB are the limit of the Viterbi algorithm in most cases. The exact error achieved on the its rate, its free the available channel SNR and the demodulator as wen as other factors.

!P(l)!it!h We noted in Section 11.1 that convolutional codes are most efficient when the of the information sequence is large. The difficulty this causes is that each of the 2" words of storage must be capable of storing a K* = khbit its metric. For very large h, this is clearly impossible, and some must be made. The that is usually taken is to truncate the path memory of the decoder the most recent r blocks of information bits for each survivor, where r < < h. Hence, after the first r blocks of the received sequence have been the decoder, the decoder memory is full. After the next block is a decoding decision must be made on the first block of k information bits, since it can no longer be stored in the decoder's memory. There are several strategies for making this decision. Among these are the

lio Choose an

survivor, and select the first information block on this

2, Select from among the possible first information blocks the one that appears most often in the 2') survivors. :t Choose the survivor with the best and select the first information block on this After the first decision is made, additional decoding decisions are made in the same way for each new received block processed. Hence, the decoding decisions lag the progress of the decoder by an amount equal to the path memory, that is, r blocks. At the end of a terminated trellis, there remain r ~ m information blocks to decode. These are simply selected as the last r - m information blocks on the final °'"""rnr,~ The decoding decisions made in this way are no longer maximum likelihood, but can be almost as good as maximum likelihood if T is not too small. Experience and analysis have shown that if r is on the order of 5 times the encoder memory order or more, with probability approaching 1 all 2" survivors stem from the same information block r time units back; thus, there is no ambiguity in making the decoding decision. This situation is illustrated in Figure 12.12. In addition, this must be the maximum likelihood decision, since no matter which survivor eventually becomes the maximum likelihood path, it must contain this decoded information block. Hence, if r is large enough, almost all decoding decisions will be maximum

546

Chapter 12

Optimum Decoding of Convolutional Codes Survivor

Common stem

j+

j

T

+1

- - Time - -

FIGURE 12.12: Decoding decisions with a finite path memory.

likelihood, and the final decoded path will be close to the maximum likelihood path. This point is illustrated in Problem 12.19. There are two ways in which errors can occur in a truncated decoder. Assume that a branch decision at time unit t is made by selecting the survivor at time unit t + r + 1 with the best metric and then decoding the information bits on that path at time unit t. H a decoder with unlimited path memory contains a decoding error (i.e., the maximum likelihood path diverges from the correct path) at time unit t, it is reasonable to assume that the maximum likelihood path is the best survivor at time unit t + r + 1, and hence a decoder with finite path memory will make the same error. An additional source of error with a truncated decoder occurs when some incorrect path that is unmerged with the correct path from time unit t through time unit t + r + 1 is the best survivor at time unit t + r + 1. In this case a decoding error may be made at time unit t, even though this incorrect path may be eliminated when it later remerges with the correct path and thus will not cause an error in a decoder with unlimited path memory. Decoding errors of this type are called decoding errors due to truncation. The subset of incorrect paths that can cause a decoding error due to truncation is shown in Figure 12.13. Note that it includes all unmerged paths of length greater than r that diverge from the correct path at time unit j or earlier. For a convolutional encoder with codeword WEF A(W, X, L), the event-error probability on a BSC of a truncated decoder is bounded by P(E) < [A(W, X,

L) +

t

2

1

1-l

1=;:-;-1

Af (W, X,

L)]

,

(12.67)

X=LJp(l-p), W=l,L=l

where Af (W, X, L) is the codeword WEF for the subset of incorrect paths that can cause decoding errors due to truncation [23]. In other words, Af (W, X, L) is

Section 12.4

Implementation and Performance

the Viterbi

Incorrec1 path ubset

1

\ I \ // /\7 I ) XJ\ 0

0

0

C---

0

0

0

{_

\

/\ 0Y I

/\

\ , I ,V}/\ \

Il L I

0

0

0

I 0

2

0

f~

\

l1

0

0

0

!;)(~ // \~\

j\f\

/\!(' I

\

\ \ .)\.,1(1 __\J.!. ,\ f \ \(l

I

I

I

I\

\

.

I

~-""="=P o o o

0

0

0

I II d

~u/

\\,, ~OCJO

I,Lo-r--, "",

c~--'.)-----0

j

j+l

I j+2

j+T-l

j+T

i+T+l

Time->

Correct path

FIGURE 12.13:

Incorrect path subset for a truncated decoder.

more than r branches the codeword WEF for the set of aH unmerged in the augmented modified encoder state that connect the all-zero state with the ith state. The first term in (12.67) the errors made a decoder with unlimited path memory. whereas the second term n~,rn~·"p· decoding errors due to truncation. (12.67) can be DMCs and the AWGN channel X = D0 and X = e-RE;,/No. respectively. similar to (12.67) can be obtained for the bit-error probability starting with the bit '\JVEF of the encoder. When pis small (if the BSC is derived from a AV\/GN ~H<'""'""' this means large E1,/ No), (12.67) can be P(E) ~ Ad/iee ( 2Jp(l - p) )

d/i"ee

+ Ad(r)

(

2Jp(l - p)

)d(r)

.

where d(r) is the smallest power of D, and Ar1(r) is the number of terms with 1 d(r) in the unmerged codeword WEF Af (W, X. Further

I):1

(12.68) yields P(E)

~

A d1,ee . 2dfieepdfiee/2

+ A~d(r) 2d(,) l'Jcl(r)/2 ·

(12.69)

From (12.69) it is dear that for small values of p, if d(r) > dfree, the second term is negligible compared with the first term, and the additional error due to truncation can be ignored. Hence, the path memory r should be chosen large enough so that d(r) > djree in a truncated decoder. The minimum value of r for

5~8

Chapter 12

Optimum Decoding of Convolutional Codes

Minimum

TABLE 12.3:

truncation lengths for rate R = 1/2 optimum free

distance codes.

m

djree

Tmin

2 3 4 5 6 7

5 6 7 8 10 10

7 9 14 18 26 27

d(Tmin)

6 7 8 9 11 11

which d(r) > dfree is called the minimum truncation length Ymin of the encoder. The minimum truncation lengths for some of the rate R = 1/2 optimum free distance codes listed in Table 12.l(c) are given in Table 12.3. Note that Ymin ~ 4m in most cases. A random coding analysis by Forney [4] shows that in general Ymin depends on R, but that typically a small multiple of m is sufficient to ensure that the additional error probability due to truncation is negligible. Extensive simulations and actual experience have verified that 4 to 5 times the memory order of the encoder is usually an acceptable truncation length to use in practice. IEXAMIP'llE U.7

MinimlLllm Tmrlrnto«:m length «:l>f a (2, 1, 2) IEm:oidlerr

Consider the (2, 1, 2) nonsystematic feedforward encoder with G(D) = [1 + D + D 2 1 + D 2 ]. The augmented modified state diagram for this encoder is shown in Figure 12.14, and the codeword WEF is given by A(W, X, L)

=

x 5 wL 3 1- XWL(l

+ L).

(12.70)

Letting Ai (W, X, L) be the WEF for all paths connecting the all-zero state (So) with the ith state (Si) we find that X 2 WL(l - XWL) Ai(W, X, L) = 1- XWL(l + L)'

(12.71a)

WXL

FIGURE 12.14: Augmented modified state diagram for a (2, 1, 2) encoder.

Section 12.4

and Performance

or the Viterbi

Time-~>-

FIGURE 12. 15: Determining the truncation distance of a (2, 1, 2) encoder.

w2L2 (W, X, L) = 1 - XW L(l

+ L).

(12.71c)

H we now expurgate each of these functions to include only of length more than r d(r) is the smallest power of X in any of the three expurgated functions. For example, d(O) = 2. d(l) = 3. d(2) = 3, and so on. Because dtree = 5 for this code, Tmin is the minimum value of r for which d(r) = dji-ee + 1 = 6. A1(W, L) contains a term X 5 W 4 L 7 , and hence d(6):::: 5 and Tmin must be at least 7. The particular path yielding this term is shown darkened on the trellis diagram of Figure 12.15. A careful inspection of the trellis diagram shows that there is no path of length 8 branches that terminates on S1. S2, or S3 and has less than 6. There are five such paths, however, that have 6, and these are shown dotted in Figure 12.15. Hence, d(7) = 6, Ad(7) = 5, and Ymin = 7, which is 3.5 times the memory order of the encoder in this case. Hence, a decoder with a path memory of 7 should be sufficient to ensure a negligible increase in event-error probability over a decoder with unlimited path memory for this code. Finally. it is important to point out the distinction between the column distance d1 defined in Section 11.3 and the truncation distance d(r) defined here. d1 is the

550

Chapter 12

Optimum Decoding of Convolutional Codes

minimum weight of any codeword of length l + 1 branches. Because this includes codewords that have remerged with the all-zero state and hence whose weight has stopped increasing beyond a certain point, d1 reaches a maximum value of dfree as l increases. On the other hand, d (r) is the minimum weight of any codeword of length more than r branches that has not yet remerged with the all-zero state. Because remergers are not allowed, d(r) will continue to increase without bound as r increases. For example, for the encoder state diagram of Figure 12.14, d(9) = 7, d(l9) = 12, d(29) = 17,. ··,and, in general, d(r) = + 2 for odd r. Catastrophic encoders are the only exception to this rule. In a catastrophic encoder, the zero-weight loop in the state diagram prevents d(r) from increasing as r increases. Hence, catastrophic encoders contain very long codewords with low weight, which makes them susceptible to high error probabilities when used with Viterbi decoding, whether truncated or not. For example, the catastrophic encoder of Figure 11.14 has dfree = 4 but contains an unmerged codeword of infinite length with weight 3, and d ( r) = 3 for all r > 1. Hence, the additional error probability due to truncation will dominate the error probability expression of (12.69), no matter what truncation length is chosen, and the code will not perform as well as a code generated by a noncatastrophic encoder with dfree = 4. This performance difference between catastrophic and noncatastrophic encoders is discussed further in [24].

ril

Decoder SyU'HchmuniwtiolfD, In practice, decoding does not always commence with the first branch transmitted after the encoder is set to the all-zero state but may begin with the encoder in an unknown state, in midstream, so to speak. In this case, all state metrics are initially set to zero, and decoding starts in the middle of the trellis. If path memory truncation is used, the initial decisions taken from the survivor with the best metric are unreliable, causing some decoding errors. But Forney [4] has shown, using random coding arguments, that after about Sm branches are decoded, the effect of the initial lack of branch synchronization becomes negligible. Hence, in practice the decoding decisions over the first Sm branches are usually discarded, and all later decisions are then treated as reliable. Bit (or symbol) synchronization is also required by the decoder; that is, the decoder must know which of n consecutive received symbols is the first one on a branch. In attempting to synchronize symbols, the decoder makes an initial assumption. If this assumption is incorrect, the survivor metrics typically remain relatively closely bunched. This is usually indicative of a stretch of noisy received data, since if the received sequence is noise-free, the metric of the correct path typically dominates the metrics of the other survivors. This point is illustrated in Problem 12.21. If this condition persists over a long enough span, it is indicative of incorrect symbol synchronization, since long stretches of noise are very unlikely. In this case, the symbol synchronization assumption is changed until correct synchronization is achieved. Note that at most n attempts are needed to acquire correct symbol synchronization. Receiver Qiumtization. For a binary-input A WGN channel with finiteoutput quantization, we gave performance bounds in (12.39) that depended on the Bhattacharyya parameter Q-1

Do= ) , j=O

J P(JIO)P(Jll),

(12.72)

and Performance of the Viterbi

Section 1

where P(JIO) and P(Jll) are channel transition probabilities, and Q is the number of symbols. fo particular, smaller values of Do, which depends DMC result in lower error probabilities. Because we are free to design the that to select the we should do this in such a way that Do is minimized. fo general, for there are Q - 1 thresholds to and Do should be minimized with to these Q - 1 with respect to zero, Because the transmitted symbols ±"'(]!:; are the distributions p(r 10) and 11) associated with an received value r are with to ±~, and since Q is a power of 2, that even, one threshold should be selected at To = 0, and the other thresholds should be selected at values ±T1. ±T2, · · · , ±TcQ/2)-1· the reduces to Do with . In 12.16 we illustrate the calculation of the P(±q; 10) and P(±q; 11) based on a set of channel transition v0,csv,u0, where the notation ±q, denotes the whose interval is bounded by ±T1 and ±T;-1, i = 1. 2, · · · . Q/2. and TQ/2 oo. Using this notation and the 1-+ +~ and O ->-~,we can rewrite as

=

Q/2

Do=

L [J P(q; IO)P(q; 11)-+ J P(-q; IO)P(-q; 11)],

(12.73)

i=l

P(-q,ll) 7

~~~.-r-~Z7~~~-_-_·~~~~-~.f-~~L-~----'CLLL/~~-+-~~~-==~ -T,+1 -T, -T,-1

FIGURE 12.16: .,._,""""'""H')e,

output DMC.

0

T,-1

T,

T,+1

channel transition probabilities for a binary-input, Q-ary

552

Chapter 12

Optimum Decoding of Convolutional Codes

and we can calculate the transition probability P(qi 10) using (12.13) as follows: P(qi!O)=

T;

1

p(r!O)dr=

1T;

T;_ 1

~s

~ E., (r+l)2 dr.

(12.74)

- e No n No

T;_ 1

We now minimize (12.72) with respect to Ti by solving the following equation for Ti:

(12.75) The solution of (12.75) involves terms of the form aP(q1 Iv)

a[JL

oT;

1

p(rlv)dr]

v

oT;

= 0 or 1.

(12.76)

Using the relations

a[J[:~

1

p(rlv)dr] oT; = p(Ti Iv)

(12.77a)

and (12.77b) we can write oP(qilv)

-~·- = aTi

I

p(T;lv), -p(Tilv), O,

i

= '.

J = . -1J r

1 · l,

+ 1 · -1- · J r l

(12.78)

+ 1.

(From Figure 12.16 we can see that only the transition probabilities P(qi 10), P(qi 11). P(qi+i 10), and P(q;+1 ll) depend on the selection of the threshold T; .) Now, using (12.78) in (12.75) we obtain

1 2

P(q;+1ll) p(T;IO) P(q;+1IO)

(12.79)

=0.

We can also write (12.79) as P(q;IO) (T.ll)+ P(q;ll) p '

P(q; 11) (T. IO) = 1 P(q; 10) p

P(q;+1IO) p(T;ll) + P(q;+1ll)

P(q;+1 ll) P(q;+1IO)

pCTi !O) (12.80)

and Performance of the Viterbi

Section 12.4

or 10) [

P(q, \1) P(q,10)

P(q;+1\l)J

I P(q;+1\0)

1

-

I )

[

l

P(q;+1IO) P 10) P(q1+1ll) - \ P(q, 11) -

or

J

\1)

[J P(q; 11)/ P(q, 10) -

10)

[JP(q1+1\0)/P(q1+1\l) --/P(q,\0)/P(q,\l)]

j

-/l-'(q1+1 \l)/ P(q1+1 \0)

P(q;+1 il) P(q, 11)

-J P(q, IO)P(q1+1 \0) P(q,\l) \ P(q, 10)

P(q1+t\O)

we define the likelihood ratio of a received value r at the unquantized binary-input channel as J\.(r)

p(rll)

= --. \0)

Similarly, we define the likelihood ratio of a received binary-input, Q-ary output DMC as J\.(q;)

of an

=

q 1 at the

P(q, 11) P(q; \0)

Using (12.83), we can now express the condition of minimizes Do as

of a

(12.83b) on the threshold T; that

>._(T;) = /A(q;)A(q1+1). i = 1. 2. · · ·. Q/2.

(12.84)

a necessary condition for a set of thresholds T;, i = Do and hence also to minimize the bounds of (12.39) on error the likelihood ratio of each quantizer threshold value T; must equal the mean of the likelihood ratios of the two quantization symbols q; and q;+i that border T;. Because (12.84) does not have a closed-form the set of threshold values must be determined using a trial-and-error approach. In Problem thresholds are calculated for several values of Q the condition of (12.84). His demonstrated that the required SNR E.1/ No needed to achieve a given quantization value of Do is only slightly larger when Q = 8 than when no is used. In other an 8-level involves very little loss compared with a continuous-output channel. Condition (12.84)

1. 2. · · .. Q /2 to minimize the

Cmnp//)jft1[1Jifi![J)r1ud C![J)mplexirty. The Viterbi must 2v ACS operations per unit time, one for each state, and each ACS additions, one for each branch entering a state, and i' - 1 comparisons. Hence, the computational complexity (decoding of the Viterbi algorithm is

554

Chapter 12

Optimum Decoding of Convolutional Codes

proportional to the branch complexity l1'2v = 2k+v of the decoding trellis. Thus, as noted previou.sly for decoder me1nory, the exponential dependence of decoding time on the constraint length v limits the practical application of the Viterbi algorithm to relatively small values of v. In addition, since the branch complexity increases exponentially with k, codes with high rates take more time to decode. This computational disadvantage of high-rate codes can be eliminated using a technique called puncturing that is discussed in Section 12.7. High-speed decoding can be achieved with the Viterbi algorithm by employing parallel processing. Because the 2v ACS operations that must be performed at each time unit are identical, 2v identical processors can be used to do the operations in parallel rather than having a single processor do all 2v operations serially. Thus, a parallel implementation of the Viterbi algorithm has a factor of 2" advantage in speed compared with a serial decoder, but it requires 2" times as much hardware. Decoding speed can be further improved by a factor of about 1/3 for a large subclass of nonsystematic feedforward encoders by using a compare-select-add (CSA) operation instead of the usual ACS operation. Details of this differential Viterbi algorithm can be found in [25].

Code PeufoU"ma//lJce. Computer simulation results illustrating the performance of the Viterbi algorithm are presented in Figure 12.17. The bit-error probability P1;(E) of the optimum rate R = 1/2 codes with constraint lengths v = 2 through v = 7 listed in Table 12.l(c) is plotted as a function of the bit SNR E1;/No (in decibels) for a continuous-output AWGN channel in Figure 12.17(a). These simulations are repeated for a BSC, that is, a hard-quantized ( Q = 2) channel output, in Figure 12.l 7(b ). In both cases the path memory was r = 32. Note that there is about a 2-dB improvement in the performance of soft decisions ( unquantized channel outputs) compared with hard decisions (Q = 2). This improvement is illustrated again in Figure 12.17(c), where the performance of the optimum constraint length v = 4, rate R = 1/2 code with Q = 2, 4, 8, and oo (unquantized outputs) and path memory r = 32 is shown. Also shown in Figure 12.17(c) is the uncoded curve of (12.37). Comparing this curve with the coding curves shows real coding gains of about 2.4 dB in the hard-decision case (Q = 2), 4.4 dB in the quantized (Q = 8) soft-decision case, and 4.6 dB in the unquantized (Q = oo) soft-decision case at a bit-error probability of 10-5 . Also, we note that there is only about 0.2-dB difference between the Q = 8 quantized channel performance and the unquantized (Q = oo) channel performance, suggesting that there is not much to gain by using more than 8 channel-output quantization levels. In Figure 12.17(d), the performance of this same code is shown for path memories r = 8, 16, 32, and oo (no truncation) for a BSC (Q = 2) and an unquantized (Q = oo) channel output. Note that in both cases a path memory of r = 8 = 2v degrades the performance by about 1.25 dB, that r = 16 = 4v is almost as good as r = 32 = 8v, and that r = 32 = 8v performs the same as no truncation. (Recall that v = m for rate R = 1/2 codes.) The performance of the optimum rate R = 1/3 codes with constraint lengths v = 3, 5, and 7 listed in Table 12.l(b) is shown in Figure 12.17(e) for both a continuous-output A WGN channel and a BSC. Note that these codes do better than the corresponding rate R = 1/2 codes of Figures 12.17 (a) and (b) by between 0.25 dB and 0.5 dB. This is because the coding gain, that is, the product of R and dtree in decibels, is larger at

Implementation and Performance o-f the Viterbi Algorithrn

Section 12.4

555

,--------1 v=2 --e\

~ : !5 =

····-0····

I

v=6

-c:,--

i 1

V

~--=~--

I

1v=7-e-_

"'

~

le-04

le-05

3

2.5

3.5

4

4.5 E1,IN,i (dB)

5.5

5

6

(a)

I,----1---v=2 1; = 3 / 11=4

1

-

1

--<-

I

- -><- - I --~:~-- /-

1 11 =

5

: v=6 I 71=7

..

0--

1:

I -

-G--: --e-

I

le-03

le-04 _

4.5

5

5.5

6

6.5

7

7.5

E1,IN0 (dB) (b)

FIGURE 12.17: Simulation results for the Viterbi algorithm.

8.5

556

Chapter 12

Optimum Decoding of Convolutional Codes 1 Uncoded --+-- : . :Q= 2 - - ; < - - 1· IQ= 4 __ ,. __ iJ I I .1 I Q = 8 ··fr··· I I Q = oo _____ _

le-02

le-03

' '1--, le-04

' .., '\

'\

\

\

\

.., \

le-05

\\ \ X

2

3

5

4

6

8

7

9

10

Et,IN11 (dB)

( c)

le-02 .

~

"'-

X.

II,~,

"' " ' "'-m"'-

·.,

'

·.\_~~,

"

JI!, '

' ',' '

I},'' "

le-03

~-------------------1 BSC path memory = 8 --+-: BSC path memory = 16 - -><- BSC path memory = 32 - - .,_ 1 1 BSC path memory = oo ... o 1 AWGNpathmemory = 8 --<>-1 AWGN path memory= 16 - e- : AWGN path memory= 32 - - <>- AWGN path memory= oo · .;, .. 1 ---------------------

. '.\,~©,\ ·,ia, \

'\\ \ \ '\

a, ·,

'\

' \

le-04

.,.,.,'\\

'\,

\.

'\

'\

\

\\

"'M, \ \

''

\.

'\\'El\

'

\

\

\

'1'

\

\

\\

\\

\<Sl,

·,

\

Iii. \

''

\

.\ \

le-05

'

' ',' '

\

\

Tu,, \

\

\\ \\

\

\

\

'

\

\

'

')( \

~,.,',\ \

\'x [fl

\ 'b 2

3

4

''

5

6

E1,IN11 (dB)

( cl)

FIGURE 12.17:

\

\\

(continued)

7

8

9

Section 12.Li.

and Performance of the Viterbi Algorithm

551

:- i=i°s-c ~- ~- 3----~~.:: -: : BSC v = 5 - ~><- ~ : : BSC V = 7 ° S:· : AWGN V = 3 ····8···· :AVVGNv=S ~ , . :AWGNv=7 ~©>-

le-03

0

0

le-05 _

6

2

8

7

E1,IN11 (dB)

(e)

t

·,

~---- ------F=----

BSC R = 2/3 11 = 6 --<- : :AWGNR=2/3v=6 --><-- I 1

"-----------------1

\

\

\

le-03

\ \ \ \ \ \ \

"'

\ \

\ \ \ \ \ \

le-04 _

~

\ \ \ \ \ \

\ \

\ \ j(.

\ \

le-05

\

\ \ \

\ 3

3.5

4

5

4.5

5.5

£ 1/No (dB) (f) FIGURE 12.17:

(continued)

6

6.5

7

558

R

Chapter 12

= 1/3 than at

Optimum Decoding of Convolutional Codes

R

= 1/2 for the same constraint length. 5 Finally, Figure 12.17(f)

shows the performance of the optimum rate R = code ,vith v = 6 listed in Table 12.1(d) for both a continuous-output A WGN channel and a BSC. Again. note that the corresponding rate R = 1/2 code with v = 6 in Figures 12.17(a) and (b) does better than the rate R = 2/3 code by about 0.4 dB and 0.25 dB, respectively. All these observations are consistent with the performance analysis presented earlier in this chapter. 12.5

THE SOFT-OUTPUT V!TIEIRBI ALGORITHM (SOVA)

The subject of code concatenation. in which two or more encoders (and decoders) are connected in series or in parallel, will be discussed in Chapters 15 and 16. In a concatenated decoding system, it is common for one decoder to pass reliability (confidence) information about its decoded outputs, so-called soft outputs, to a second decoder. This allows the second decoder to use soft-decision decoding, as opposed to simply processing the hard decisions made by the first decoder. Decoders that accept soft-input values from the channel (or from another decoder) and deliver soft-output values to another decoder are referred to as soft-in, soft-out (SISO) decoders. The Soft-Output Viterbi Algorithm (SOVA) was first introduced in 1989 in a paper by Hagenauer and Hoeher [9]. We describe the SOVA here for the case of rate R = 1/n convolutional codes used on a binary-input, continuousoutput A WGN channel; that is, we describe a SISO version of the Viterbi algorithm. In our presentation of the SOVA we deviate from the usual assumption that the a priori probabilities of the information bits, P (u1 ), l = 0, 1, · · · . h 1, are equally likely by allowing the possibility of non-equally likely a priori probabilities. This generality is necessary to apply a SISO decoding algorithm to an iterative decoding procedure, such as those to be discussed in Chapter 16. The basic operation of the SOVA is identical to the Viterbi algorithm. The only difference is that a reliability indicator is attached to the hard-decision output for each information bit. The combination of the hard-decision output and the reliability indicator is called a soft output. At time unit l = t, the partial path metric that must be maximized by the Viterbi algorithm for a binary-input, continuous-output A WGN channel given the partial received sequence [ir] 1 = (iro, ir1, · · · , 1r1 _1) = (0) (]) _(n-1). (0) (1) (11-l). . (0) (1) _(11-l)) · (r0 ,r0 , · · · , 1 0 ,r1 .r1 ,···.r1 ,···,r1_ 1,r1_ 1. · · · , 1 1_ 1 can bewntten as (12.85) Equation (12.85) differs from (12.14) only in the inclusion of the a priori path probability P([v] 1 ), since these will, in general, not be equally likely when the a priori probabilities of the information bits are not equally likely. Noting that the a priori path probability P([v] 1) is simply the a priori probability of the associated information sequence [llll] 1, we can separate the contribution to the metric before 5 Recal1 fro1n Chapter 1, ho'vvcvcr, that a rate R = 1/3 code requires n1ore channel ba.ndvv·idth expansion than a rate R = 1/2 code.

Section 12.5

.I[t-1Ll t-1

'\/lterbi

The

l

time t from the contribution at time t as follows:

M([r\v"Jr)

=

ln

p(r1\v1)P(u1)

fi

=ln~n 1l 1=0

ln

r 1-1

= 1n

1

Ill1

I

Ji.

l=O

We now the time t term in (12.86) and introducing constants c/il and as follows:

each tern1 in the sum

1[. j)

where the constants

c?l = ln [p(r/ 1l\v;il = +n]+1n [pcr?l1v;il = -1>]. i ell= ln [P(u1 = +1)] +

ln [P(u1

=

o. L .... n -

l

= -1)]

are independent of the [v]1 , and we assume the 1 -+ + 1 and O-, -1. Similarly modifying each of the terms in before time t and that the modifications do not affect the [v]1 that maximizes (12.86). we can express the modified n1etric as (see Problem

11-l

([ir\v] 1 )

=

M*([n'l'v]r_i)

+L

{21n [p(r/il\v;il)]--

ciil} +

-ell}

ln [P

.i=O

-

* -, -

- M ([u\Y]1_i)

We can simplify the likelihood ratio, or channel with

11-l

+~ L-d v (j) ln ./~

I

1

[- -c·(j)\ ]-'

It

(1 l

p(rI

(j) -

Vt

( )

\v I

1

-

_J

r

1)] .t

=-1) _

+

,

1,1

-

ln

+1)j . l---=-=c•

-

-

_L

--1)

for the modified metric in (12.89) of a received r at the v =±las

(Note that L(r) = ln[)c(r)]. defined in information bit u is defined as L(u)=ln

p(u [ p(u

the L-value of an

+l)J

= = -1)

.

(12.91)

56«:l

Chapter 12

Optimum Decoding of Convolutional Codes

An L-value can be interpreted as a measure of reliability for a binary random variable. For example, assuming that the a priori (before transmission) probabilities of the code bits v are equally likely, that is, P(v = +1) = P(v = -1) = 1/2, an assumption that always holds when the information bits are equally likely and the code is linear, we can rewrite (12.90) using Bayes' rule as follows: L(r)

= 1n [ p(r!v = +1)] = ln [p(v = +l!r)J p(r!v = -1) p(v = -l!r)

(12.92)

.

From (12.92) we see that given the received symbol r, a large positive value of L(r) indicates a high reliability that v = + 1, a large negative value of L (r) indicates a high reliability that v = -1, and a value of L(r) near O indicates that a decision about the value of v based only on r is very unreliable. Similarly, a large positive value of L (u) indicates a high reliability that u = + 1. For an A WGN channel with received symbol r', binary input symbols v' = ±,JI!;, and SNR Es/ No, we obtain (see Problem 12.24) L(r')

=

(12.93a)

(4)£;/No)r'.

We can rewrite (12.93a) as L(r)

=

(12.93b)

(4Es/No)r,

where the normalized value r = r' / ,JI!; corresponds to the received symbol with binary-input symbols v = ±1. Equation (12.93b) illustrates that the reliability (Lvalue) associated with a received symbol r, assuming a transmitted symbol v = ±1, increases linearly with r and that the constant of proportionality increases linearly with the channel SNR Es/ No, an intuitively appealing result. Defining Le = 4E., / No as the channel reliability factor, we can now express the modified metric for SOVA decoding as 11-l

M*([rr!v})

= M*([rr!v] 1-1) +

L Lcv?\Ul + u L(u 1

1 ),

(12.94)

J=O

where the summation term in (12.94), along with the corresponding terms in M*([rr!v] 1_1), represents the correlation metric first noted in (12.14). In other words, the constants C1 and C2 in the metric of (12.14) are absorbed into the term Le in the modified metric of (12.94), and the term u 1 L(u 1 ) in (12.94) does not appear in (12.14), since the a priori probabilities of the information bits were assumed to be equally likely in that case. As noted previously, the SOVA operates exactly like the Viterbi algorithm, except that it appends a reliability measure to each hard-decision output. Assume that a comparison is being made at state S;, i = 0, 1, · ·. , 2" - 1, between the maximum likelihood (ML) path [w] 1 and an incorrect path [w'] 1 at time l = t. We define the metric difference as 1 (12.95) .6.1-1(S;) = {M*([r!v] 1 ) - M*([rr!w'] 1)}.

2

The probability P(C) that the ML path is correctly selected at time tis given by (12.96)

Viterbi Algorithm

The

Section 12.5

We note that P([vlrr]1)

p([rrlw]1)P([v]1)

=

p(rr)

eM([rrlv],l

= --p(rr)

(12.97a)

and p([rr\v1] 1)P([v1] 1 )

1

P([w lrr]1) =

p(rr)

eM([rrlv'],) = --p(rr)

the modifications to the metric made in (12.87), we can write (12.98a) and (12.98b) 1

where c is a constant that does not depend on [,1] 1 or [w ] 1 (see Problem 12.25). Now, vve can rewrite (12.96) using (12.97) and (12.98) as [ e{M

P(C)

=

([rrlv], )/2}+c/ p(rr)]

[e{M''([rrlv],)/2J+c/p(rr)]

+ [e{M'([rrlv'],)/2l+c/p(rr)]

eM''([Il'lv], )/2 eM''([rrlv],)/2

(12.99)

+ eM'([rrlv'],)/2

eti.1-I (Si)

1 + eti.,_1(Si).

the log-likelihood

or reliability, of this P(C) } ln { [1 - P(C)] -

decision is given (S; ).

We now show how the reliability of a path decision is associated with the harddecision of a Viterbi decoder. consider the decisions made at time I = m + 1. that the initial set of 2v path decisions. The decision at state S; has reliability (S; ), as shown in (12.100). The modulo-2 sum of the two information sequences being that is, [au] 111 corresponding to the codeword [w]111 and [au 1 ] 111 corresponding to the codeword [v1 ] 111 , indicates the error positions, namely, the bit positions in which the two information sequences differ. These are the information bits that are affected by the reliability of this path decision; that when these bits are decoded, the reliability of these bit decisions will depend on the reliability of this path decision. On the other the reliability of the decisions made for those bit positions where [au] 111 and [au1 ] 111 are the same will not depend on the reliability of this path decision. Thus, we form a reliability vector at time I = m + 1 for state S; as follows: L,,,+1(S;)

=

[Lo(S;), L1(S;), · · ·, Lm(S;)],

where

u;

if U/ f·t , , l=O,l, ... ,m.

I

U/

= u1

(12.101)

(12.102)

562

Chapter 12

Optimum Decoding of Convolutional Codes

In other words, the reliability of a bit position is either oo, if it is not affected by the path decision, or !:,. 111 (S; ), the reliability of the path decision, if it is affected by the path decision. In the same way, we can obtain a reliability vector L 111 +1 (S;) for each state S;, i = 0, 1, · · · , 2v - 1, at time l = m + 1. At time l = m + 2, the path decisions result in reliabilities !:,. 111 +1 (S; ), and the reliability vectors are given by L 111 +2(S;) = [Lo(S;), L1(S;), · · ·, Lm+1CS;)], i = 0, 1, · · · , 2v - 1. The reliability vectors are updated by first determining L 111 +1 (S;) as in (12.102) and then, for the remaining entries, taking the minimum of t,. 111 +1(S;) and the previous entry in the reliability vector for the error positions. The previous entries in the reliability vector for the bit positions in which [ID!]m+l and [ID1'Jm+ 1 are the same are left unchanged. This update procedure is repeated after each path decision, and for each state, a reliability vector whose length is equal to the surviving path must be stored along with the surviving path and its metric. For example, at time l = t and for state S;, we update the reliability vector as follows: (12.103)

where min[t,.1-1(S;), Lz(S;)] { L1(S;) min{Li, _ 1(S 1), L, - sCS1)}

t- 6

t-4

if uz -1- u; 'f I

l

U/

= u1

,

[

= 0, 1, ''' , f

-

1.

(12.104)

min{Li, _ 1(Si), L, - 3(S 1)}

t- 3

t-2

I

-1

/-

FiGURE 12. i8: Updating the reiiability vector following a path decision at time t.

Section 12.6

The BCJR.

12.18 vve illustrate the vector procedure that follows a illustration assumes that the branch path decision at time unit t. 1, whereas the "down" branch represents an input 0.) each state vector involves a few of the entries. The determination of the error requires no additional since the information sequences are stored by the as paths. (We [un]i-1 and [un']i-1 agree in them note here that for feedforward most recent bit but that this is not true in general for feedback vU'v~•-•v• The vector associated with a state keeps track of the reliabilities of all the information bits for that state. "When the end of the ~Vv~'-'"'·"" treHis is vector lLm+h (So) associated with the final the soft the same of the SOVA with finite path memory ,: is as described (see Problem 12.26). In this case, the reliability vectors store the reliabilities of the ,: most recent bits on each Thus, the vectors have the same memory as the Bits are decoded as in the truncated Viterbi algorithm, that at each time unit the survivor with the '""'"A'•A", and the bit ,: time units back on that is decoded. fo the SOVA the associated with that bit becomes the soft The of the SOVA is greater than that of the Viterbi algorithm because the SOVA must store vectors, one for each state. Computational is also increased, because of the need to update a reliability vector after each ACS however, the overall increase in complexity is modest, and the SOVA always decodes the ML in addition, provides soft outputs for concatenated SISO In the next section, we introduce MAP of convolutional codes, a decoding algorithm that maximizes the of correct decoding for each information bit and also soft outputs.

zv

12.®J nu:: ~cm 1/l\,!JGOlrffr~iM Given a Teceived sequence rt, the Viterbi finds the codeword ,y that maximizes the function. For a BSC. this is to finding the (binary) codeword v that is closest to the (binary) received sequence ir in Hamming distance. For the more general A WGN this is equivalent to the (binary) codeword v that is closest to the (real-valued) received sequence n· in Euclidean distance. Once the ML codeword i1 is determined, its corresponding information sequence au becomes the decoded output. Because the Viterbi algorithm finds the most likely codeword, it minimizes the VI/ER (E), that the =I= vlir) that the decoded (ML) codeword v is not equal to the transmitted codeword v, given the received sequence ir. In many cases, however, we are interested in minimizing the BER Pb(E), that is, the P(111 =I= u1 Irr) that the decoded information bit u1 at time l is not equal to the transmitted information bit u1 at time l. 1 = 0, 1. ... , K* - 1, rather than the WER (E). To minimize the BER, the a posteriori probability P(i11 = u1lrr) that an information bit u1 is correctly decoded must be maximized. An algorithm that maximizes P(i11 = u1lr) is called a maximum a probability (MAP) decoder. When the information bits are equally likely, the (ML) Viterbi algorithm

564

Chapter 12

Optimum Decoding of Convolutional Codes

maximizes P(v = vlr). Although this does not guarantee that P(u 1 = u!lr) is also maximized, P(v = vlr) is closely related to P(u1 = u1 Ir), as shown next, and the Viterbi algorithm results in near-optimum BER performance. The Viterbi algorithm maximizes P(v = vlr) = P(fi = milr), where 1U1 and 11.ll are the information sequences corresponding to the codewords v and v, respectively. This is equivalent to minimizing P(M -I- ulr). The BER is given by A

P(u1

lU!) J -I- u1lr) = [d(in, K* P(u -I- ulr), l = 0, 1, · · ·, K · -1. A

;•

(12.105)

We see from (12.105) that the BER depends on the Hamming distance between \Ul and u as well as on the WER P(M -I- Mir). Thus, minimizing P(it1 -=f. u1 Ir) also involves selecting an encoder with the property that low-weight codewords correspond to low-weight information sequences. This choice ensures that the most likely codeword errors, i.e., those with low weight, result in a small number of information bit errors, so that the BER is also minimized. We note that the class of systematic encoders satisfies this property, whereas the class of catastrophic encoders does not. In 1974 Bahl, Cocke, Jelinek, and Raviv [5] introduced a MAP decoder, called the BCJR algorithm, that can be applied to any linear code, block or convolutional. The computational complexity of the BCJR algorithm is greater than that of the Viterbi algorithm, and thus Viterbi decoding is preferred in the case of equally likely information bits. When the information bits are not equally likely, however, better performance is achieved with MAP decoding. Also, when iterative decoding is employed (see, for example, Chapter 16), and the a priori probabilities of the information bits change from iteration to iteration, a MAP decoder gives the best performance. In this section we describe the BCJR algorithm for the case of rate R = 1/n convolutional codes used on a binary-input, continuous-output A WGN channel and on a DMC. (The application of the BCJR algorithm to block codes is covered in Chapter 14.) Our presentation is based on the log-likelihood ratios, or L-values, introduced in the previous section. The decoder inputs are the received sequence r and the a priori L-values of the information bits La(u1), l = 0, 1, · .. , h - 1. As in the case of the SOYA, we do not assume that the information bits are equally likely. The algorithm calculates the a posteriori L-values L(u) 1

=

-

In [P(u1 = +llr)J P(u1 = -llr) '

(12.106)

called the APPL-values, of each information bit, and the decoder output is given by A

u1

=

{

+lifL(u1)>0 -1 if L(u1) < 0, I= 0, 1, ... , h -1.

(12.107)

In iterative decoding, the APP L-values can be taken as the decoder outputs, resulting in a SISO decoding algorithm. We begin our development of the BCJR algorithm by rewriting the APP value P(uz = +llr) as follows: p(ur P(uz

= +llir) =

= +1, ir) P(r)

=

LundJt p(rlv)P(l!ll) Lun p(irlv)P(llll) ,

/1")1()0\ ~l.L,.iVOJ

Section 12.6

The BCJR

where lUj is the set of an information sequences llil such that u 1 v is the transmitted codeword corresponding to the information sequence 11u, and is the pdf of the received sequence li" given v. Rewriting P (u1 = -1 lli") in the same way, we can write the expression in (12.106) for the APPL-value as (12.109)

where lUI is the set of all information sequences llil such that U/ = -1. IviAP can be achieved by computing the APPL-values L(u 1), l = 0, 1. .... h - 1 from (12.109) and then applying (12.107); however, for very short block lengths h, the amount of is For codes with a trellis structure and a reasonable number of states, such as short constraint convolutional codes and some block codes, employing a recursive based on the trellis structure of the code considerably process. making use of the trellis structure of the we can reformulate (12.108) as follows: uv~~·~OH

p(u1 = +l, li") P(u1 = +llli") = - - - - P(li")

L(s',s)El}

p(s1

= s', S/+l = s, Ir)

P(Il')

(12.110)

where r:,( is the set of all state pairs s1 = s' and s1+1 = s that to the = -1 Irr) in the same way, bit u1 = +1 at time l. Reformulating the expression we can now write (12.106) for the APPL-value as L(u1)=ln

L(s',s)EI/

p(s1

( L(s',s)EI:,-

p(s1

l

= s', S/+1 = s, Ir) , = s', Sf+l = s, n·)

(12.111)

where r:, 1 is the set of all state pairs s, = s' and s1+1 = s that bit u1 = -1 at time l. Equations (12.109) are equivalent for the APPL-value L(u1 ), but whereas the summations in (12.109) extend over a set of 211 - 1 information sequences, the summations in (12.111) extend only over a set of 2v state pairs. Hence, for large block lengths h, (12.111) is considerably simpler to evaluate. Note that every branch in the trellis connecting a state at time I to a state at time l + 1 is included in one of the summations in (12.111). We now show how the joint pdf's p(s', s, rt) in (12.111) can be evaluated recursively. We begin by writing p(s', s, rr)

=

p(s', s, 'iftI),

where li"t 1 represents the portion of the received sequence u: after time l. Now, apphcation of Bayes' rule yields p(s', s, u:)

=

p(Il't>lls', s, D:1<1, rr:1)p(s', s, fi't
=

p(rr:1>1ls', s, n·r
= p(li"1>1ls)p(s, li"1ls')p(s'.

ff:1<1),

ff:1<1)

(12.113)

566

Chapter 12

Optimum Decoding of Convolutional Codes

where the last equality follows from the fact that the probability of the received branch at time l depends only on the state and input bit at time l. Defining a1(s 1) = p(s', Jr1 <1)

=

(12.114a)

p(s, r1ls')

(12.114b)

f31+1(s) = p(r1>1ls),

(12.114c)

n(s', s)

we can write (12.113) as

= f31+1Cs)n(s 1 , s)a1(s 1).

p(s', s, ir)

(12.115)

We can now rewrite the expression for the probability a1+1 (s) as a1+1Cs)

=

p(s, ll'1<1+1)

=

L

p(s', s, ll't
1

s Ea1

=

L

1

1

p(s, r1/s , r 1<1)p(s , r 1<1)

s'Eat

=

L p(s, r1ls )p(s 1

(12.116) 1 ,

r1<1)

1

s Ea1

=

L

Y1(s

1 ,

s)a1(s 1),

1

s Ea1

where CJ/ is the set of all states at time l. Thus, we can compute a forward metric a 1+1 (s) for each states at time l + 1 using the forward recursion (12.116). Similarly, we can write the expression for the probability f31(s 1 ) as (see Problem 12.27) f31(s

1 )

L

=

n(s', s)f31+1Cs),

(12.117)

SECT/+[

where CJ/+l is the set of all states at time l + 1, and we can compute a backward metric (3 1(s') for each state s' at time l using the backward recursion (12.117). The forward recursion begins at time l = 0 with the initial condition ao(s)

={

1 s s

o:

=0 #0

,

(12.118)

since the encoder starts in the all-zero state So = 0, and we use (12.116) to recursively compute a 1+1Cs), l = 0, 1, · · ·, K -1, where K = h +mis the length of the input sequence. Similarly, the backward recursion begins at time l = K with the initial condition J 1, s = 0 (12.119) f3K(S) = 0, S 'F O ,

l

since the encoder also ends in the all-zero state So recursively compute f31(s), l = K - 1, K - 2, · · · , 0.

= 0, and we use (12.117) to

Section 12.6

The BCJR Algorithm

51!ii7

VVe can write the branch metric YI (s', s) as 1

/

I

Y1(s, s) = p(s, ll'1ls) =

p(S , S,

!1'/)

P(s')

= [P(s'. s)J [p(s', s. Ir1)] P(s')

(12.120)

P(s', s)

= P(sls')p(Irils', s) = P(u1)p(Ir1IY1), where u1 is the bit and s' -,, s at time /. For a

'V/

the output bits to the state transition A WGN channel, ifs' ---+ s is a valid state

(12.121)

where Ilu'/ - Yz 11 2 is the squared Euclidean distance between the (normalized by ./Es) received branch Ir/ and the transmitted branch Y/ at time l; however, ifs' -> s is not a valid state transition, P (s Is') and YI (s', s) are both zero. The algorithm that computes the APPL-values L(u1) using (12.111), (12.115), and the metrics defined in (12.116)-(12.119) and (12.121) is called the MAP algorithm. We now introduce some modifications to the algorithm outlined that result in greater computational efficiency. we note from (12.115)-(12.117) and ;if;\/1

(12.121) thatthe constant term ( V !fto)

appears raised to the power h in the

expression for the pdf p(s', s, rr). !fto will be a factor of every term in (V;i;;)"h the numerator and denominator summations of (12.111), and its effect will cancel. Hence, for simplicity, we can drop the constant term, which results in the modified branch metric (12.122) Next, we express the a priori probabilities P(u1 writing P(u1

= ±1) as exponential terms

[P(u1 = +1)/ P(u1 = -1)]±1 = ± 1) = - - - - - - - - - - - {1 + [P(u1 = +l)/P(u1 = -1)]±1}

(12.123) e-Lu(ui)/2

{1

=

+ e-L11
e"tLu(Llf)/2

Aie"1L11(u1)/2,

where, since L-values do not depend on the value of their argument, the parameter A1 is independent of the actual value of u1 (see Problem 12.28). We then use (12.123) to replace P(u1) in (12.122) for l = 0, 1. · · · , h - 1. that is, for each information bit. For the termination bits u1, l = h. h + 1, · · · , h + m - 1 = K - 1, however, where

568

Chapter 12

Optimum Decoding of Convolutional Codes

P(uz) = 1 and La(u1) = ±oo for each valid state transition, we simply use (12.122) directly. Thus, we can write (12.122) as

=

Aie"tLa (ui)/2 e(2E, I No)(r1-v1)-llr111

2

-llvi 11 2

(12.124a)

= Aze-(llr111 2 +11) eu1La(u1 )/2 e(L,/2)(r1·V1) = n(s', s)

A1Bze''tLa(u1)!2e
= P(uz)e-(E,/No)llr1-v111

l

= 0, 1, ...

'h - l,

2

= e-(E.,/No)llr1-v111 2 ,

(12.124b)

where B1 = I lll'1 I 12 +n is a constant independent of the codeword vi, and Le= 4£.,/ No is the channel reliability factor. We now note from (12.115)-(12.119) and (12.124) that the pdf p(s', s, r) 1 1 1 contains the factors fl(':5 At and [1~0 B1. Thus, A1 and [1~0 B1 will be factors of every term in the numerator and denominator summations of (12.111), and their effect will cancel. Hence, we can drop these factors and use the exponential function

fl/ :5

Y/(s', s)

= eu1L

n(s'. s)

=

11

(u1)/2e(L,/2)(r1-v1),

e
l

[

= 0, 1, .... h _

= h, h + 1, ... , K

1,

- l,

(12.125a) (12.125b)

as a simplified branch metric. Note that when the input bits are equally likely, La (u1) = 0, and the simplified branch metric is given by n(s', s)

= e
l

= 0. 1, ...

, K - l.

(12.126)

Finally, we see from (12.116), (12.117), and (12.125) that the forward and backward metrics are sums of 2k = 2 exponential terms. one corresponding to each valid state transition in the trellis. This suggests simplifying the calculations by making use of the identity (see Problem 12.29) max*(x, y)

=In(ex + ev) = max(x, y) + ln(l + e-lx-yl)

(12.127)

to replace the (computationally more difficult) operation ln(ex + eY) with a max function plus a lookup table for evaluating ln(l + e-lx-vl). To benefit from this simplification, we must first introduce the following log-domain metrics:

* (s,s)=lnn(s.s)I I -

y1

{

u1La(u1) 2 L, 2lf/·"1/,

+ L,,,. ·" 2 u/ v/,

1 = O, l, · · · 'h - l,

l=h,h+l,··· ,K-l,

(12.128a)

Section 12.6

oti~-l (s) = ln

ot1+1Cs)

= In

L

The BC.JR Algorithm

5@9

n(s', s)ot1(s')

s' EC!/

= ln

L

elY/(s'.s)+a1'(s')]

(12.128b)

1

s ECJ't

= max';, E<Jt [y/' (s', s) + ot;' (s')],

oti5-·· cS ) = 1n f,/(s')

= ln

c ) = lr o. ~ -oo,

OIO S

f,1(s')

= ln

L

l

= 0, 1, · · · . K

s = _@ ii]) s =;=1 •

- 1,

(12.128c)

Y1(s', s)P1+1Cs)

SEU/+]

=ln

:E

,s)+.Bf+ 1(s)]

SE
[y/'(s', s) + N+i (s)], l p;<(s)

= ln

f,K(s)

=J

l

O, -00,

s= s ¥=

=

K - 1, K - 2, · · · , 0,

ii])

(12.128e)

ii])_

The use of the max* function in (12.128b) and (12.128d) follows from the fact that these recursive equations involve calculating the log of a sum of two exponential one corresponding to each valid state transition. Further, we can now ·write the for the pdf p(s'. s, ir) in (12.115) and the APPL-value L (u1) in (12.111) as p(s', s, Ir)= e.Bi~1(s)+y/(s'.s)+o,;'(s') (12.129) and

- ln

I,

L,_ ,,;, ,c,1+r,'(i ,)-N/U'l 1,

(s .s)EL 1

j

(12.130)

and we see that each of the two terms in (12.130) involves calculating the log of a sum of 2v exponential terms, one corresponding to each state in the trellis. Now, we note that we can apply the max* function defined in (12.127) to sums of more than two exponential terms making use of the result (see Problem 12.29) max*

y,

z) = ln(ex

+ eY + e~) = max*[max*(x, y), z],

(12.131)

in other we can evaluate a 3-variable max* function by applying the 2-variable max* function twice. Finally, using (12.131) in (12.130) allows us to express the APP L-values as .s)E2::f [.Bt'+i (s)

+ y((s', s) + ot/(s')]

-max*('s ,s JE,_,"-[Ptt- 1 (s)+y;"(s',s)+a;'(s')]. 1

(12.132)

570

Chapter 12

Optimum Decoding of Convolutional Codes

The algorithm that computes the APPL-values L(u1) using (12.132), the logdomain metrics defined in (12.128), and the max* functions defined in (12.127) and (12.131) is called the log-MAP algorithm, or the log-domain BCIR algorithm. The log-MAP algorithm, because it uses just a max(®) function and a lookup table, is considerably simpler to implement and provides greater numerical stability than the MAP algorithm.

The (Log-Domain) BCIR Algorithm Step 1, Initialize the forward and backward metrics a 0(s) and fJf (s) using (12.128c) and (12.128e). Step 2, Compute the branch metrics Y/'(s', s), l = 0, 1, ... , K - 1, using (12.128a). Step 3, Compute the forward metrics a?+i (s), I = 0, 1, · · · , K - 1, using (12.128b). Step 4. Compute the backward metrics fJz* (s'), l = K - 1, K - 2, ... , 0, using (12.128d). Step 5, Compute the APPL-values L(u1), I= 0, 1, · · · , h -1, using (12.132). Step 6. (Optional) Compute the hard decisions u1 , I = 0, 1, ... , h - 1, using (12.107). The calculations in each of steps 3, 4, and 5 involve essentially the same amount of computation as the Viterbi algorithm, and thus the BCJR algorithm is roughly three times as complex as the Viterbi algorithm. Also, we see from (12.128a) that the BCJR algorithm requires knowledge of the channel SNR Es/ No (Le= 4Es/ No) to calculate its branch metrics. The branch metrics for the Viterbi algorithm, on the other hand, are just the correlations rr1 ·Vz, and no knowledge of the channel SNR is needed. (Note that the constant C1 (= Lc/2) can be dropped from the Viterbi algorithm metric in (12.14) because it does not affect the decoding result, but the constant Lc/2 cannot be dropped from the BCJR algorithm metric in (12.128a), because its relation to the a priori factor u1La(u1)/2 does affect the decoding result.) When the BCJR algorithm is used for iterative decoding (see Chapter 16), the computation of the hard decisions in step 6 takes place only at the final iteration. At earlier iterations, the APPL-values from step 5 are the (soft) decoder outputs. The basic operations of the log-domain BCJR algorithm are illustrated in Figure 12.19. An even simpler algorithm results if we ignore the correction term ln(l + e-lx-yl) in the max* function and simply use the approximation max*(x, y)

~

max(x, y)

(12.133)

instead. Because the correction term is bounded by 0 < ln(l

+ e-lx-yl)

::: ln(2)

= 0.693,

(12.134)

the approximation is reasonably good whenever I max(x, y) I :::: 7. The algorithm that computes the APP L-values L(u1) using the function max instead of max* is called the Max-log-MAP algorithm. Because the max function plays the same role as the compare-and-select operations in the Viterbi algorithm, the forward recursion in the Max-log-MAP algorithm is equivalent to a forward Viterbi algorithm, and

The BCJR

Section 12.6

a.1(s . ';), / . (,,, ''J') 0

a.1 +

'-5\,S)

o

f3i (s')

1(s)

~ o

'"jl

a.1(s;)

Y; ( s ~ o

a./+ 1(s)=max(oftwosums)

/31 . i(s)

{3 1 (s') = max (of two sums)

(b)

(a)

a.1

---- Input -1

(sf)

- - Input +l

Y; (s'

''"l)

/3/T L(u 1) = max (/3 1+ 1 + y/ +

a.1

I (s,i)

for solid lines) - max (/3'1+ 1 + y 1 + (c)

a./

for dashed lines)

FIGURE 12. 19: Trellis diagrams representing (a) the forward recursion of

(b) the backward recursion of (12.128d), and (c) the APP L-value computation of (12.132) (adapted from [26]). the backward recursion in the Max-log-MAP algorithm is equivalent to a backward Viterbi algorithm. The difference between the BCJR and the Viterbi algorithms, is in the correction term ln(l + e-1.r-YI), which accounts for the optimality of the BCJR algorithm with respect to BER The Viterbi algorithm, which is optimized for WER typically results in a slight BER performance degradation compared with the BCJR algorithm, All three algorithms presented in this section: MAP, log-MAP, and Maxlog-MAP, are examples of forward-backward algorithms, since they involve both a forward and a backward recursion. They are also examples of SISO decoders, since we can take the (real) APPL-values L(u1) as soft decoder outputs instead of applying (12.107) to obtain hard decoder outputs. We now give an example of the operation of the log-domain BCJR algorithm,

512

Chapter 12

EXAMPLE 12.:S:

Optimum Decoding of Convolutional Codes

BCJIR Dernding of a (2, 1, 1) Systematic !Recursive Ccmvoh.1-

Consider the (2, 1, 1) SRCC with generator matrix G(D)

= [1 1/(1 + D)]

(12.135)

whose encoder diagram is shown in Figure 12.20(a). The 2-state trellis diagram, with branches labeled according to the mapping O---+ -1 and 1 ---+ +1, corresponding to an input sequence of length 4, is shown in Figure 12.20(b). 6 Let u = (uo, u 1 , u2, u 3 ) denote the input vector of length K = h + m = 4, and w = (wo, w1, w2, w3) denote the codeword of length N = nK = 8. We assume a channel SNR of Es/No 1/4 (-6.02 dB) and a (normalized by~) received vector l!'

=

(ro, r1, r2, l!'3)

=

0 1 0 1 0 1 (rci l, rci ); ri ), r?); ri°), rJ l; rj l, rj ))

= (+0.8, +0.1; +1.0, -0.5; -1.8, +1.1; +1.6, -1.6).

(12.136)

(a)

+ 1.6, -1.6)

+ 1.0. -0.5 (b)

FIGURE 12.20: (a) A (2, 1, 1) systematic feedback encoder and (b) the decoding trellis for the (2, 1, 1) SRCC with K = 4 and N = 8.

6In Figure 12.20(b), the branch labels

vf

0 1 0 u1/v; )v) ) indicate both the input u1 and the outputs v) l

and l. This is because, for feedback encoder trellises, the upper branch leaving each state does not necessarily correspond to a 1 ( + 1) inpvt, and the lo,.ver branch to a O (-1) input. ..Also note, since the encoder is systematic, that the first output label on each branch equals the input label.

Section 12.6

The BCJR Algorithm

5B

(Note that, since the rate of the terminated code is R = h / N = 3 /8, an Es/ No = 1/4 (-6.02dB) corresponds to an E1,/No = Es/RNo = 2/3 (-l.76dB).) The received vector rr is also shown in Figure 12.20(b ). Assuming that the a probabilities of the information bits are equally likely, La (u1) = 0. l = 0, 1. 2, and we the branch metrics (12.128a) (note that = 4Es / No = 1) as follows: Yo'(So, So) =

-1

2 1

= -(-0.8 - 0.1) = -0.45 2

I

Yo'(So, S1)

+1 2

1 + -rro·Vo

=

~La(uo)

=

1 2(0,8 + 0.1)

*

Yi (So, So) =

2

= 0.45

-1

2 1

= 2(-1.0 + 0.5) = -0.25 ,

Jli'(So, S1) =

+1

2

L"(iq)

1

+ 21ri·v1

1 2

= (1.0 - 0.5) = 0.25 -1 La ' (U[ ) + 1 ir1·V1 Yi'_,, (Sl• S1 ) = 2 2

1

= 2(-1.0 - 0.5) = -0.75 (S1, So)=

+1

2 1

= 2(1.0 + 0.5) = 0.75 *

y 2 (So, So)=

-1

2

La(u2)

+1

1

= 2(1.8 - 1.1) = 0.35 ,

+1

Yi (So, S1) = 2

La(u2)

1

+ 2ir2·v2

1

= 2(-1.8 + 1.1) = -0.35 Yi"'(S1, S1)

=

-1

2

La(u2)

1

+ 2ir2·v2

574

Chapter 12

Optimum Decoding of Convolutional Codes

1 = 2(1.8 + 1.1) = 1.45 ,_ +1 1 y£(S1, So)= 2 La(u2) + 2 r2·V2 1 = 2(-1.8 -1.1) = -1.45

*

+1

Y3 (So, So) = 2r3·V3

1 = 2(-1.6 + 1.6) = 0 +1 Y3* (S1, So) = 2r3•W3 1 = 2 (1.6 + 1.6) = 1.60.

(12.137)

We then compute the log-domain forward metrics using (12.128b) as follows:

a'{' (So) = [y0(So, So) + a0(So)] = -0.45 + 0 = -0.45 aT(S1) = [yo'(So, S1) + <Xo(So)] = 0.45 + 0 = 0.45 a 2'(So) = max*{[y((So, So)+ ai'(So)]. [yi*(S1, So)+ ai'(S1)]} = max* {[ (-0.25) + (-0.45)]. [(0.75) + (0.45) ]l

(12.138)

= max*(-0.70, +1.20) = 1.20 + ln(l + e-1- 19 1) = 1.34 a 2(S1) = max*{[yi*(So. S1) + ai'(So)]. [yi*(S1. S1) + ai'(S1)]} = max*(-0.20, -0.30) = -0.20 + ln(l +

e-lO

11) = 0.44.

Similarly, we compute the log-domain backward metrics using (12.128d) as follows: /3{(So) = [y3(So, So)+ /34'(So)] = 0 + 0 = 0 /3{(S1) = [y3*(S1, So)+ /34(So)] = 1.60 + 0 = 1.60 /32'(So) = max*{[Y:2(So, So)+ /3{(So)]. [y2''(So, Si)+ /33'(S1)]} = max*{[(0.35) + (0)), [(-0.35) + (1.60)]} = max*(0.35, 1.25) = 1.25 + ln(l + e-1-o. 9o1) = 1.59 /32(Si) = max*{[y2(S1, So)+ /3{(So)), [y2(S1, S1) + /3 3'(S1)]l = max*(-1.45, 3.05) = 3.05 + ln(l +

e-l- 45 1)

= 3.06

/3I(So) = max*{[y((So. So)+ /3 2(So)), [y{'(So. S1) + /32(S1)]} = max*(l.34, 3.31) = 3.44 /3i'(S1) = max*{[y{'(S1, So)+ /32'(So)], [y{'(S1, S1) + /3 2'(S1)]} = max*(2.34, 2.31) = 3.02.

(12.139)

Section 12.6

the APP L-values for the three information bits

we as follows: L(uo) =

(S1) +

Yo(So.

= (3.47) L(zq)

515

(12.132)

S1) + oto(So)] - [/3t(So) + Yo'(So, So)+ oto(So)]

= +0.48

= max*{[/3 2(So) +

y{"(S1,

- max*{[/3;{(So) +

So)+ a 1(S1)], [/32(S1) + y;'(So, S1) + a 1(So)]}

y{'(So.

[(2.79), (2.86)] - (2.90) L(u2) =

The BCJR Algorithm

So)+ ot1'(So)], [/32(S1) +

(S1, S1) + ai'(S1)]}

[(0.89), (2.76)]

= +0.62

max*{[/33'(So) +

(S1. So)+ ot 2(S1)], [/3 3'(S1) +

- max* {[/33'(So) + y2'(So, So) +

ot

2

y '(So,

S1) + ot 2(So)]}

2(So)], [/33(S1) + Y:{ (S'i. S1) +

(S1)]}

[(-1.01), (2.59)] - max*[(l.69), (3.49)]

= (2.62) - (3.64) = -1.02.

(12.140)

Using (12.107), we obtain the hard-decision outputs of the BCJR decoder for the three information bits: (12.141) ill!= (+l, +l, -1). Even though termination bit u3 is not an information bit, it is also possible to use the same to find its APP L-value. As will be seen in Chapter 16, this is necessary in iterative decoding, where "soft-output" APPL-values are as a priori inputs to a second decoder. In the preceding example, the decoder is operating at an SNR of E&/ No = -1. 76 dB, well below the capacity of E1, / No = -0.33 dB for rate R = 3 /8 codes (see Table 1.2). Although this is not normally the case for BCJR decoding of a convolutional code, it is not at all unusual in iterative decoding of turbo codes, where decoders at SNRs below capacity (see Chapter 16). We now revisit Example 12.8 to see the effect of applying the Max-log-MAP algorithm.

fo the Max-log-MAP algorithm, the branch metrics remain as computed in (12.137). The max* y) ~ . y) of (12.133) affects the computation of the forward and backward metrics (see (12.138) and (12.139), respectively), as follows:

cv 2(So) = max(-0.70, +1.20) = 1.20

2(S1) = max(-0.20, -0.30) = -0.20 /3 2'(So) = max(0.35, 1.25) = 1.25 /3 2'(Si) = max(-1.45, 3.05) = 3.05

ot

576

Chapter 12

Optimum Decoding of Convolutional Codes

= max(l.34, 3.31) = 3.31

/3i'(So)

/3~' (S1) = max(2.34, 2.31) = 2.34.

(12.142)

Then, we compute the APP L-values for the three information bits using (12.132). (12.133), (12.137), and (12.142) as follows: L(uo) = [/3!(S1) + y0'(So, S1) + ao(So)] - [/3!(So) + Yo'(So, So)+ ao(So)]

= (2.79) - (2.86) = -0.07 L(u1)

= max{[/32'(So) +

Yt(S1,

So)+ a!(S1)], [/32(S1) + Yt(So, S1) + a!(So)]}

- max{[/32'(So) + Yt(So, So)+ a!(So)], [/32 (S1) + y{(S1, S1) + af(S1)]} = max[(2.79), (2.86)] - max[(0.89), (2.76)]

= (2.86) - (2.76) = +0.1000

2

2

2

2

L(u2) = max{[/3;'(So) + y (S1, So)+ a (Si)], [/3~'(S1) + y (So, S1) + a (So)]}

- max{[/3;'(So) + y 2(So, So)+ a 2(So)], [/3i(S1) + y2*(S1, S1) + a 2(S1)]} = max[(-1.65), (2.45)] - max[(l.55). (2.85)]

= (2.45) - (2.85) = -0.4000.

(12.143)

Using (12.107), we obtain the hard-decision outputs of the Max-log-MAP decoder: Ill!= (-1, +1. -1).

(12.144)

and we see that the Max-log-MAP algorithm does not give the same result as the log-MAP algorithm in this case. The log-MAP and Max-log-MAP algorithms do not give the same decoding result because the decoders are operating at an SNR well below capacity. Typically. when these algorithms are used to decode convolutional codes at SNRs above capacity, they will give the same result, although the performance of the Maxlog-MAP algorithm is somewhat degraded at very low SNRs. Also, as will be discussed in Chapter 16, the performance loss of the Max-log-MAP algorithm is more pronounced when it is used for iterative decoding, since the approximation error accumulates as additional iterations are performed. We now give an example illustrating the application of the BCJR algorithm to decoding on a DMC. In this case it is convenient to compute the APPL-values L(u1) using the probability-domain version of the algorithm given by expressions (12.111) and (12.115)-(12.120), where in (12.120) we compute the branch metrics n(s', s) by substituting the (discrete) channel transition probabilities

n

n-1

P(r1lv1)

=

P(r?Jlv?), l

= 0, 1, · · ·, K

- l

(12.145)

j=O

for the (continuous) channel transition pdf's p(r1lv1 ). We then use the branch metrics in (12.115) to calculate the (discrete) joint probabilities P(s', s, r), which we

Section 12.6

The BCJR

57/7/

in turn use to calculate the APP L-values using (12.111). Also, a of the forward and backward metrics a1 (s) and fh (s 1) results in exceedingly small values, which can lead to numerical precision problems. Hence, we introduce normalization constants a1 and b1 and normalized forward and backward metrics (s) and B1(/), which are defined as foHows:

L

Cl/ (s)

= G/,

A1(s)

= ot1(s)/a1,

/31(/)

= b1,

B1(/)

= /31(/)/b1,

f

= 1, 2, · · · ,

(12.146a)

SECY/

L

l

=

= 1, 2. · · ·, /(,

l

alls E

(12.14-6b)

CJ/,

K - 1, K - 2, · · · , 0,

s'Ecr;

l = K - 1, K - 2, · · · , 0, all/

E CJ/.

(12.146d)

From (12.146) we can see that the normalized forward and backward metrics sum to 1 at each time unit!; that is

L

A1(s)

=

1, l

= 1, 2, · · · ,

(12.147a)

=K-

(12.147b)

SECY/

L (/) = 1,

l

1, K - 2, · · · , 0.

1

S EO)

Using the normalized forward and backward metrics A1(s) and B1(s 1 ), (12.116), (12.117), and (12.146), instead of a1(s) and /31(s 1), evaluate the joint probabilities in (12.115) avoids numerical precision and has no effect on the calculation of the final APP L-values in (12.111) (see Problem 12.32). IEXAMPLIE 12.!'ll (©ro1® ©111 ©J DM(

Assume a binary-input, 8-ary output DMC with transition given by the following table:

P(r?)lv;fl)

,.(J)

,I

0

0.434

0.197

0.167

0.111

0.058

0.023

0.008

0.002

1

0.002

0.008

0.023

0.058

0.111

0.167

0.197

0.434

Now, consider the (2, 1, 2) nonsystematic convolutional code with generator matrix G(D)

=

[1

+ D + D2 1 + D2]

(12.148)

whose encoder diagram is shown in Figure 12.21(a). The 4-state trellis diagram corresponding to an input sequence of length 6 is shown in Figure 12.21(b). Let 1lll = (uo, iq, u2, u3, u4, us) denote the input vector of length K = h + m = 6,

578

Chapter 12

Optimum Decoding of Convolutional Codes

ll!

(a)

(b)

FIGURE 12.21: (a) A (2, 1, 2) nonsystematic feedforward encoder and (b) the decoding trellis for the (2, 1, 2) code with K = 6 and N = 12.

and v = (vo, v1, v2, v3, v4, vs) denote the codeword of length N assume that P( U/ -_ 0)

={

2/3,

1,

l

[

= 0, 1, 2, 3 = 4, 5

(information bits) ( termination bits),

= nK = 12.

We

(12.149)

that is, the information bits are not equally likely, and that the received vector is given by (12.150)

Section 12.6

The BCJR Algorithm

51~

The received vector iris also shown in Figure 12.21(b). (Note since the encoder is feedforward, the upper branch leaving each state corresponds to a 1 input and the lower branch to a Oinput, and the termination bits must be O's.) Vve begin the algorithm the (probability-domain) branch metrics using (12.120) as follows: Yo(So, So)= P(uo = O)P(l401\00) = (2/3)P(l4\0)P(01\0)

= (2/3)(0.058)(.434) = 0.01678, (12.151) Yo(So, S1) = P(uo = l)P(L1.01\ll) = (1/3)P(l4\l)P(0 1\1)

= (1/3)(0.111)(.002) = 0.000074, and so on. The complete set of branch metrics for the trellis is shown in Figure 12.22(a). Next, we can the (probability-domain) normalized forward and backward metrics using (12.116)-(12.119), (12.146), and (12.147) as follows: a1 (So) = Yo(So, So)ao(So) = (0.01678)(1) = 0.01678, a1 (S1) = Yo(So, S1)ao(So) = (0.000074)(1) = 0.000074, a1 = a1 (So)+ a1 (S1) = 0.01678 + 0.000074 = 0.016854, A1(S0) = a1(So)/a1 = (0.01678)/(0.016854) = 0.9956, A1 (S1) = 1 - A1 (So) = 1 - 0.9956 = 0.0044, (12.152) /3s(So) = Ys(So, So)/36(So) = (0.0855)(1) = 0.0855,

/35(S2) = y5(S2, So)/36(So) = (0.000016)(1) = 0.000016, bs = ,Bs(So) + /35(S2) = 0.0855 + 0.000016 = 0.085516, B5(So) = j3 5 (So)/bs = (0.0855)/(0.085516) = 0.9998,

B5(S2) = 1 - Bs(So) = 1 - 0.9998 = 0.0002, and so on. The complete set of normalized forward and backward metrics for the trellis is shown in Figures 12.22(b) and (12.22c ), respectively. Finally, using the normalized forward and backward metrics in (12.115) to compute the joint probabilities P(s', s, rr), which we then use to calculate the APP L-values in (12.111), and recalling the mapping O--,,. -1 and 1 --,,. + 1, we obtain L(uo)

= ln {

P(so P(so

= =

So,

s1 = S1, n·)}

So, s1

=

So, Il')

I l I l = in I co.8162)co.000074)(1)} = l (0.1838)(0.01678)(1) = ln B1 (S1)Yo(So, S1)Ao(So) B1 (So)Yo(So, So)Ao(So)

(12.153a) _ 3 .933

'

580

Chapter 12

Optimum Decoding of Convolutional Codes

(a)

1.000

.9956

.0055

.0903

.0695

.3430

.1773

.5266

.5581

1.000

(b)

FIGURE 12.22: (a) The branch metric values n(s', s), (b) the normalized forward metric values Az (s ), and (c) the normalized backward metric values Bi (s') for Example 12.9.

Section 12.6

1.000

.1838

moo

.1060

The BCJR Algorithm

.0721

.9998

Si~1

1.000

(c)

FIGURE 12.22: (continued)

P(s1 = So, s2 = S1, Ir)+ P(s1 = S1, s2 = S3, Ir) L(u1) = ln - - - - - - - - - - - - - - - P(s1 = So. s2 = So, Ir)+ P(s1 = S1, s2 = S2, Ir)

+ B2(S3)Y1(S1, S3)A1(S1) + B2 (S2)Y1 (S1, S2)A1 (S1) (12.153b) = ln (0.2028)(0.003229)(0.9956) + (0.5851)(0.006179)(0.0044) (0.1060)(0.001702)(0.9956) + (0.1060)(0.0008893)(0.0044) = ln

B2(S1)Y1(So, S1)A1(S0)

B2 (So)Yl (So. So)A 1 (So)

and, following the same procedure (see Problem 12.33), we have L(u2)

= +1.234,

(12.153c)

L(u3)

= -8.817.

(12.153d)

and Using (12.107), and again recalling the mapping O---+ -1 and 1---+ +1. we see that the hard-decision outputs of the BCJR decoder for the four information bits are given by l1l = (i10, ii1. Ct2, ii3) = (0, 1, 1, 0). (12.154) In this case, since the encoder is feedforward and the termination bits are known to be O's, it is not necessary to find the APPL-values of u4 and u5, even as part of an iterative decoding procedure.

582

Chapter 12

Optimum Decoding of Convolutional Codes

Although it is convenient to use the probability-domain BCJR algorithm for DMCs, the log-domain algorithm can also be used. Similarly, the probability-domain algorithm can be used for A WGN channels. 12.7

PUNCTURED AND TAil-BmNG CONVOUJTIONAl CODIES

The trellis-based decoding procedures for convolutional codes described previously in this chapter are well suited for medium-to-low encoder rates R = k/n and moderate-to-large information sequence block lengths K* = hk. For high-rate codes and short block lengths, however, modified encoding and decoding procedures are sometimes useful. In particular, since the branch complexity of the decoding trellis increases exponentially with k for a standard convolutional code, a modified form of high-rate code, called a punctured convolutional code, which has only two branches leaving each state, was introduced in a paper by Cain, Clark, and Geist [27]. Also, since the fractional rate loss of a terminated convolutional code is large when h is short, a modified form of terminated code, called a tail-biting convolutional code, which has no termination bits and therefore no rate loss, was introduced in papers by Solomon and van Tilborg [28] and Ma and Wolf [29]. A rate R = (11 - 1) / 11 punctured convolutional code is obtained by periodically deleting or puncturing certain bits from the codewords of a rate R = 1/2 mother code, where 11 ~ 3. These codes can then be decoded using the Viterbi algorithm ( or the BCJR algorithm) with roughly the same decoding complexity as the rate R = 1/2 mother code. This is because, in the rate R = (n-1)/n punctured code, only two branches enter each state, and thus only one binary comparison is performed at each state, rather than the 2
Consider the 4-state, rate R = 1/2 mother code generated by the (2, 1, 2) nonsystematic feedforward convolutional encoder with generator matrix G(D) = [1

+ D2

1+D

+ D 2 ].

(12.155)

This is the optimum free distance code from Table 12.l(c) with dfree = 5. Now, consider forming a rate R = 2/3 punctured code by deleting the first bit on every other branch of the trellis. Six sections of the trellis diagram for this punctured code, with the deleted bits indicated by an x, are shown in Figure 12.23(a). In the punctured code, only one symbol is transmitted on those trellis branches where a bit has been deleted. Thus, the trellis diagram for the punctured code is time-varying, with a period of two branches. A Viterbi or BCJR decoder for this rate R = 2/3 punctured code would operate exactly like the decoder for the mother code, except that no metrics would be computed for the deleted symbols. Thus, the metric computation would involve two symbols on half the branches, and only one symbol on the other half. In each twobranch section of the decoding trellis for the punctured code, a total of two binary comparisons per state are required to decode two information bits. By contrast, in each section of the decoding trellis for a standard rate R = 2/3 code, a total of three binary comparisons per state are required to decode two information bits. Hence,

Section 12.7

Punctured and Tail-Biting Convolutional Codes

5~35

decoding the rate R = 2/3 code, based on the structure of the rate R = 1/2 mother code, is less complex than decoding a standard rate R = 2/3 code. A rate R = 3/4 punctured code can be obtained from the same rate R = 1/2 mother code by deleting two bits from each set of three as shown in

(a)

(b)

FIGURE 12.23: Trellis diagrams for (a) a rate R = 2/3 punctured convolutional code and (b) a rate R = 3/ 4 punctured convolutional code.

584

Chapter 12

Optimum Decoding of Convolutional Codes

Figure 12.23(b). In this case, in each three-branch section of the trellis for the punctured code, a total of three binary comparisons per state are required to decode three information bits. In contrast, in each section of the trellis for a standard rate R = 3/4 code, a total of seven binary comparisons per state are required to decode three information bits. Again, decoding the rate R = 3/4 punctured code, based on the simpler structure of the rate R = 1/2 mother code, is less complex than decoding a standard rate R = 3/4 code. By carefully examining the trellises in Figure 12.23, we see that the minimum free distance of each punctured code is dfree = 3. In other words, puncturing a rate R = 1/2 mother code to a higher rate, as expected, reduces the free distance. These free distances, however, are equal to the free distances of the best standard 4-state rate R = 2/3 and 3/4 codes listed in Tables 12.l(d) & (e), respectively. In other words, no penalty in free distance is paid for the reduced decoding complexity of the punctured codes in these cases. In general, however, this is not so, since rate R = (n - 1)/n codes obtained by puncturing a rate R = 1/2 mother code form just a subclass of all possible rate R = (n - 1)/n convolutional codes.

In Table 12.4 (a) and (b), we list the best rate R = 2/3 and 3/4 codes, respectively, that can be obtained by puncturing a rate R = 1/2 mother code. In each case, the puncturing pattern is indicated using a 2 x T binary matrix JP', where T is the puncturing period, the first row of lP' indicates the bits to be deleted from the first encoded sequence, and the second row of JP' indicates the bits to be deleted from the second encoded sequence. (In the matrix P, a Oindicates a bit to be deleted, and a 1 indicates a bit to be transmitted.) For example, in Figure 12.23, the puncturing patterns are given by JP' = [

and lP'

=[

i~ J( i ~ ~ J.

Figure 12.23(a))

(12.156a)

(Figure 12.23(b))

(12.156b)

In applications where it is necessary to support two or more different code rates, it is sometimes convenient to make use of rate-compatible punctured convolutional (RCPC) codes [31]. An RCPC code is a set of two or more convolutional codes punctured from the same mother code in such a way that the codewords of a higher-rate code can be obtained from the codewords of a lower-rate code simply by deleting additional bits. In other words, the set of puncturing patterns must be such that the IP' matrix of a higher-rate code is obtained from the P matrix of a lowerrate code by simply changing some of the 1's to O's. (This property implies using puncturing matrices with the same period T for all code rates and does not apply to the two puncturing matrices in (12.156).) An RCPC code then has the property that all the codes in the set have the same encoder and decoder. The higher-rate codes simply have additional bits deleted. This property is particularly convenient in two-way communication systems involving retransmission requests, where the initial transmission uses a high-rate punctured code and then, if the transmission is unsuccessful, punctured bits are sent during later transmissions, resulting in a more

Section 12.7

Punctured and Tail-Biting Convolutional Codes

5~5

powerful lower-rate code for decoding. (See Chapter 22 for more details on two-way communication TABLE 12.4: Optimum (a) rate R = 2/3 and (b) rate R = 3/4 convolutional codes obtained a rate R = 1/2 mother code.

Adapted from [30]. M,r])'~!lneir (C(!])«!]e

IP'i!Ilillldi]]]I!'le«!l (Cl!})cdl
j)

~(O)

~(1)

2

5

7

3

13

17

4

31

27

5

65

57

6

155

117

IP 1 1 1 0 1

~(O)

~(1)

2

5

7

3

13

17

4

31

27

5

65

57

6

155

117

0

1 1 1 1 1 0 1 1 0

0

1 1 1 1

3

1

4

3

4

1

6

19

6

1

JP'i]]]ll11(\.'.lii]]]Jrle«!l (C(!])«!l
M(!])\lllnerr (C(!])cdle j)

AaFee

IP'

1 1 1 1 1 1 1 1 1 1

0 1 1 0

dfree

1 0

0 1 0 1 1 0 0 0 1 1 1 0 0 1

3

6

4

29

3

1

4

1

5

8

(b)

A tail-biting convolutional code is obtained by terminating the encoder output sequence after the last information block in the input sequence. In other words, no "tail" of input blocks is used to force the encoder back to the all-zero state. In this case, the input sequence has length K* = hk, the output sequence has length JV* = hn, and the rate of the resulting tail-biting convolutional code is [(*

Rib= -

JV*

hk

k

hn

n

= - = -.

(12.157)

586

Chapter 12

Optimum Decoding of Convolutional Codes

We see from (12.157) that the rate Rib of the tail-biting code is the same as the rate R of its convolutional encoder; that is, there is no rate loss associated ;vith this treUis termination technique. H is not difficult to show that with any of the standard trellis-based decoding techniques, the lack of a tail causes the information bits near the end of the trellis to receive less protection against the effects of channel noise than the bits earlier in the trellis. (In fact, the reason for having a tail is to provide the same level of protection for all the information bits.) In other words, the bits near the end of the trellis will have a higher decoded error probability than the bits near the beginning of the trellis, leading to an unequal error protection property. Although this may actually be desirable in some applications, in most cases we want a uniform level of error protection throughout the entire information sequence. To provide a uniform level of error protection for the entire information sequence and to maintain the zero-rate-loss property, we can modify the tail-biting technique to require that valid codewords start and end in the same state. Because codewords without a tail can end in any state, they must also be allowed to start in any state. In other words, rather than requiring the encoder to start in the all-zero state So, we require the encoder to start in the same state as its ending state, which can be determined from the information sequence. This restriction results in a set of 2K"' = 2hk codewords, of which a subset of 211 k-v codewords starts and ends in each of 2v states. (If hk :s= v, then a fraction 2hk-v of the states contain one valid codeword each.) A tail-biting convolutional code can thus be viewed as a set of 2v subcodes, each of which contains 2hk-v codewords. Maximum likelihood decoding is then performed by applying the Viterbi algorithm separately to each of the 2v subcodes, resulting in 2v candidate codewords, and then choosing the codeword with the best overall metric from among the 2v candidates. Similarly, MAP decoding can be performed by applying the BCJR algorithm to each of the 2v subcodes and then determining the APP L-values of each information bit by combining the contributions from each subcode. When the number of states is too large for ML decoding, a suboptimum "two-cycle" decoding method that starts decoding in an arbitrary state, say the all-zero state So, finds the ending state, say state Si, with the best survivor metric, and then repeats the Viterbi algorithm for the subcode that starts and ends in state Si, achieves reasonably good performance [29]. Similar suboptimum versions of MAP decoding for tail-biting codes are presented in [32]. We now illustrate the tail-biting technique with an example. IEXAMPLIE 12.11

Consider the (2, 1, 2) nonsystematic feedforward convolutional encoder of Example 12.9. The 4-state trellis diagram shown in Figure 12.2l(b) represents an information sequence of length K* = h = 4, an input sequence of length K = h + m = 6, and an output sequence of length N = (h + m)n = 12, resulting in a terminated convolutional code of rate K* 4 1 Rr = ~ = - = -. (12.158) N 12 3 Because the information sequence length h is short, the terminated code rate J?_, == 1/3 is much less than the encoder rate I?. == 1/2~ that is, a 33~{, rate loss occurs in this case. The minimum (free) distance of this (12, 4) block (terminated

Section 12.7

Punctured and Tail-Biting Convolutional Codes

5~1

convolutional) code is dmin = 5, and the number of minimum weight codewords is As= 4. Now, consider the 4-state tail-biting trellis shown in Figure 12.24(a) corre.... ~."'~''"!Co to an information sequence 1lil = (uo, iii, u2, u3, u4. us) of length K* = h = 6, where we have shown the input labels as well as the output labels on each trellis branch. There are 21c = 26 = 64 codewords v = (vci°l v61), vi°) v?). ViO) Vil). v~Ol vf).

1vl1\

0 vi

0 1 v~ lv~ l) of

N* = hn = 12 in this code, resulting in a tail-biting

convolutional code of rate Rr1,

=

K* N*

=

6 12

1

= 2'

(12.159)

the same as the encoder rate R. The 64 codewords are divided into four subcodes 16 codewords each. The subcode corresponding to state Si, i = 0, 1, 2, 3, is formed by the set of 16 information sequences whose last two bits and us) result in the state Si. (Recall that for a feedforward encoder, the last v bits of the sequence determine the binary representation of the ending state.) Because an valid codewords must start and end in the same state, the 16 codewords in the subcode corresponding to state Si are by the 16 paths that connect the starting state S; to the state S;. The four subcodes corresponding to the tail-biting trellis of Figure 12.24(a) are shown in Table 12.5(a). Note that the subcodes for states S1. S2, and S3 are cosets of the subcode for state So (see Problem VVe see from Table 12.5(a) that the minimum (free) distance of this (12, 6) block code is dmin = 3, and the number of minimum-weight codewords is = 2 (boldface codewords in Table 12.5(a)). The lower minimum distance of this code with the (12, 4) terminated convolutional code is since the rate of the (12, 6) tail-biting code is 50% higher.

In 12.11 it is straightforward to find the subcodes in the tail-biting code corresponding to each state, as noted earlier, for feedforward encoders the ending state of a fixed-length input sequence is determined by the last v input bits. the starting state of each valid tail-biting codeword is also determined by the last v bits of the input sequence. This is not the case for feedback encoders, however, where the ending state depends on the entire input sequence. Hence, in order to form tail-biting convolutional codes using encoders with feedback, we must for each information sequence ilil, a starting state Si such that 1lil also ends in state Si.

Consider the (2. 1. 2) systematic feedback convolutional encoder with generator matrix (12.160) This systematic feedback encoder is equivalent to the foregoing nonsystematic feedforward encoder; that they both produce the same set of codewords. The 4-state tail-biting trellis corresponding to this feedback encoder is shown in Figure 12.24(b) for an information sequence au of length K* = h = 6, where we have again shown

588

Chapter 12

Optimum Decoding of Convolutional Codes 1/10

1----iS1

',o/

(a)

I:

I (b)

FIGURE 12.24: A 4-state tail-biting trellis with an information sequence of length 6 corresponding to (a) a feedforward encoder and (b) a feedback encoder.

Section 12. 7

Punctured and Tail-Biting Convolutional Codes

5:8!ll

TAB LE 12.5: The four subcodes corresponding to (a) the feedforward

tail-biting trellis of Figure 12.24( a) and (b) the feedback tail-biting trellis of Figure 12.24(b). S11

S1

000000000000 000000111011 000011101100 000011010111 001110110000 001110001011 001101011100 001101100111 111011000000 111011111011 111000101100 111000010111 110101110000 110101001011 110110011100 110110100111

101100000011 101100111000 101111101111 101111010100 100010110011

S11

0000000000 1101011100 0011010111 1110110000 1101100111 0011101100 0000111011 1110001011

I

JlilJ)ilJ)ilJ)JlilJ)ilJ)@Jl(j])(j])(j])

100001011111 100001100100 010111000011 010111111000 010100101111 010100010100 011001110011 011001001000 011010011111 011010100100

S2

.§'3

110000001110 110000110101 110011100010 110011011001 111110111110 111110000101 111101010010 111101101001 001011001110 001011110101

011100001101 011100110110 011111100001 011111011010 010010111101 010010000110 010001010001 010001101010 100111001101 100111110110 100100100001 100100011010 101001111101 101001000110 101010010001 101010101010

(j])l[])Jl(j])(j])ill)Jl(j])(j])l[])Jl(j])

001000011001 000101111110 000101000101 000110010010 000110101001

S1

S2

.§'3

Jll[])l[])(j])(j])Jll[])Jll[])(j])

l[])(j])Jll[])l[])(D(j])Jl(j])Jl

([DJll[])l[])Jl(j])l[])(j])l[])Jl

(j])Jl(j])Jl(j])l[])Jlil])(j])il])

il])(j])il])Jll[])Jlil])(j])Jl(j])

0110100100 0101110011 1011000011 0110011111 1011111000 1000101111

1100110101 0001101001 1111011001 1100001110 1111100010 0010111110

1010010001 1001000110 0111001101 0100011010 1001111101 0111110110 1010101010

(b)

IJ

the input labels as well as the output labels on each trellis branch. Note that the output labels are the same as for the trellis of Figure 12.24(a) corresponding to the equivalent feedforward encoder. but that the input labels are different. In particular. the ending state is no longer determined by the previous two input bits. To determine the valid codewords in a tail-biting code. we must find, for each information sequence uu, a starting state such that Ill ends in the same state after K* = 6 time units. Proceeding in a straightforward fashion, we see that the information sequence Ill = (000000). when starting in state So. ends in state So after six time units and generates the codeword v = (00. 00, 00. 00, 00, 00), which is thus a valid tail-biting codeword. Now. consider the information sequence 1 11Il = (100000). H So is the starting state, the ending state is S2, so the associated

590

Chapter 12

Optimum Decoding of Convolutional Codes

codeword is not a valid codeword in a tail-biting code. Trying the other possible starting states, 'Ne see that if ru1' starts in state S2, the ending state is So; if the starting state is S1, the ending state is S3; and the starting state S3 results in the ending state S1. In other words, for a block length of K* = 6, there is no valid tail-biting codeword corresponding to the information sequence fill 1 = (100000). In addition, we note that the information sequence 1lll = (000000), which generates the valid tail-biting codeword v = (00, 00, 00, 00, 00, 00) with So as its starting state, generates (different) valid tail-biting codewords with any of the other three states as its starting state. Hence, we find that some information sequences, like fill, have several corresponding valid tail-biting codewords, resulting in an ambiguous encoding rule, whereas other information sequences, like fill 1 , have no corresponding valid tail-biting codeword. In other words, in this case, no valid tail-biting code with information block length K* = 6 exists. Now, consider an information block length of K* = 5 by ignoring the last stage of the trellis in Figure 12.24(b ). In this case, each information sequence fill of length K* = 5 corresponds to exactly one valid tail-biting codeword, and a (10, 5) block (tail-biting convolutional) code exists. The code has four subcodes, each containing eight codewords, with one subcode corresponding to each of the four trellis states. The four subcodes are shown in Table 12.5(b ). We again note that the subcodes for states S1, S2, and S3 are cosets of the subcode for state So. We see from Table 12.5(b) that the minimum (free) distance of this (10. 5) code is dmin = 3, and the number of minimum-weight codewords is A3 = 5 (boldface codewords in Table 12.5(b )). Example 12.11 illustrates an interesting property of tail-biting convolutional codes. For feedforward encoders it is possible to form tail-biting codes of any length, and the determination of the proper starting and ending state for each codeword is straightforward. For feedback encoders, on the other hand, a length restriction exists; that is, it is not possible to form tail-biting codes of certain lengths. Also, the determination of the proper starting and ending state is more complex. (A method for finding the proper starting and ending state, based on a state variable representation of the encoding equations (see Example 11.14), is presented in [33].) Because the tail-biting codes generated by equivalent feedback and feedforward encoders are the same, the feedforward realization is normally preferred for the stated reasons, however, for turbo coding applications (see Chapter 16), feedback encoders are required. One of the interesting structural properties of tail-biting convolutional codes is the symmetry of the trellis representation. Unlike terminated convolutional codes, where the initial and final m levels of the trellis exhibit an asymmetry (see, e.g., Figure 12.21(b)), the trellis representation of a tail-biting code (see Figure 12.24) is perfectly symmetric. This symmetry allows one to view a tail-biting code using a circular trellis, in which one wraps around each final state and attaches it to its corresponding initial state, thus forming a circular structure. For example, the circular trellis representation of the tail-biting trellis of Figure 12.24 is shown in Figure 12.25. (It is this circular trellis structure that initially suggested the terminology tail-biting.) A tail-biting codeword can then be viewed as starting at an arbitrary state on the circular trellis and then vvrapping around it exactly once so as to reach the same ending state. (Note that choosing a different starting point on the

Section 12. 7

Punctured and Tail-Biting Convolutional Codes

Stn

FIGURE 12.25: A 4-state tail-biting circular treHis corresponding to an information sequence of length 6.

treHis simply corresponds to a circular shift of the set of codewords, which does not affect the properties of the associated tail-biting code.) The described for constructing a rate Rib block (tail-biting convocode from a rate R = Rrb convolutional encoder suggests that it should be possible to determine the generator matrix of the block code from the generator matrix of the associated convolutional encoder. We illustrate the procedure for feedforward convolutional encoders. (For feedback encoders, the procedure is more complex.) First, we consider the semi-infinite (time-domain) generator matrix Ge of the unterminated convolutional code given in (11.22) as

Ge~

l

Go

G1 Go

Gm (Gm-1 Gm-2

G2 G1 Go

Gm (Gm-1

Gm

l

(12.161)

Then, we form the finite K* >< N matrix (G~ corresponding to the block (terminated convolutional) code with K* information bits. G;, is given by Go

G~=

G1 Go

Gm G1

Gm Go

G1 Go

(12.162)

Gm (Gm-1

Gm

Go

G1

Gm

592

Chapter 12

Optimum Decoding of Convolutional Codes

Finally, we obtain the K* x N* matrix Gbb corresponding to the block (tail-biting) convolutional code by adding the last run columns of G~ to the first run columns and then deleting the last nm columns (recall that N = N* +nm). Thus, Gbb is given by

Go

Gib_ b -

G1 Go

Gm

G1

Gm

Go Gm

G111-1

Gm

G1

G2

G1 Go

Gm

G1

Gm

Gm-1

(12.163)

G1 Go

Comparing (12.162) with (12.163), we see that the effect of tail-biting is to wrap to its first nm columns. The 2K* codewords in the around the last nm columns of block (tail-biting) convolutional code then correspond to the row space of Gt It is interesting to note that the procedure is reversible; that is, if a rate Rib block code exists with generator matrix Gin the form of (12.163), G can be unwrapped to form a convolutional encoder with rate R = R 11,.

Gi,

IEXAMIPlE 12.11

Applying the wrap around procedure to the generator matrix of the feedforward convolutional encoder in this example for an information block size of K* = 6, we obtain the generator matrix

,rib -

\l]Jb -

11 10 11 00 00 00 00 11 10 11 00 00 00 00 11 10 11 00 00 00 00 11 10 11 11 00 00 00 11 10 10 11 00 00 00 11

(12.164)

It is easy to verify (see Problem 12.38) that the row space of G{' in (12.164) is the

(12, 6) tail-biting code of Table 12.5( a). In Table 12.6 we list, for constraint lengths v = 1, 2, · · · , 6 and rates Rib = K*/ N* = 1/3, 1/2, and 2/3, the block (tail-biting convolutional) codes with the largest minimum distance dmin and the smallest number of nearest neighbors Ac1mi,,. In each case the best encoder is listed for all information block lengths satisfying h = K*/ k::: 2. (Recall that for rates Rib = 1/3 and 1/2, K* =hand v = m, whereas for rate Rib = 2/3, K* = 2h and v = 2m.)

TABLE 12.6: Generator matrices for and rates (a) R11, = 1/3, (b) Rr1, = ~(QI)

2 3

1 1 1 1 1

4

convolutional codes with constraint = 2/3.

z

]_

l/

K*

5 6 7 8 9

and (c)

1);(1)

1 1 3 3 3

10 11

12 13 14 15 16 17 18 19 20 21 22 23 Adapted from [13].

~(Z)

imiJm

Admi11

3 3 3 3 3

4

3 3 1 6 6 K*

4 4 5 5

!l;(ll)

@CTI.) 8

1 1 1 1 5 5 5 5 5 5 5 5

3 3 3 3 3 3 3 3 7 7 7 7

~

(Z)

7 7 7 7 7 7 7 7 7 7 7 7

v

=

1, 2, ... 6

3\ dmh1

A1 d111;11

!l;(QD)

~(1)

~(Z)

4 4 6 6 6 7 7 7 8 8 8 8

3 3 12 10 1

15 15 1 1 15 15 11

11 11

3 7 3 3 3 7 13 13 13

15 15

13 13

7 17 13 7 7 17 17 17 17 17 17

8 8 9 45 33 27 26 2K*

lmffuo

Adm;,,

4 4 6 7 8 8 8 8 9 10 10

3 3 12 15 45 28 13 9 20 44 36 3K*

V, 11) I'>

r:·

o· ::, !" -..] ,:J

C

::, Fl

2" ;;; Q_

ru

::, Q_

-I

ru

' 00

~'E

::, u:i

()

0

::,

<

0

C ~,

a·

::,

ru

()

0

Q_

11)

"' Y'il \\;')

w

TABLE 12.6:

K*

gCO)

il1l

g<2)

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

31 31 25 1 31 31 5 31 25 25 35 35 25 31 25 25 25 25

35 35 3 23 3 25 27 35

23 23 7 13 7 7 37 23 17 17 27 37 37 37 37 37 37 37

27

27 23 33 33 25 33 33 33 33

.I:>,

6

5

4!

JI

Y'i I.©

(continued)

dmin

4 4 6 7 8 8 8 10 10 10 10

12 12 11 12 12 12 12

n

Aamin

g(O)

g(l)

g<2)

dmin

Aamin

g(O)

g(l)

g<2)

dmin

Aamin

3 3 12 15 45 21 1 99 46 11 6 182 126 15 80 85 93 95

15 21 51 61 31 31 71 15 55 31 71 71 35 71 51 45 71

3 73 43 75 43 73 3 3 23 15 75 55 53 65 73 73 65

57 57 57 73 67 57 67 57 27 73 13 73 57 73 57 67 57

4 4 6 7 8 8 8 10 10 11 12 12 12 13 13 13 13

3 3 12 15 45 21 1 99 21 55 200 39 14 75 32 34 18

61 61 75 61 61 111 11 61 145 121 65 165 43 45 151 75 131 151 135 165 135

173 173 3 133 173 165 13 371 63 105 147 173 127 133 135 153 161 175 123 133 123

67 67 127 117 67 73 137 67 67 177 177 17 117 137

4 4 6 7 8 8 8 10 10 11 12 12 13 14 14 14 15 15 15 15 15

3 3 12 15 45 21 1 99 21 33 178 39 126 390 168 51 240 114 60 63 66

K*

SK*

77

117 167 153 157 117 157

3K*

(a)

:::, Ill

"O

,-;,

(1)

"' ..... N

0

"O

!:!".

3 C 3 0

(1) ,..,

0

c.. ~

lO

0--+,

n 0

~

< 0 C

,-;,

o· ~

Ill

n 0

c.. ro

"'

TABLE 12.6:

1i

\11

K*

2 3 4 5 6 7 8 9 10 11

12 13 14 15 16 17

1 1 1

3 3 3

/\ldmin

2 3 3

1 4

4 l(*

1);(11) 1/;(ll)

3 1 1 3 3 5 5 5 5 5

7 5 7 7 7 7 7 7 7 7

41

3

2

~(II) 1/;(Jl)

/Jld111i11

2 3 4 4 4 4 4 5 5 5

1 4

14 10 9

7 2 18 12 11 T.7*

A

~(II) ~(ll)

15 3 15 17 1 13 5 13 15 3 15 17 15 15 7 15 7 15 17 15 17 15 17 ,;

.)

A\dmi11

2 3

1

11

37

4

31 15 31 31 11 31 25 15 35 23 35 31 35 35 31

6

4

14

4

10

4

6

6 7 24 102 90 44

6 6

48 13

4 5 6 6

[(*

@

§

Vl (l)

g(ill) ~(Jl)

23 3 3 37 3 13 23 37 27 37 33 23 23 33

Aam1n

2 3 4 4

1 4 14

4

6 7 24 102 90 44 30

4 5 6 6 6 6 6 7 7 7 7

10

65 61 75 65 5 13 35 72 11 75 15 64 15 34 65

73 57 33 57 7 7

67 3

7 73 73 67 57 57 37

2 3 4

4 4

4 5 6 6 7 6 7 7 8 8 8

1 4 14 10 67 6 41133 7 161 67

24 102 40 176 20 117 56 450 348 170

65 153 135 177 155 157 111 123 141 67 111 143 153 37 141127 135 17 165 63

:::.

o·

::;

2 3 4 4

1 4 14 10

-.,

4

6 7 24 102 40 176 759 117 546 450 238 153

C ::;

4 5 6 6 7 8 7 8 8 8

8

N '-I

Fl

ct ro

CL Ill

::;

Q_

-I

Ill

ro ~: ::; lQ

('I

0

::;

<

0

C

~-

0

::;

Ill ('I

0

Q_

(I)

"' ll,j'j

I.© ll,j'j

IJ'i

w

er, ()

::r

Ill

-0 ,-,.

TABLE 12.6:

..., ([)

(continued)

6i 4 1 2 3 5 . A dmin g(ill g(ll d mm . A dmin g(ill g(ll d mm . A dmi11 g
:r

18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

35 23 35 23

7 7

54 38

75 35 2K* 65 65 51 55 65

63 57 57 57 77 57 57

8 8 8

8 8 8 8

117 76 60 42 44 46 24

135 113 55 163 131 117 161155 135 163 135 163 151163 K* 155 117 135 163 155 117 135 163 155 117 135 163 155 117 135 163 155 117

8 8 9 9 10 10

9 10

10 10

10 10 10 10 10 10

0

72

-0

19 280 189 1364 989 96 575 494 378 364 319 390 341 384 363

3 C 3

llK*

(b)

N

!:±.

0

([) ...,

0

Q.

::::s lQ

0

-+,

()

0 ::::s

< 0

C .-..

a· ::::s

Ill ()

0

Q. ([)

"'

TABLE 12.6:

'I!

2

Ill

K*

11n(Z)

11n(l)

11n(ill)

4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50

7 7 7 7 7 7

3 3 3 3 5 5

1 1 1 1 3 3

dmin

Adm in

2 2 3 3 3 3

3 3 16 15 8 7 K*/2

11n(Z)

11n (1)

11n(ill)

25 5 25 7 35 31 27

7 23 7 15 7 7 25 31 31 23 23 37 25 35 31

3 27 3 1 3 3 33 15 15 27 27 35 33 23 27

17

17 5 5 23 27 27 25

dmiu1

2

2 3 4

4 4 4 4 5 5 5 5 5 5 5

d,,,;,,

3 3 16 105 78 56 26 18 172 132 96 52 42 45 32 [(*

(c)

Jln(Z)

nnm

127 173 127 133 161 155 177 57 145 105 51 165 153 137 163 165 135 77 137 121 121 121 121 121

45 45 45 117 27 107 35 41 67 177 43 67 21 31 145 133 103 113 163 123 123 123 123 123

({ii nn(ill)

d,11iTJI

Ad,,,;,.

123

2

71

2

123 141 135

3 4 4

1

4

33 173 63 121 177 163 135 151 165 137 131 121 51 147 147 147 147 147

4

3 3 16 105 78 56 26 18 162 1276 1036 767 329 195 104 51 36 38 20 528 418 414 432 425

4

5 6 6 6 6 6 6 6 6 6 6 7 7 7 7 7

Vl

(i)

&.

0

::) ~

N

'1

'1 C ::)

Fl

F0

C "9

(i) Q.

w ::)

Q_

-I

w

' 00

;:;:

::)

\.Q

n 0

::)

< 0

C

~'

6"

::)

w

n

0 11 (i)

"' ll!fij ~ ~

598

Chapter 12

Optimum Decoding of Convolutional Codes

We see from Table 12.6 that, in general, for a given rate Rib and constraint length v, the minimum distance dmin of the best block (tail-biting convolutional) code increases, or the number of nearest neighbors decreases, as the information block length K* increases. Once K* reaches a certain value, though, the minimum distance dmin of the best block (tail-biting convolutional) code is limited by the free distance dtree of the best terminated convolutional code with constraint length v, and no further increase in dmin is possible; however, the number of nearest neighbors Ad111 ;,, continues to grow linearly with K*. Once this limit is reached, the generator sequences g
PROBLEMS

l!.2.1 Draw the trellis diagram for the (3, 2, 2) encoder in Example 11.2 and an information sequence of length h = 3 blocks. Find the codeword corresponding to the information sequence Ill!= (11, 01, 10). Compare the result with (11.16) in Example 11.2. 12.2 Show that the path V that maximizes L!o 1 log p (r1 IV/) also maximizes L!o1 c2[log P(n IV/)+ c1J, where CI is any real number and c2 is any positive real number. 12.3 Find the integer metric table for the DMC of Figure 12.3 when c1 = 1 and c2 = 10. Use the Viterbi algorithm to decode the received sequence ir of Example 12.1 with this integer metric table and the trellis diagram of Figure 12.1. Compare your answer with the result of Example 12.1. 12.41 Consider a binary-input, 8-ary output DMC with transition probabilities P(,"/ IV/) given by the following table: r(j) I v(J)

01

02

03

04

14

Jl3

12

11

0.434

0.197

0.167

0.111

0.058

0.023

0.008

0.002

0.002

0.008

0.023

0.058

0.111

0.167

0.197

0.434

__.-__L_

0

Find the metric table and an integer metric table for this channel. 12.5 Consider the (2, 1, 3) encoder of Figure 11.1 with

Problems

59)9)

i11o Draw the trellis for an information sequence of length h = 4. lbio Assume a codeword is transmitted over the DMC of Problem 12.4. Use the

l12Ai

Jl2o'7

Uoa~

Uil!I Uoli®

Viterbi algorithm to decode the received sequence fl'.= (1211, h01, 0301, 0113, 1202. 0311, 0302). The DMC of Problem 12.4 is converted to a BSC by combining the soft-decision outputs 01 , 02 . and 04 into a single hard-decision output 0, and the soft-decision outputs 11 , 12 • 13, and 14 into a single hard-decision output 1. A codeword from the code of Problem 12.5 is transmitted over this channel. Use the Viterbi algorithm to decode the hard-decision version of the received sequence in Problem 12.5 and compare the result with Problem 12.5. A codeword from the code of Problem 12.5 is transmitted over a continuousoutput AWGN channel. Use the Viterbi algorithm to decode the (normalized by ./Es) received sequence n· = (+ 1.72, +0.93, +2.34. -3.42, -0.14, -2.84, -1.92, +0.23, +0.78, -0.63, -0.05, +2.95, -0.11, -0.55). Consider a binary-input, continuous-output A WGN channel with signal-to-noise ratio Es/No= 0 dB. talo Sketch the conditional of the (normalized by ~ ) received signal r1 given the transmitted bits v1 = ±1. tOJo Convert this channel into a binary-input, 4-ary output symmetric DMC by placing quantization thresholds at the values n = -1, 0, and+ 1, and compute the transition probabilities for the resulting DMC. {Co Find the metric table and an integer metric table for this DMC. cul Repeat parts (b) and (c) using quantization thresholds ri = -2, 0, and + 2. Show that (12.21) is an upper bound on Pd ford even. Consider the (2, 1, 3) encoder of Problem 12.5. Evaluate the upper bounds on event-error probability (12.25) and bit-error probability (12.29) for a BSC with transition probability talo /J = 0.1, Jbio JJ = O.Ql.

(Hint: Use the VI/EFs derived for this encoder in Example 11.12.) UoU Repeat Problem 12.10 using the approximate expressions for P(E) and P1,(E) given by (12.26) and (12.30). UoU Consider the (3, 1, 2) encoder of (12.1). Plot the approximate expression (12.36) for bit-error probability Pb(E) on a BSC as a function of E1,/ No in decibels. Also plot on the same set of axes the approximate expression (12.37) for P1,(E) without coding. The coding gain (in decibels) is defined as the difference between the Eb/ No ratio needed to achieve a given bit-error probability with coding and without coding. Plot the coding gain as a function of P1,(E). Find the value of E1,/ No for which the coding gain is O dB, that is, the coding threshold. Jl'.tB Repeat Problem 12.12 for an A WGN channel with unquantized demodulator outputs, that a continuous-output A WGN channel, using the approximate expression for P1,(E) given in (12.46). Uol4l Consider using the (3, 1, 2) encoder of (12.1) on the DMC of Problem 12.4. Calculate an approximate value for the bit-error probability P1,(E) based on the bound of (12.39b ). Now, convert the DMC to a BSC, as described in Problem 12.6, compute an approximate value for P1,(E) on this BSC using (12.29), and compare the two results. l2ol5 Prove that the rate R = 1/2 quick-look-in encoders defined by (12.58) are noncatastrophic.

600

Chapter 12

Optimum Decoding of Convolutional Codes

12.16 Consider the following two nonsystematic feedforward encoders: (1) the encoder for the (2, 1, 7) optimum code listed in Table 12.l(c) and (2) the encoder for the (2, 1, 7) quick-look-in code listed in Table 12.2. For each of these codes find a. the soft-decision asymptotic coding gain y; b. the approximate event-error probability on a BSC with p = 10- 2 ; c. the approximate bit-error probability on a BSC with p = 10- 2 ; d. the error probability amplification factor A. 12.17 Using trial-and-error methods, construct a (2, 1, 7) systematic feedforward encoder with maximum dtree· Repeat Problem 12.16 for this code. 12.18 Consider the (15,7) and (31,16) cyclic BCH codes. For each of these codes find a. the polynomial generator matrix and a lower bound on dtree for the rate R = 1/2 convolutional code derived from the cyclic code using Construction 12.1; b. the polynomial generator matrix and a lower bound on dtree for the rate R = 1/4 convolutional code derived from the cyclic code using Construction 12.2. (Hint: d17 is at least one more than the maximum number of consecutive powers of a that are roots of h.(X).) 12.19 Consider the (2, 1, 1) systematic feedforward encoder with G(D) = [1 1 + D]. a. For a continuous-output A WGN channel and a truncated Viterbi decoder with path memory r = 2, decode the received sequence r = (+ 1.5339, +0.6390, -0.6747, -3.0183, +1.5096, +0.7664, -0.4019, +0.3185, +2.7121, -0.7304, +1.4169, -2.0341, +0.8971, -0.3951, +1.6254, -1.1768, +2.6954, -1.0575) corresponding to an information sequence of length h = 8. Assume that at each level the survivor with the best metric is selected and that the information bit r time units back on this path is decoded. lbi. Repeat (a) for a truncated Viterbi decoder with path memory r = 4. c. Repeat (a) for a Viterbi decoder without truncation. d. Are the final decoded paths the same in all cases ? Explain. 12.20 Consider the (3, 1, 2) encoder of Problem 11.19. a. Find A 1 (W, X, L), A2(W, X, L), and A 3 (W, X, L). b. Find Tmin· c. Find d(r) and Ad(r) for T = 0, 1, 2, · · · , Tmin· d. Find an expression for limr---+oo d(r). 12.21 A codeword from the trellis diagram of Figure 12.1 is transmitted over a BSC. To determine correct symbol synchronization, each of the three 21-bit subsequences of the sequence ll'=

01110011001011001000111

must be decoded, where the two extra bits in ir are assumed to be part of a preceding and/or a succeeding codeword. Decode each of these subsequences and determine which one is most likely to be the correctly synchronized received sequence. 12.22 Consider the binary-input, continuous-output A WGN channel of Problem 12.8. a. Using the optimality condition of (12.84), calculate quantization thresholds for DMCs with Q = 2, 4, and 8 output symbols. Compare the thresholds obtained for Q = 4 with the values used in Problem 12.8. b. Find the value of the Bhattacharyya parameter Do for each of these channels and for a continuous-output A WGN channel. c. Fixing the signal energy ,/Es = 1 and allowing the channel SNR E 5 /Noto vary, determine the increase in the SNR required for each of the DMCs to achieve

Problems

51[])41

the same value of Do as the continuous-output channel. This SNR difference is called the decibel loss associated with receiver quantization. (Note: Changing the SNR also changes the quantization thresholds.) (Hint: You will need to write a computer program to solve this problem.) JL::tZ3 Verify that the two expressions given in (12.89) for the modified metric used in the SOVA algorithm are equivalent. UJ',4l Define L(r) = ln A(r) as the log-likelihood ratio, or L-value, of a received symbol r at the output of an unquantized binary input channel. Show that the L-value of an A WGN channel with binary inputs ±,/Es and SNR Es/ No is given by L(r)

=

(4,/Es/No)r.

JLZJl,§ Verify that the expressions given in (12.98) are correct, and find the constant c. UJ',((ii Consider the encoder, channel, and received sequence of Problem 12.19. !llo Use the SOVA with full memory to produce a soft output value for each decoded information bit. lblo Repeat (a) for the SOVA with path memory r = 4. UZl Derive the expression for the backward metric given in (12.117). Lu{111 l/2

JL::tW Verify the derivation of (12.123) and show that A 1 = e - 1+,-L""' 11 is independent of the actual value of u 1. UoZ91 Derive the expressions for the max* (x, y) and max* (x. y. z.) functions given in (12.127) and (12.131), respectively. Uo31(]) Consider the encoder and received sequence of Problem 12.19. mo For an AWGN channel with Es/ No= 1/2 (-3 dB), use the log-MAP version of the BCJR algorithm to produce a soft output value for each decoded information bit, Find the decoded information sequence 1ll!. lblo Repeat (a) using the Max-log-MAP algorithm. UoJJL Repeat Problem 12.5 using the probability-domain version of the BCJR algorithm. JL:t2l2 Show that using the normalized forward and backward metrics A1 (s) and Bt (s') instead of ot1(s) and /31(s'), respectively, to evaluate the joint pdf's in (12.115) has no effect on the APPL-values computed using (12.111). JL232l Verify all the computations leading to the determination of the final APPL-values in Example 12.9. Uo34l Repeat Example 12.9 for the case when the DMC is converted to a BSC, as described in Problem 12.6, and the received sequence ir is replaced by its hard-decision version. Compare the final APP L-values in the two cases. UoJ§ Consider an 8-state rate R = 1/2 mother code with generator matrix


= [1 + D + D 3 1 + D 2 + D 3 ].

Find puncturing matrices JP for the rate R = 2/3 and R = 3/4 punctured codes that give the best free distances. Compare your results with the free distances obtained using the 8-state mother code in Table 12.4. Uo2l6 Prove that the subcocle corresponding to any nonzero state S;, i i= 0, in a tailbiting convolutional code is a coset of the subcode corresponding to the all-zero state So. JL237 For the rate R = 1/2 feedback encoder tail-biting trellis in Figure 12.24(b), determine the parameters d 111 ;11 and Ac1,,,,,, for information block lengths K* = 7, 8, and 9. Is it possible to form a tail-biting code in each of these cases? Uo3~ Verify that the row space of the tail-biting generator matrix in (12.164) is identical to the tail-biting code of Table 12.5(a).

602

Chapter 12

Optimum Decoding of Convolutional Codes

12.39 Consider the rate R = 4/8, constraint length v = 4 feedforward convolutional encoder with generator matrix

G(D) _ -

l+D

O

1

0 D

1+D D D

1

I

0

l+D 0

0 1 0 l+D

l+D D

0 D

1 l+D D D

1 1 l+D D

1 0 1 l+D

a. Draw the controller canonical form encoder diagram. b. Draw an h = 3 (K* = 12), 16-state tail-biting trellis for this encoder. c. Find the tail-biting generator matrix Gbb for the resulting (24, 12) tail-biting

code. dmin = 8 and is equivalent to the (24, 12) extended Golay code. (Note: The convolutional code generated by G(D) is called the Golay convolutional code.)

d. Show that this code has

BIBLIOGRAPHY

1. A. J. Viterbi, "Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm," IEEE Trans. Inform. Theory, ff13: 260-69, April 1967.

2. J. K. Omura, "On the Viterbi Decoding Algorithm," IEEE Trans. Inform. Theory, IT-15: 177-79, January 1969. 3. G.D. Forney, Jr., "The Viterbi Algorithm," Proc. IEEE, 61: 268-78, March 1973.

4. G.D. Forney, Jr., "Convolutional Codes H: Maximum Likelihood Decoding," Inform. Control, 25: 222-66, July 1974. 5. L. R. Bahl, J. Cocke, F. Jelinek, and J. Raviv, "Optimal Decoding of Linear Codes for Minimizing Symbol Error Rate," IEEE Trans. Inform. Theory, IT20: 284-87, March 1974.

6. P. L. McAdam, L. R. Welch, and C. L. Weber, "M.A.P. Bit Decoding of Convolutional Codes," in Book of Abstracts IEEE International Symposium on Information Theory, p. 91, Asilomar, Calif., February 1972. 7. L. N. Lee, "Real-Time Minimal-Bit-Error Probability Decoding of Convolutional Codes," IEEE Trans. Commun., COM-22: 146-51, February 1974.

8. C.R. P. Hartmann and L. D. Rudolph, "An Optimum Symbol-by-Symbol Decoding Rule for Linear Codes," IEEE Trans. Inform. Theory, IT22: 514-17, September 1976. 9. J. Hagenauer and P. Hoeher, "A Viterbi Decoding Algorithm with SoftDecision Outputs and Its Applications," Proc. IEEE Global Conference on Communications, pp. 1680-86, Dallas, Tex., November 1989.

Bibliography

(51{))3l

liilllo A. J. Viterbi, "Convolutional Codes and Their Performance in Communication "IEEE Trans. Commun. Technol., COM-19: 751-72, October 1971.

H, L. 'Van de Meeberg, "A Tightened Upper Bound on the Error Probability of Binary Convolutiona] Codes with Viterbi Decoding," IEEE Trans. ff-20: 389-91, May 1974. U, A. J. Viterbi and J. K. Omura,

of

Communication and

McGraw-Hill, New York, 1979. 12lo R. Johannesson and K. S. Zigangirov, Fundamentals IEEE Press, N.J., 1999. li4L J.M. Wozencraft and t M. Jacobs, Principles of Communication Engineering. John Wiley, New York, 1965.

C. D. W.D. and F. for Computing Free Distance," IEEE Trans. 18: 437-39, May 1972.

"An Ef:ficien t Theory, IT-

Il(fii, K. J. Larsen, "Comments on 'An Efficient Algorithm for Computing Free Distance'," IEEE Trans. Inform. Theory, IT-19: 577-79, July 1973. li'!, J. J.

D. J. I-Iwang, and M. C. "Some Extended Results on the Search for Good Convolutional Codes," IEEE Trans. Inform. IT43: 1682-97, 1997.

Ut J. L. Massey and D. J. Costello, Jr., "Nonsystematic Convolutional Codes

for Decoding in Space Applications," IEEE Trans. Commun. Technol., COM-19: 806-B, October 1971. Ili!J), J. L. Massey, D. J. Costello, and J. Justesen, "Polynomial Weights and Code "IEEE Trans. Inform. Theory, IT-19: 101-10, January 1973. 2iJ]), J. Justesen, "New Convolutional Code Constructions and a Class of Asymptotically Good Time-Varying Codes," IEEE Trans. Inform. Theory, IT19: 220-25, March 1973. M. Tanner, "Convolutional Codes from Quasi-Cyclic Codes: A Link between the Theories of Block and Convolutional Codes," Technical Report, Computer Science Research Laboratory, UC Santa Cruz, November 1987.

2]., R.

22, 0. M. Collins, "The Subtleties and Intricacies of Building a Constraint Length 15 Convolutional Decoder," IEEE Trans. Commun., COM-40: 1810-19, December 1992.

2:t F. Hemmati and D. J. Costello,

"Truncation Error Probability in Viterbi Decoding," IEEE Trans. Commun. COM-25: 530-32, May 1977.

24\, F. Hemmati and D. J. Costello, Jr., "Asymptotically Catastrophic Convolutional Codes," IEEE Trans. Inform. Theory, H-26: 298-304, May 1980.

604

Chapter 12

Optimum Decoding of Convolutional Codes

25, M. P. C. Fossorier and S. Lin, "Differential Trellis Decoding of Convolutional Codes," IEEE Trans. Inform. Theory, IT-46, 1046-53, May 2000. 26. W. E. Ryan, "Concatenated Convolutional Codes and Iterative Decoding," in Wiley Encyclopedia of Telecommunications, edited by J. G. Proakis, John Wiley, New York, 2002. 27, J.B. Cain, G. C. Clark, and J.M. Geist, "Punctured Convolutional Codes of Rate (n -1)/n and Simplified Maximum Likelihood Decoding," IEEE Trans. Inform. Theory, IT-25: 97-100, January 1979. 28, G. Solomon and H. C. A. van Tilborg, "A Connection between Block and Convolutional Codes," SIAM J. Appl. Math., 37(2): 358-69, October 1979. 29, H. H. Ma and J. K. Wolf, "On Tail-Biting Convolutional Codes," IEEE Trans. Commun., COM-34: 104-11, February 1986. 30, L. H. C. Lee, Convolutional Coding: Fundamentals and Applications, Artech House, Boston, Mass., 1997. 31. J. Hagenauer, "Rate-Compatible Punctured Convolutional Codes (RCPC Codes) and Their Applications," IEEE Trans. Commun., COM-36: 389-400, April 1988. 32, J. B. Anderson and M. Hladik, "Tailbiting MAP Decoders," IEEE J. Selected Areas Commun., SAC-16(2): 297-302, February 1998. 33, C. Weiss, C. Bettstetter, and S. Riedel, "Code Construction and Decoding of Parallel Concatenated Tail-Biting codes," IEEE Trans. Information Theory, IT-47: 366-86, January 2001.

§ [lJJ ~Q)@ ~Q) it o~111rnJJ uuu [Q) ®[@ (QJ oGu (£J @

if

(@ uu w@ ~ [lJJ it o@ ITTJ @ ~ (@ (QJ ® ~

with Viterbi and BCJR decoding of convolutional codes is deco•:J!mtg methods, the low error the random bound of (1.14) are not achievable in This is because the decoding effort is fixed and at rates dose to grows with constraint length, and thus short constraint length codes can be used. In this we introduce two decoding sequential and majority-logic or threshold decoding, that can be used to decode constraint length convolutional codes. We begin by that the fixed amount of the Viterbi and BCJR is not needed, particularly when the noise is For assume that an encoded sequence of length N is transmitted without error over a BSC; that ll' = w. The Viterbi and BCJR algorithms still on the order of 2v computations per decoded information block, all of which is wasted effort in this case. In other it is sometimes desirable to have a procedure whose effort is to the noise level. Sequential decoding is such a procedure. As we shall see, the decoding effort of sequential uv-.v,.,w,;c is independent of v, so that large constraint lengths can be used. The number of computations per decoded information block is a random variable, however. This nature of sequential decoding requires the use of buffers to store data. Although most encoded sequences are decoded very quickly, some undergo long which can lead to buffer overflow, causing information to be lost or erased. This problem limits the application of sequential to rates less than An algebraic can also be taken to the decoding of convolutional codes. fo particular, majority-logic decoding, first introduced in Chapter 8 for block codes, is applicable to convolutional codes. (In the soft-decision case, we use the more general term threshold decoding.) Majority-logic decoding differs from BCJR, and decoding in that the final decision made on a given information block is based on only (m + 1) received blocks rather than on the entire received sequence. Also, the codes used with majority-logic decoding must be orthogonalizable, a constraint that yields codes with poor distance properties. The result is inferior performance when compared with Viterbi, BCJR, or sequential decoding, but the implementation of the decoder is much simpler. sequential decoding was introduced by Wozencraft [1, 2] in 1957 as the first practical decoding method for convolutional codes. In 1963 Fano [3] introduced a new version of sequential decoding, subsequently referred to as the Fano algorithm. A few years later another version of sequential decoding, called the ZJ or stack algorithm, was discovered independently by Zigangirov [4] in 1966

606

Chapter 13

Suboptimum Decoding of Convolutional Codes

and Jelinek [5] in 1969. Majority-logic or threshold decoding of convolutional codes, both the hard-decision and soft-decision (APP) versions, was introduced by Massey [6] in 1963. 13.1

THIE ZJ {STACK) SEQIUENTiAl DIECODING AlGORiTHM

In discussing sequential decoding, it is convenient to represent the 2kh codewords of length N = n(h + m) for a rate R = k/n encoder with memory order m and an information sequence of length K* = kh as paths through a code tree containing h + m + 1 time units or levels. The code tree is just an expanded version of the trellis

diagram in which every path is distinct from every other path. The code tree for the (3, 1, 2) feedforward encoder with G(D)

=

[1

+D

1 + D2 1+ D

+ D2]

(13.1)

is shown in Figure 13.1 for an information sequence of length h = 5. The trellis diagram for this encoder was shown in Figure 12.1. The h + m + 1 tree levels are labeled from Oto h + m in Figure 13.1. The leftmost node in the tree is called the origin node and represents the starting state So of the encoder. For a rate R = k/n encoder, there are i' branches leaving each node in the first h levels of the tree. This region is called the dividing part of the tree. In Figure 13.1 the upper branch leaving each node in the dividing part of the tree represents the input bit u 1 = 1, and the lower branch represents u1 = 0. After h time units there is only one branch leaving each node, which represents the inputs uz (= 0, for feedforward encoders), for l = h, h + 1, · · · , h +m -1, and corresponds to the encoder's return to the all-zero state So. Hence, it is called the tail part of the tree, and the 2" 11 rightmost nodes are called terminal nodes. Each branch is labeled with the n outputs v1 corresponding to the input sequence, and each of the 2" 11 codewords of length N is represented by a totally distinct path through the tree. For example, the codeword corresponding to the input sequence 1!ll = (1 1 1 0 1 0 0) is shown boldfaced in Figure 13.1. It is important to realize that the code tree contains exactly the same information about the code as the trellis diagram or state diagram. As we shall see, however, the tree is better suited to explaining the operation of a sequential decoder. There are a variety of tree-searching algorithms that fall under the general heading of sequential decoding. In this chapter we discuss the two most common of these, the ZJ or stack algorithm and the Fano algorithm, in considerable detail. The purpose of a sequential decoding algorithm is to search through the nodes of the code tree in an efficient way, that is, without having to examine too many nodes, in an attempt to find the maximum likelihood path. Each node examined represents a path through part of the tree. Whether a particular path is likely to be part of the maximum likelihood path depends on the metric value associated with that path. The metric is a measure of the "closeness" of a path to the received sequence. For a binary-input, Q-ary output DMC, the metrics in the Viterbi algorithm are given by the log-likelihood function of (12.3). This function is the optimum metric for the Viterbi algorithm, since the paths being compared at any decoding step are all of the same length. In sequential decoding, however, the set of paths that have been examined after any decoding step are generally of many different lengths. If the log-likelihood metric is used for these paths, a distorted picture of the "closeness" of paths to the received sequence results.

Section 13.1

Sequential Decoding Algorithm

The ZJ

001

001

?

110

110 8

011

~_0_11_---n-_oo_o_---0

100

101

011

~~

~

010 ~--~

100

0

101

110 ~ -0-1-1 ------
111

011 010

000 110

Q

011

000 011

0

000

111 000

101 011 0 0 :)-.-00_0_____, ,--00_0_____,

001

-

-~-,

010 100 101

j

r j

110 011

111

I

000

001

?

101

?

000

111

001 l

T

010 110

000

T

111 100

011

l

T

101

J!

000 111 000

.1

000

-- r/ ~) Ill 0

Systematic feedforward encoder

'

J

r/" - l) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _--;:>1

+

s/f'l s/" - I)

FIGURE 13.22: Syndrome forming circuit for an (n, k, m) systematic feedforward encoder.

656

Chapter 13

Suboptimum Decoding of Convolutional Codes

triangles, one corresponding to each generator polynomial. The general structure of the parity triangles used to form orthogonal parity checks is as follows: (k) gl.O

(k)

so

(kl

(k)

gl.m

Sm

(k+l)

(k+l) gl.0

(k+l)

(k+l) gl.111 .

so

S,n

(11-l)

so

(11-l) Sm

(11-l) gl.0

(11-l) gl.111 .

(k)

gl.0

(k)

(k)

g2.0

gk.0

(k) g2.m

(k) g2.0

(k+l)

(k)

gk.O

(k+l)

g2.0

.

(k+l)

(k+ll

gk 111

gk.O

(11-l)

gk.111

em

(k+l)

(1)

em

ea

+

(k-l)

gk.0

(11-l) g2.0 .

(k)

(Ol

em

(1)

(11-l)

(11-l)

(kl

ea

ea

gk.O

g2.0

(11-l) (11-l) gl.O g2.m .

(k)

gk.111 (k+l)

g2.0

(k+l) (k+l) g2.m · gl.O

(0)

ea

.

(11-l)

gk.O

(k+l)

em

(11-l)

eo

eo

(k-l) e,11

em

(11-l)

(13.59)

The first set of m + 1 rows corresponds to syndrome sequence §(k), the second set of m + 1 rows corresponds to syndrome sequence §(k+l), and so on. Syndrome bits, or sums of syndrome bits, are then used to form orthogonal check-sums on the information error bits e2), · · · , eg'- l). H at least J orthogonal check-sums can be formed on each of these information error bits, then fML = LJ /2J is the majority-logic error-correcting capability of the code; that is, any pattern of fML or fewer errors within the nE error bits checked by the k sets of orthogonal check-sums will be corrected by the majority-logic decoding rule. A feedback majority-logic decoder for a tML -error-correcting (n, k, m) systematic code operates as follows:

e6°l,

Step 1. The first (m + l)(n - k) syndrome bits are calculated. Step 2o A set of J orthogonal check-sums are formed on each of the k information error bits from the syndrome bits calculated in step 1. Step 3. Each set of J check-sums is fed into a majority gate that produces an output of 1 if and only if more than half of its inputs are l's. If its output is O (e6j) = 0), r6j) is assumed to be correct. If the output of the jth gate is 1 (e6j) = 1), rcii) is assumed to be incorrect, and hence it must be corrected. The corrections are performed by adding the output of each majority gate to the corresponding received bit. The output of each majority gate is also fed back and subtracted from each syndrome it affects. (It is not necessary to modify the time unit 0 syndrome bits.) = r(j) + 0 ' · = 0 • 1• ... • Step 41. The estimated information bits ,ii+l) 0 0 k - 1, are shifted out of the decoder. The syndrome registers are shifted once to the right, the next block of n received bits is shifted into the decoder, and the next set of n - k syndrome bits is calculated and shifted into the leftmost stages of the n - k syndrome registers. Step 5o The syndrome registers now contain the modified syndrome bits :ilong with the new set of syndrnme hits. The decorler re'pe,:,k dPp"

e
Section 13.5

Decoding

557

2, 3, and 4 and estimates the next block of information error bits eiO), eil), · · · , ef<-ll. AH successive blocks of information error bits are then estimated in the same way. As noted if all previous sets of k estimates are correct, their effect is removed by the feedback, and the next set of k estimates on only + 1) error blocks. A block of a general feedback decoder for an (n, k, m) code is shown in Figure 1323.

A

A 000

-,

'

l-{'~~a l~~r

I

+

0

0

,;I

~

0

O

~I

,;I fl

~I

~t> ~~\t> : -, :tb ~

~

~->®~~s--- ---~~ ._,,;;-

,---~-----v +

C

"i=:

--.<:

Q)

~~

.£,~faaa---~

A ->-

0

"'

->~

:=:-

I :3;

:::+

A 000

-"'

+

000

I

3,:

FIGURE 13.23: Complete majority-logic decoder for an (n, k, m) systematic code.

653

Chapter 13

Suboptimum Decoding of Convolutional Codes

EXAMPLE 13.12

A Rate R

= 2/3 Self-Orthogonal Code

gi

Consider the (3, 2, 13) systematic code with 2 ) (D) = 1 + D 8 + D 9 + D 12 and 2 g~ \D) = 1 + D 6 + Dll + D 13 . Following the procedure of Example 13.10, we can write the first (m + 1) = 14 syndrome bits as (0)

eo

(0)

1 0 0 0 0 0

so s1

s2 S3 S4 S5 S6 s7

[§T]l3 =

0

0 1 1 0 0 1 0

ss

s9 s10

Sll s12 s13

el

(0)

1 0 0 0 0 0 0 0 1 1 0 0 1

e2

1 0 0 0 0 0 0 0 1 1 0 0

(0)

e3

1 0 0

0 0 0 0 0 1 1 0

(0)

1 0 0 0 0 0 0 0 1 1

e4

(0)

1 0 0 0 0 0 0 0 1

e5

1 0 0 0 0 0 0 0

(0)

e6

1 0 0 0 0 0 0

(0)

1 0 0 0 0 0

e7

(0)

1 0 0 0 0

eg

(0)

1 0 1 0 0 1 0 0 0 1

e9

(0)

elO

(0)

ell

0)

el2 (0)

e13 (1)

(2)

eo

eo

(1)

+

1 0 0 0 0 0 1 0 0 0

(2)

el

el

(1)

0 0 0

0 0 1 0 0 0 0 1 0 0 1 1 0

(2)

e2

1 1 0 1 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 1 0

e2

(1)

(2)

e3

e3

(1)

1 0 0 0 0 0 1 0

e4

(1)

1 0 0 0 0 0 1 0 0 0

(2)

e4

0

(2)

e5

es

(1)

1

e6

0

1

0 0 0 0 1 0

0

1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 1 1 0 0 0 0 0 1 0

(1)

e7

(1)

es

(1)

e9

(1)

elO

(1)

ell

(1)

el2 (1)

e13

(2)

+

e6

(2)

e7

(2)

eg

(2)

e9

(2)

elO

(2)

ell

(2)

el2 (2)

el3

(13.60)

Section 13.5

Majority-Logic Decoding

lSS~

There are two triangles in this case, one corresponding to each generator '"'""'~" We can use these to form a set of J = 4 orthogonal check-sums on the information error bit e6°l as follows:

IT]

--), 1

0 1 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 __,,1000

1 0 1 0 0 1 0 0 0 1

0 0 0 0 0 1 0

oooo[JJ

0

o[I]ooooo[I]

__,,1[J]oo 0110 0011

ooooo[I] 0 0 0 0 0 01 0 0 0 0 0 001

0 0 1

0 0 0

0 0 0

IT]

o

__,,

o

o[i][J]ooo o 0 0 l 1 0 0 0

0

o 0

oo[JJ 0 0 0 0 1

1

0

1

0 0 0 0 1

0 0 0 0 0

1 0 1 0 0 1 0001 00001

IT] 0

0 0 0 0 0 1 0 0 0 0 001000 o oo[J]o o 000010

0

ITJ 0 1 001 o oo[I] 0 0 0 0 1

we can form a set of J = 4 orthogonal check-sums on e61l as follows: --),

--),

[I] 0 0 0 0 0

1 0 0 0 0

0 0 1

0 0 0 1

1 --),

--),

0

1

0 1

0 0

0

[I]

1 0 0 0 0 0 0 0 1

1 0 0 0 0 0 0 0

[I] IT] 0

1

0

0

1 0 0 0 0 0 0

0 0 0 0 0

0 1

0 0

1

[I] [I]

IT] 0 0 0 0

1 0 1 0 0 1 0 0 0 l

0 oooo[I] 0 0 0 0 0 0 1 0 0 0 0 0 0 0

1 0 0 0 0 0 1 0 0 0 0

1 0 0 0 0 0 l 0 0 0

1 0 0 0 0 0 1 0 0

1 0 1 0 0 1 0 0 0 1 0

0 0 0 0 1

IT]

0 0 0 0 0

ITJ

1 0 0 0 0 0 1 0 0 0 0

ITJ 1 0 ITJ O 0

0

0 0 0 0 0 1

l 0 1 0 0 1 0 0 0 1 0 ooo[I] 0 0 0 0 0 1

0 [J]ooo

0 0

IT].

Because an check-sums are formed from syndrome bits alone, and not sums of syndrome bits, this is a self-orthogonal code. A total of 31 different channel error bits (24 information error bits and 7 parity error bits) are checked by the two orthogonal sets, and the effective decoding length is 11£ = 31. Hence, the majorityand 1) whenever fML = 2 or logic decoding rule will correctly estimate both fewer of these nE = 31 error bits are l's. A block diagram of the decoder is shown in Figure 13.24-. 6

e6°l

e6

61n the remainder of this chapter the labeling of Figure 11.5 is used to describe rate R systematic feedforward encoders.

=

(11 -

l)/11

660

Chapter 13

Suboptimum Decoding of Convolutional Codes f:i: __ ,;:i

,;:i

A

A

FIGURE 13.24: Complete majority-logic decoder for a (3, 2, 13) self-orthogonal code.

Section 13.5

The total number of channel error bits that appear in the (13.59) is nA ~ n(m + 1), called the actual In JwAa!HIJ''-' nA = 3(14) = 42 channel error bits appear in the syndrome Hence, 11 channel error bits have no effect on the estimates of and 1 As decoding proceeds, these 11 error bits will affect the estimates of successive of more than iM L = 2 information error bits. Also note that there are many channel errors that are corrected by this code; however, there are some of 3 channel errors that cannot be corrected.

e6°l

IE)ZAM!Pl!E

n. n

A lftarit® J?.

e6 l.

= li/3 Oll"itrtu©9J©u1ZJ!i:r:Z11Ql!® IC©!Dl®

Now, consider a (3, 1, 4) systematic code with ~Cll(D) = 1 + D and ~( 2l(D) = 1 + D 2 + sequences, s(l) = 1l. + D 4 . fo this case, there are two 2 ), · · · ), and we can write the first (m + 1) = 5 blocks of and s< 2l = (sfl, syndrome bits as

(s6 s?), sil). ·. ·)

s?), si

~

(1) (1)

sl (1)

s2

(1)

S3 (1)

[rl)4 =

Jl)

Iel

eo

so

S4 (2)

so

(2)

sl (2)

s2

(2)

S3

(1)

1 1 0 0 0

1 0 1 1 1

1

1 1 1 0 1 1 0 0 1 1

(1)

1 1

(1)

(0)

e3

(OJ

e4

el

(1)

+

e2

1 0

e2

(0)

eo

eo

(0)

el

e3 1 0 1 1 0

(2)

(0)

(2)

e4

(2)

e2

(2)

1

e3

(2)

(2)

e4

S4

As in the case of the (3, 2, 13) code of Example 13.12, there are two fo Example 13.12 we used the two parity triangles to form two orthogonal check-sums on two different information error bits. fo this v",.,",J'v, there is only one information error bit per unit time, and we use the two triangles to form a single set of orthogonal check-sums on the information error bit as follows:

e6°l

1 1 0 0 0 1 0 1 1 1

DJ 1 0

0

1 0 1 1

1 1 0

DJ 0 1

1 1

1

[I] 0

DJ

16162

Chapter 13

Suboptimum Decoding of Convolutional Codes

(0)

r1 + 4

r/~

4

FIGURE 13.25: Complete majority-logic decoder for a (3, 1, 4) orthogonalizable code.

The check-sums s(l) / 2l /ll s< 2J / 2) 0'0'1'2'1

+ s(32),

and /ll 2

+ sC42l

form a set of J = 6

orthogonal check-sums on e6°J. The effective decoding length n E = 13 for this orthogonalizable code, and the majority-logic decoding rule correctly estimates e6°l whenever fML = 3 or fewer of these 13 error bits are l's. A block diagram of the decoder is shown in Figure 13.25. In the foregoing discussion of feedback majority-logic decoding, it was assumed that the past estimates subtracted (added modulo-2) from the syndrome were all correct. This is of course not always true. When an incorrect estimate is fed back to the syndrome, it has the same effect as an additional transmission error and can cause further decoding errors that would not occur otherwise. This difficulty is called the error propagation effect of feedback decoders [33]. IEXAM/PlllE B.1(())

e;

0

e;

0

For the (2, 1, 6) self-orthogonal code of Example 13.10, let 0) = e; ) + l be the result of adding the estimate e1°l to a syndrome equation containing e;0l. Because

Section 13.5

e;

Majority-Logic

e;

is formed after 0l has been decoded, it is called a error. H all to estimates are assumed to be correct, the postdecoding errors are all 0, and the modified syndrome by (13.55). If this is not the case, however. the unmodified by 0

)

sf I

-(0)

-(0)

e/-6

+el-4 -(0)

-(0)

5

1+1

5

I

-(0)

/+4

e/-2

el-5

+e/-3

s1+6

(13.62)

ei

reduces to (13.55) if 0l = 0, Jc = I - 6. · · · , I - L If any it has the same effect as a transmission error in the and can cause error propagation.

Clearly,

ei0l =

1,

Several have been taken to the effects of error propagation. One is to periodically the decoder inserting a string of km zeros into the information sequence after every kh information bits. When the resynchronization sequence is received, the decoder is instructed to decode a string of km consecutive zeros. During this m time unit span of correct decoding, all postdecoding errors must be zero, and the effect of the past is removed from the syndrome. This periodic of the decoder limits error propagation to at most h + m time units; for the fractional rate loss m/(h + m) to be small, h must be much larger than m, and the effects of error propagation can still be quite Another approach to limiting error is to select codes with an automatic resynchronization Certain codes have the that if the channel is error-free over a limited span of time units, the effect of past errors on the syndrome is automatically thus halting error propagation. For example, it will be shown in Section 13.7 that codes possess this property. Also, uniform codes, an class of low-rate codes with large majority-logic error-correcting capability (similar to maximum-length block codes) introduced Massey [34], have been shown by Sullivan [35] to possess an automatic resynchronization property. Error propagation can be completely eliminated simply not using feedback in the decoder; that is, past estimates are not fed back to modify the This approach. first suggested Robinson [36], is called definite decoding. Because the effects of previously estimated error bits are not removed from the syndrome in a definite decoder, however, these error bits can continue to influence future decoding estimates, thus possibly causing errors that would not be made by a feedback decoder. On the other hand, error propagation due to erroneous decoding estimates is eliminated in definite decoding. An analysis Morrissey [37] comparing the effect of error propagation with feedback to the reduced errorcorrecting capability without feedback concludes that feedback decoders outperform definite decoders unless the channel is very noisy.

664

Chapter 13

Suboptimum Decoding of Convolutional Codes

Systematic feedforward encoders are preferred over nonsystematic feedforward encoders for majority-logic decoding applications because, with systematic encoders, orthogonal parity checks must be formed only on the information error bits. With nonsystematic encoders, orthogonal checks must be formed on all the error bits, and the estimated information sequence must be recovered from the estimated codeword using an encoder inverse. This recovery process can cause error amplification, as noted previously in Section 12.3. (Recall that for catastrophic encoders, the lack of a feedforward encoder inverse results in an infinite error amplification factor.) In addition, syndrome bits typically check more error bits for a nonsystematic encoder than for a systematic encoder with the same memory order, thus making it more difficult to find the same number of orthogonal parity checks. For example, we can view the construction of orthogonal parity checks for rate R = 1/2 nonsystematic feedforward encoders as being essentially equivalent to the construction of orthogonal parity checks for rate R = 2/3 systematic feedforward encoders. As an illustration, consider the rate R = 2/3 systematic code of Example 13.12. If the two generator polynomials are used instead to construct a rate R = 1/2 nonsystematic code, exactly the same procedure can be used to form sets of 1 J = 4 orthogonal parity checks on the error bits e60) and e6 ). (The effective decoding length of the rate R = 1/2 nonsystematic code is only 11£ = 24 in this case, compared with 11£ = 31 for the rate R = 2/3 systematic code.) It follows that for a given memory order m, we can form the same number of orthogonal parity checks on a rate R = 2/3 systematic code as on a rate R = 1/2 nonsystematic code, although the nonsystematic code will have a smaller effective decoding length. Also, this nonsystematic, rate R = 1/2, J = 4 selforthogonal code has memory order m = 13 and effective decoding length n E = 24, whereas the systematic, rate R = 1/2, J = 4 self-orthogonal code of Example 13.10 has m = 6 and 11£ = 11. Thus, the systematic feedforward encoder results in a more efficient realization of a rate R = 1/2 code with majority-logic errorcorrecting capability tML = 2 than the nonsystematic feedforward encoder. In other words, although nonsystematic feedforward encoders have larger free distances than systematic feedforward encoders with the same memory order, this does not imply that they have larger majority-logic error-correcting capabilities (see Problem 13.23). We now give an example showing that it is also possible to use systematic feedback encoders in majority-logic decoding applications. EXA.MPLIE 13.14

Consider the (2,1,4) systematic feedback encoder with generator matrix (13.63) The parity generator sequence of this encoder is given by g< 1l = (1100101001 010010 ···),where the 5-bit sequence (10010) repeats periodically after an initial 1. This semi-infinite parity generator sequence results in the follov1ing semi-infinite parity triangle:

6155

Section 13.5 ~

~-

~

~

,,_j_ 1 0 0 1 0 1 0 0 1 0 1 0 0 1 0

IT] 1 0 0 1

1 1

0 1 0 0 1 0 1 0 0

IT]

1

0 0

0 1 0 0 0 1 0 0

1

IT] IT] 0 0 1 0 1 0 0 1 0

1 0

1 0 0 1 0 1 0 0

1 0 1

1

IT] IT] 0 0 1 0 1 0 0 1 0

1 0 0 1 0 1 0 0

1

1

1

1 1 0 0 1 1 0 0 1 0 0

0 0 1 0

1 1 0 0 1 0 1

1 1 1 0 1 1 0 0 1 1 1 0 0 1 0 1 0 0

1 1

1

to form J = 4 orthogonal checks on the and that the effective decoding is nE = 11. Because each is a single the code is self-orthogonaL This (2. L 4) self-orthogonal code with J = 4, tML = 2, and nE = 11 is obtained with a memory order m = 4feedback encoder. In Example the same were obtained with an m = 6 systematic feedforward encoder. Th.us. in this case, a feedback encoder a more efficient realization of a self-orthogonal code with capability fML = 2 than a feedforvvard encoder. It is to note that if the preceding semi-infinite sequence ~(ll = (1100101001010010 ... ) is truncated after 7 we obtain the same finite sequence ~(ll = (1100101) used to them = 6 code in ,~"'"""'"V"~ 13.10. uu.,~uJ1'-'

with J orthogonal check-sums on each information form of the majority-logic decoding rule can be applied um,,a,,,,.,u A WGN channel. In these cases, the decoding rule takes the form of estimating an error bit e1 = 1 if and only if a sum of the checks (formed, as on a BSC, from the hard-decision channel exceeds some threshold. this form of decoding is referred to as threshold decoding. A WGN channel, the (real-number) received sequence , ri 0\?l ... r{"-l), rJ0\J1l ... ri'7-ll, ... ) corresponding to the transmitter mapping 1 -> +1 and O - -1 can be quantized sequence by using the mapping 1 if > O; 13.26). Then, these hard-decision outputs can be used to form sequence§. Now, let {A 1} represent a set of orthogonal checkbits or sums of syndrome bits) on an error bit e1, i = 1, 2, · · · , J.

r?) -;.

r?)

666i

Chapter 13

Suboptimum Decoding of Convolutional Codes

Threshold decoding is designed to produce an estimate ez of the error bit ez such that the a posteriori probability P(ez = e1 l{Ai}) is maximized. On a DMC, harddecision outputs and orthogonal check-sums are formed in an analogous manner. For this reason, on DMCs or an unquantized A WGN channel, the term a posteriori probability threshold decoding, or APP threshold decoding, is used to refer to the generalized majority-logic decoding rule. We now proceed to develop the APP threshold decoding rule for an unquantized A WGN channel. First, we let Pi be the probability that check-sum Ai contains an odd number of 1's, excluding error bit ez , and we let qi = 1 - Pi. (Clearly, qi is the probability that check-sum Ai contains an even number of 1's, excluding error bit e1.) Then, if ni is the number of error bits included in check-sum Ai, excluding error bit e1, we let ei.i be the jth error bit in check-sum Ai, ri.i be the corresponding soft received value, and Yi.i be the conditional probability that ei.i = 1 given ri.i, j = 1, 2, · · · , ni. In other words, YiJ=P(eij=llrij),

i=l,2,···,l,

J=l,2,···,ni.

(13.64)

Assuming BPSK modulation, soft received values ~riJ, i = 1, 2, · · · , J, j = 1, 2, . · · , ni, and an unquantized A WGN channel with one-sided noise power spectral density No, we can show that (see Problem 13.24) e-L,lriil

Yi}

(13.65)

= 1 + e-L, lr;;I'

where Le= 4Es/No is the channel reliability factor. For DMCs, the APP threshold decoding rule also quantizes the received sequence ir into hard decisions and then uses the channel transition probabilities to calculate the conditional probabilities Yi.i in (13.64). We now establish the relationship between Pi and the YiJ 's by making use of a well-known result from discrete probability theory, expressed here as a lemma (see Problem 13.25).

B.1 Let {e.i} be a set of n independent binary random variables, let = 1), j = 1, 2, . · · , n, and let p be the probability that an odd number of the e1 's are 1's. Then,

lLEMMA

YJ

=

P(e 1

11

p

=2 1 [1-

fl (1 -

(13.66)

2yj) ] .

j=l

Using (13.66) we can write 111

Pi = 1 [ 1 2

fl (1 -

2yij) ] ,

i = 1, 2, · · · , J.

(13.67)

.1=1

Finally, we let po = P(ez = llri) (computed using (13.65)) and qo = 1 - Po and define the weighting factors Wi = log(qi /Pi), i = 0, 1, · · · , J. We can now state the APP threshold decoding rule for an unquantized A WGN channel.

APP Thresiwld Decoding Rll.ule (A WGN Channel) T

We define the threshold

= L(O.c:i :::I) w;, and we choose the estimate e1 = 1 if and only if

Section 13.5 J

LA;w;

2':_

T/2.

i=l

Note that if we choose an the APP threshold decoding rule becomes J

+ 1)/2,

LA;2:_ i=l

which is equivalent to the note, however, that if we calculate the factor (13.64) and a BSC with crossover probability p, the APP threshold decoding rule is to the majority-logic decoding rule only if aH J orthogonal check-sums include the same number of error bits, that only if n1 = n2 = · · · = n 1 Problem '!rlHIJEl[))IR?JEM

P(ez

BJ; The APP threshold decoding rule maximizes the APP value

= ez l{A;}) of error bit ez.

!PnJ@f. Using Bayes' rule, we can express the APP value P(e 1 error bit ez as A

=

e11{A;})

of

P(e7=e1) . P({A;}) .

A

P(e1 = e!l{A;}) = P({A; }le,= e1)

where P({Adle1 = ez) represents the probability of the J check-sum values, given that = A decoding rule that maximizes the APP value vviH therefore choose 1 = 1 if and if

e,

e,.

e

P({A; )lez = l)P(e1 = 1) 2:. P({A; )lei = O)P(e1 = 0).

71)

Now, note that the check-sums A;, i = 1, 2, · · · , J, given a value of the error bit e1, are independent random variables. This observation follows of the check-sums. Hence, we can write from the orthogonality

n J

P({A; }lei= e1) =

P(A; lei= e1),

i=l

and a decoding rule that maximizes the APP value will choose e1 only if

n

n

i=l

i=l

J

P(e1 = 1)

=

1 if and

J

P(A;le1 = 1) 2:. P(e1 = 0)

P(A;le1 =

Taking logs of both sides and rearranging terms, we can rewrite J

P(A; I€/

= 1)

L. log P(A; lei = 0) >1=1

l P(ez og

= 0)

P(ez = 1)

.

(13.73)

as (13.74)

Next, we note that P(A; = Ole1 = 0) = P(A; = lle1 = 1) = q;

(13.75a)

P(A; = lle1 = 0) = P(A; = Olez = 1) = p;.

(13.75b)

and

668

Chapter 13

Suboptimum Decoding of Convolutional Codes

We can now write inequality (13.74) as J

P

qo

2)1 - 2A;) log_:_ ~log-. i=l q; PO Hence, a decoding rule that maximizes the APP value will choose e1 only if

(13.76)

= 1 if and (13.77a)

(13.77b) or

J

LA;w; ~ T/2.

(13.77c)

i=l

Q.JEJ), It is interesting to compare the APP threshold decoding rule with the MAP decoding algorithm developed in Section 12.6. In APP threshold decoding, the a posteriori probability of an error bit e1 is maximized given the values of a set of J orthogonal check-sums on e1, whereas in MAP decoding the a posteriori probability of an information bit u1 is maximized given the entire received sequence ir. Thus, each decision made by a MAP decoder is optimum, since it is based on the entire received sequence, whereas APP threshold decoding is suboptimum, because each decision is based on only a portion of the received sequence. Also, APP threshold decoding produces its estimates of the information bits indirectly by first estimating whether a hard-decision received bit was "correct" or "incorrect," whereas MAP decoding directly estimates the information bits based on their APP values. The APP threshold decoding rule is considerably simpler to implement than the MAP decoding algorithm, however. To implement the APP threshold decoding rule for an unquantized A WGN channel, hard decisions must be made on the received symbols to compute the syndrome and the orthogonal check-sums as described earlier in this section. The soft received symbols must also be fed into a computation module, where, using (13.65) and (13.67), the weighting factors w;, i = 0, 1, · · · , J, and the threshold Tare computed for each information error bit e;i) to be estimated, l = 0, 1, 2, · · · , j = 0, 1, .... k - 1. (Note that using (13.65) requires that the receiver maintain an estimate of the channel SNR Es/No, just as in MAP decoding.) Because the threshold T is, in general, different for each error bit to be estimated, decisions are made by comparing the (variable) weighted statistic L(l:::i ::cl) A; w; - T /2 with the fixed value O; that is, we choose the estimate = 1 if and only if

e;il

J

LA;w;

T/2 ~ 0.

(13.78)

i=l

A block diagram of an APP threshold decoder for the (2, 1, 6) self-orthogonal code of Example 13.10 and an unquantized A WGN channel is shown in Figure 1::nn.

Section 13.5

ct>,------~- - .---·-----

0

6 0 6 x

A

-~ mi_ ~

~

Lh

;J

A

oao

] 1

:P FIGURE 13.26: Complete APP threshold decoder for a (2. 1. 6) self-orthogonal code and an A WON channel.

671>

B.6

Chapter 13

Suboptimum Decoding of Convolutional Codes

PIEIRFOIRMANCIE CHAIRACTIE!RiS1KS Of MA.IOIRITY-LOGK IDIECOD!NG

In Section 11.3 the minimum distance of a convolutional code was defined as dmin

=

min11 {d(['./]m, [v

[llll'],,,,[llll

11 ]111):

[1U!']o -f [mi"]o}

],,,

= min{w[v] [llll],,,

111 :

ru10

(13.79)

-f IO},

where v, v', and v" are the codewords corresponding to the information sequences mi', and llli°, respectively. Note that in the computation of dmin, only the first (m + 1) time units are considered, and only codewords that differ in the first information block are compared. Now, consider a BSC with received sequencer and define a feedback decoder as one that produces an estimate iiio = llllo of the first information block if and only if the codeword v = IDJG minimizes d([r] 111 , [v] 111 ), where llllO is the first block in the information sequence l!l!. The following theorem demonstrates that the minimum distance of a convolutional code guarantees a certain error-correcting capability for a feedback decoder.

lll!,

13.3 For an (n, k, m) convolutional code with minimum distance is correctly decoded by a feedback decoder if L(dmin - l)/2J or fewer channel errors occur in the first (m + 1) blocks [r] 111 of the received sequence. THIEORJEM

dmin, llllO

JProoJ: Assume that the codeword v corresponding to the information sequence u is transmitted over a BSC and that r

d([Ir]m, [w]m)

=

w([r]m - [v]m)

= v + e is received. Then, = w([e]m).

Now, let v' be a codeword corresponding to an information sequence -f ll!Q. Then, the triangle inequality yields

(13.80) 1Ul,

with

llll~

d([r]m, [v']m)::: d([v] 111 , [w']m) - d([r] 111 , [v] 111 ) = d([v]m, [v']m) - w([e]m) ::: dmin -

(13.81)

w([e] 111 ).

By assumption, the number of channel errors w([e] 111 ) < (dmin - 1)/2 and hence, ,

d([r]m, [v ]111) >

dmill

2

+1

>

dmin -

2

1

::: d([r] 111 , [v]m),

(13.82)

Therefore, no codeword v' with 1!1!~ -f llllo can be closer to ll' over the first (m + 1) time units than the transmitted codeword v, and a feedback decoder will correctly decode the first information block Mo. (Note that the codeword, say v', that minimizes d([r] 111 , [v'] 111 ) may not equal v, but it must agree with v in the first block.) Q.IE.D, In Theorem 13.3 we see that a feedback decoder that finds the codeword that is closest to the received sequence r over the first (1n + 1) ti1ne units, and then

Section 13.6

Performance Characte:-istics o-f

chooses the first infon:nation block in this codeword as its estimate of uu0 , concect of 1mo if there are L(dmin - l)/2J or fewer channel errors in the first + 1) blocks of it. Theorem 13.3 also has a converse; that a feedback estimates uuo from [rr] 111 whenever [rr] 111 contains LCdm;11 - 1)/2J decoder that or fewer channel errors cannot estimate ®o from [rc] 111 for an [1r] 111 - l)/2J + 1 channel errors. fo other there is at least one received -1) /2J + 1 errors in its first (m + 1) blocks that will resuh Problem 13.27).Hence,tFB ~ l(d111 ; 11 -1)/2J iscalled of a code. H the ~vvv·~'"''"' that are fed back do not cause any of each successive information block uu1 based on to the + 1) received blocks (n·1, n·1+1 , ... , r 1+111 ). feedback decoder that bases its estimate of ®o on + 1) blocks of the re(;eived sequence thus has feedback decoding Pr1C.CU'-<",'>n,•prnn,n - l)/2J. In the case of a feedback majority-logic since checks correct decoding of uuo whenever [rc] 111 contains t 1v1L = LJ /2J or fewer channel errors, it follows that J

:00:::

dmin -1.

(13.83)

checks can be formed on each information error bit, orthogonalizable; that the feedback decodcan be achieved with majority-logic decoding (see H the code is not completely cannot achieve ihe feedback error~'""'"''"-''~ 13.16 and Problem 13.31). Hence, it is desirable when to select codes that are completely orthogonalizable. This restricts the choice of codes that can be used with majority-logic since most codes are not orthogonalizable; however. as will be seen in Section several classes of orthogonalizable convolutional codes have been found. Note that the result of Theorem 13.3 does not to BCJR, or since these methods process the entire received sequence n· before a final decision on uuo. 7 The longer decoding delay of these "-'""v,c,rn,~ methods is one reason for their superior performance when compared decoding; majority-logic decoders are much simpler to (m + 1) blocks of the received sequence at any time. The minimum distance of a convolutional code can be found the CDF di and then letting i = m dmin = d 111 ). A more direct way of finding dmin makes use of the parity-check matrix IHL Because w is a codeword if and if wlh1T = @, the minimum number of rows of JHIT, or columns of JHI, that add to I!]) to the nonzero codeword. Because dmin is the of the codeword with uno f: 0 that has minimum weight over its first (m + 1) blocks, it can be by finding the minimum number of rows of JHIT, or columns of

H

7 Even a truncated Viterbi decoder, or a backsearch-limited sequential decoder, must process at least 4111 or 5111 blocks of the received sequence before making any final decisions.

672

Chapter 13

Suboptimum Decoding of Convolutional Codes

H, including at least one of the first k, that add to O over the first (m + 1) time units. In other words, we must find the minimum-weight codeword [w] 111 with 1uo i= 0 for which [w] 111 [HT] 111 = 0, where [HT] 111 includes only the first (m + l)(n - k) columns of HT. This is equivalent to forming the first (m + l)(n - k) rows of lHI and then finding the minimum number of columns of this matrix, including at least one of the first k columns, that add to 0. !EXAMPLE 13.15

finding dmin U:sinig the H Matrix

Consider the (2, 1, 6) systematic code of Example 13.10. The first (m (7)(1) = 7 rows of !HI are 1 1 0 0 1 0 1

[H]m = [JHI]6 =

+ l)(n

k)

=

1 1 1 0 1 0 1 1 0 0 0 1 0 1 1 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0

0

(13.84) 1 1 1 0 0 0

1 1 1 0 1 1

The minimum distance dm;" is the minimum number of columns of this matrix, including the first column, that add to zero. Because J = 4 orthogonal check-sums can be found for this code, we know that dmin ~ J + 1 = 5. But columns 1, 2, 4, 10, and 14 add to zero, implying that dmin ::= 5. Hence, dmin must equal 5 for this code, and the code is completely orthogonalizable. The minimum-weight codeword with auo i= 0 in this case is given by

[w]

111

=

(11,01,00,00,01,00,01).

We can also obtain dmin by finding the minimum-weight linear combination of rows of the generator matrix [G] 111 that includes at least one of the first k rows, that is, the codeword with lll!O i= 0 that has minimum weight over the first (m + 1) time units. IEXAMPLIE 13.16

findinig dmin Usilrilg 11:he G Matrix

Consider the (2, 1, 5) systematic code with gOl (D) = 1 + D 1 1 0 1

[G]m = [G]s =

1 0 0 0 1 0 0 1 0 1 0 0 0 1 1 1 0 1 0 0 1 1 0 1 1 1

+ D 3 + D 5 . Then, 0 0 0 0 0 1

1 0 1 0 1 1

(13.85)

There are several information sequences with uo = 1 that produce codewords of weight 5. For example. the information sequence [ll.ll]m = (11100 0) produces the rnnPW(Wn [ W ]m

=

(11 , 10, 1 f\

()f),

()1 , ()()); hnWP\/er, thPrP

,ci,-p

no ]inP,ir rrnn hin,:itinn~

Section 13.6

Performance Characteristics of

of rows of [G] 111 including the first row that have code. The parity triangle is given by 1 1 1 0 1 1 0 0 1 1 0

4. Hence,

= 5 for this

1 1 1 0 1 1 1 0 1 1.

The maximum number of orthogonal check-sums that can be formed in this case is J = 3. For example, {so, s1, s3} and {so, s2 + s3, ss} are both sets of J = 3 check-sums on Thus, since J = 3 < dmin - 1 = 4, this code is not v~,n,-,,v•·~, orthogonalizable; that its feedback decoding error-correcting its majority-logic error-correcting capability is only L

e6°l.

Examples 13.14 and 13.15 both consider was shown in Section 11.1 that any (n, k, m) feedforward ~u·v~•.,v, by means of a linear transformation on the rows of its generator can be converted to an equivalent (n, k, m') systematic feedback encoder. (Note that for k > 1, m' may be different from m.) Truncating the generaL semi-infinite) generator sequences of the systematic feedback encoder to length m + 1 then results in an (n, k, m) systematic feedforward encoder with the same value of d171 ;n as the original nonsystematic feedforward encoder. Hence, the feedback ~VV~'""''~ correcting capability cannot be improved by considering encoders. This result implies that in the selection of encoders with memory order m and large d111 ;n for possible use with majority-logic decoding, it suffices to consider systematic feedforward encoders. The situation is different for where nonsystematic feedforward encoders offer larger values of than systematic feedforward encoders with the same constraint noted previously in Sections 11.3 and 12.3. This advantage in dtree of feedforward encoders (or equivalent systematic feedback encoders) with systematic feedforward encoders with the same value of v accounts for their almost exclusive use in applications involving Viterbi or BCJR decoding, where decoding complexity increases exponentially with v.

Consider the (2, 1, 4) systematic feedback encoder of 13.14 and its equivalent (2, 1. 4) nonsystematic feedforward encoder with cG(D) = [1 + D + D 2 + D 3 + D 4 1 + D 4 ]. Then, for the nonsystematic encoder, 11

[G]4 = [

10 10 10 11 10 10 11 11 10 10 11 10 11

l ,

(13.86)

614

Chapter 13

Suboptimum Decoding of Convolutional Codes

and dm,n is the minimum-weight linear combination of rows of [

01 11

00 01 11

00 01 00 01 00 00 01 00 01 00 00 01

··· 01 00

(13.87)

01

[ of the systematic feedback encoder. Truncating the semi-infinite generator sequence 5, we obtain a systematic feedforward encoder with generator matrix
il1l = (11001010010 · · ·) to length m + 1 = 11

[G']4

=

[

01 11

00 01 00 00 11 01 11

01 00 00 01 11

l .

(13.88)

The minimum distance d~iin of this code is the minimum-weight linear combination of rows of [G']4 that includes the first row. In this case, row 1 by itself is a weight-4 codeword, and d~iin = 4 = d111 ;17 • Hence, the systematic feedforward encoder with generator matrix G' (D) has the same minimum distance as the original nonsystematic feedforward encoder with generator matrix G(D). The free distance of the nonsystematic encoder is 6, however, whereas the free distance of the systematic encoder is only 4.

The performance of an (n, k, m) convolutional code with majority-logic decoding on a BSC can be estimated from the distance properties of the code. First, for a feedback decoder with error-correcting capability tFs = L(d111 ;17 -1)/2J, it follows from Theorem 13.3 that a decoding error can occur in estimating uo only if more than tFs channel errors occur in the first (m + 1) blocks of the received sequence. Hence, recalling that the total number of channel error bits that appear in the syndrome equations (13.59) is the actual decoding length nA = n(m + 1), we see that the bit-error probability in decoding the first information block, Pbl (£), can be upper bounded by (13.89)

where pis the channel transition probability. For small p, this bound is dominated by its first term, so that - ( l',~)

I'b1

1 I(

~ Ir

"

\

Section 13.6

Performance Characteristics of

LJ /2J and

For a majority-logic decoder with effective length n E, an

urn~,~-~~

"'

1

11£

(

~

k 1=t,1n ~+1 \

and Pbl ( le,"')

~

11£

11

1 (

~ -

k

fML

) l(l -

£

+1

) plML+l.

(13.91)

(13.92)

strictly apply only to decoding the first information estimated information error block e1 is subtracted from the it and if these estimates are correct, the modified ucn.,v,,0 used to estimate ((;l+l are identical to those used to estimate(\;;/, except that different error bits are checked. This point was illustrated in Section 13.5 in connection with Example 13.10. Hence, for a feedback decoder, the bit-error Pb1 (£), that probability in any information previous estimates have been correct. Under this a feedback decoder has bit-error probability upper bounded 1

L 11,\

:": k

(

'

pi(l - p)11ri,

11A )

(13.93)

i=trn+l

by

and, for small p,

Similarly, for a majority-logic decoder that uses feedback to equations, Pb(E) :": -1 k

and (E)

l1£ ~

L

(

Ii£ 1

)

,i/ (1

- p)"E-i

the syndrome (13.95)

i=t;n +1

~!( k

11£

fML

+1

) 1/M1.+l_

(13.96)

As noted in Section 13.5, if previous decoding estimates are not all correct, postdecoding errors can appear in the modified syndrome equations, thereby causing error propagation and degrading the performance of the decoder. If the channel transition probability p is not too large, however, and if the code contains good resynchronization properties, such as the self-orthogonal codes to be discussed in the next section, the effect on bit-error probability is small, and (13.93) through (13.96) remain valid [37]. Finally, for the same code, soft-decision APP threshold decoding will generally perform about 2 dB better than hard-decision majority-logic decoding. We now do a rudimentary comparison of majority-logic decoding and Viterbi decoding on the basis of their performance, decoding speed, decoding delay, and implementation complexity.

616

Chapter 13

Suboptimum Decoding of Convolutional Codes

Peuformam:e. Viterbi decoding is an optimum decoding procedure for convolutional codes, and its bit-error probability on the BSC was shown in (12.36) to be approximated by

(13.97) when Eb/ No is large, where Cv is a constant associated with the code structure. For a (suboptimum) feedback decoder, it was shown previously that for a BSC, Pb(E)

~

!( k

nA fFB

+1

) ptFs+l

(13.98)

for small p. Using the approximation (see (12.33) and (12.35)) (13.99) we obtain Pb(E)

~

i(

fF;:

l ) (~)

e-E1,R(tFs+l)/No

(13.100)

~ CFse-R(dn,;n/2)(E1,/No)

when Eb/No is large, where CFs is a constant associated with the code structure. H the code is completely orthogonalizable, (13.100) also represents the performance of majority-logic decoding. The approximations of (13.97) and (13.100) differ in that Pb(E) decreases exponentially with dtree for Viterbi decoding, whereas Pb(E) decreases exponentially with dmin for feedback (majority-logic) decoding. Because for most codes (13.101) dmin = dm < lim dz = djree, 1---+oo

the asymptotic performance of feedback (majority-logic) decoding is inferior to that of Viterbi decoding, since a Viterbi decoder delays making decisions until the entire received sequence is processed or, in the case of a truncated decoder, until at least 4m or Sm blocks are received. It thus bases its decisions on the total (or free) distance between sequences. A feedback (majority-logic) decoder, on the other hand, bases its decisions on the minimum distance over only (m + 1) time units. Random coding bounds indicate that for a given rate and memory order, the free distance of the best nonsystematic feedforward encoders (or systematic feedback encoders) is about twice the minimum distance of the best systematic feedforward encoders [38]. (Recall that nonsystematic encoders cannot achieve larger values of dmin than systematic encoders, however.) This gives Viterbi decoding a roughly 3-dB advantage in performance over feedback decoding for the same rate and memory order. In other words, the memory order for feedback decoding must be about twice that for Viterbi decoding to achieve comparable performance. If majority-logic decoding is being used, an even longer memory order must be employed. This is because optimum minimum distance codes are usually not completely orthogonalizable; that is, to achieve a given dmin for a completely orthogonalizable code requires a longer memory order than would otherwise be necessary. If the code must be self-orthogonal (say, to protect against error propagation), the memory order must be longer yet. For example, the best rate

Section 1307

Code Construction for

code with d111 ;11 = 7 has memory order 1n = 17, the code vvHh = 7 has memory order m = 11, the = 7 has memory order m = 10, and the best = 7 has memory order m = 4"

Jl?.

best R

_,.v·~~·usur,c,

A Viterbi decoder 2v whereas a majority-logic decision) per decoded bit is somewhat different in each case, this decoders are (As noted in Chapter the can be increased a factor of 2v, making it attractive for a paraHel This increases the of the decoder, n~rncn,~o,c

decoder has a decoding of m time bits received at time unit I are decoded at time unit l + m" Viterbi '-''"'"'v·uu,;e,, on the other has a decoding delay equal to the entire frame length h + 1n; that decisions are made until all h + m encoded blocks have h > > m, the decoding delay is substantial in these cases" 8 truncated Viterbi is used, the decoding is reduced to 4m or Sm time vvith a minor penalty in performance")
In this section we discuss two classes of completely orthogonahzable codes for use with majority-logic self-orthogonal codes and orthogonalizable codes" 8Recall that for any convolutional code, when data is sent in frames, an additional m time units of "known" input bits must be fed into the encoder and the resulting encoder outputs transmitted over the channel to allow proper decoding of the last 111 blocks of information bits, Thus, when 111 is large, short data frames result in considerable rate loss. In the case of Viterbi decoding, the "known" bits are used to drive the encoder back to the all-zero state" In majority-logic decoding, the "known'" bits are arbitrary, and knowledge of these bits can be used by the decoder to improve the reliability of the last 111 blocks of information bits,

678

Chapter 13

Suboptimum Decoding of Convolutional Codes

The resynchronization properties of each of these classes when used with feedback decoding is also discussed. 13. 7 .1

Self-Orth101gc»nal Codes

An (n, k, m) code is said to be self-orthogonal if, for each information error bit in block l.llo, the set of all syndrome bits that check it forms an orthogonal check-set on that bit. In other words, sums of syndrome are not used to form orthogonal check-sets in a self-orthogonal code. Self-orthogonal codes were first constructed by Massey [6]. A more efficient construction, based on the notion of difference sets, was introduced by Robinson and Bernstein [40]. The positive difference sett:,. associated with a set of nonnegative integers{l1,l2,··· ,l1},wherel1 la. A positive difference set I':,. is said to be full if all the differences in t:,. are distinct, and two positive difference sets /':,.; and t:,.i are said to be disjoint if they do not contain any differences in common. Now, consider an (n, 11 - 1, m) systematic convolutional code with generator polynomials (11-l\D) _ (n-1) + (n-1) D + ... + (n-1) DvJ+I gj+l - gj+l,0 gj+l.l gj+l.v;+I '

j

= 0, 1, · · ·, 11 -2, (13.102)

. d wh ere t h e memmy or er m

(11-l) (n-ll Cn-ll b = max(O:"J:"n-2) VJ+l· L et gi+l.li, gi+l.1 2 , · · • , gi+l.1.1; e the nonzero components of g;~~ll (D), where l1 < 12 < · · · < l1J, j = 0, 1, · · · , n - 2, and let t:,.i be the positive difference set associated with the set of integers of {[1, l2, · · · , l1J The following theorem forms the basis for the construction of self-orthogonal convolutional codes. THJEORJEM 13.4 An (11, n - 1, m) systematic code is self-orthogonal if and only if the positive difference sets l':,.o, t:,.1, · · · , 1':,. 11 _2 associated with the code generator polynomials are full and mutually disjoint.

Pmof. The proof of this theorem consists of two parts. First, assume the code is self-orthogonal, and suppose that a positive difference set t:,.i exists that is not full. Then, at least two differences in t:,. 1 are equal, say lb - la = ld le, where lb > la and lc1 > le, Using (13.57), we can write the syndrome sequence as 11-2 §(11-l\D)

=

L 11./kl(D)gf~~l\D) + e(n-1\D),

(13.103)

k=O

which we also can express as n-2

§(n-1) (D)

=

eCil (D)gj~l) (D) +

L

e(k) (D)gf~1l)

(D) + e
k=O kf.j

n-2

=

(d1 +

d 2 + ... +di; )eUl(D)+

L e(k)(D)gf~1)(D) + e
k=O kf.j

Section B.7

Code Construction for

Because la, lb, le, and lc1 all belong to the set {l1, l2, · · · . }, the . s (11-l) , s (11-l) , s (n-1) , an d s (11-l) a11 chec , 1c t h e m · f ormatlon · b 1ts error lmt· e0(j) ·· • JLn 1{/ 1h 1( 1d particular, s(n-l) = /il + /fl + other terms /1, 0 /1,-la and /n-l) = e(J) Id O

+ e(}) + Id-le

other terms.

(13.105b)

1 an d t h e set o f ''"'"'"""=·= b'1ts t,1at check the information error bit e&il is not orthogonal, thus the assumption that the code is self-orthogonal.

B ecause l b

-

la

=

l c1 - l c,

(j) e/1,-l"

=

e 1(j) d-lc,

Now, suppose that two difference sets ~i and ~ J exist that are not Then, must have at least one difference in common. Let If, - la and fc1 - fc be the common difference in ~i and ~f, respectively. lb - la = fc1 - fc, and we can write the syndrome sequence as

n-2

+

L

e
+ e
k=O k,f.i,j

= (D 11 + D 12

+

+ ... + d 1; )eiil (D) + (Df + Dh + · · · + 1

n-2 ~ e<"\D)<w(n-l\D) ~

!3/c+l

+ e
(13.106)

k=O k,f.i,f

Assume, without loss of generality, that lb 2: fc1. Then, la 2: fc, and since la and Ii, belong to set /1, !2, · · · , l1, the syndrome bits s?'-ll and s1(n-ll both check I

the information error bit eg); that is,

(13.107a)

= /il + /il . + other terms. 0 lt,-fd

(13.107b)

la

/n-l) /1,

/)

other terms

s(n-l) =

and

/il + /fl . + O la-fc

II

e;,;~f; e~;~f:t.

But lb - la = !d - fc implies that lb fc1 = la - fc, and hence = Therefore, the set of syndrome bits that check the information error bit eg) is not orthogonal, which again contradicts the assumption that the code is self-orthogonal. We conclude that all n - l positive difference sets must be full and disjoint. Second, assume that the positive difference sets ~o. ~1. · · · . 1'1 11 _2 are full and disjoint, and suppose that the code is not self-orthogonal. there must exist at least one pair of syndrome bits for which (n-l) Siu -

(i)

eO

+ ela-i Ul +

other terms

(13.108a)

1580

Chapter 13

Suboptimum Decoding of Convolutional Codes

and

= e(i) + /il . + other terms 0 l1,-fc1 • If i = j, the difference la - fc and lb

s(n-l) /1,

(13.108b)

where la - fc = 1b - fc1. - fc1 are both in 1'1;, and hence 1'1; cannot be full. This contradicts the assumption that 1'1; is full. If i i= j, then the difference lb - la is in 1'1;, and the difference fc1 - fc is in l'1 i. Because la - fc = lb - fc1 implies that lb - la = fc1 - fc, in this case the positive difference sets !'1; and l'1 i cannot be disjoint. This contradicts the assumption that 1'1; and l'1 i are disjoint. We conclude that the code must be self-orthogonal. Q.E,D, Because each of the 11 nonzero components of the generator polynomial is used to form one of a set of orthogonal check-sums on e6il, 11

il:j~11\D)

orthogonal check-sums are formed on e6i). If 11 = J, j = 0, 1,. · · , n - 2, then each generator polynomial has the same weight J, and J orthogonal parity checks are formed on each information error bit. Therefore, if tu/ 1l (D) = 1 and ll!i<2l (D) = ... = 1 1 llll(n-ll(D) = 0, the codeword v(D) = [1, 0, 0, · · .. 0, \D)] has weight J + 1, and dmin ::: J + 1. On the other hand, dmin ::0:: J + 1, since otherwise the majoritylogic error-correcting capability tML = LJ /2J would exceed the feedback decoding correcting-capability t = l(dmin - l)/2J, which is impossible. Hence, dmin = J + 1, and the (11. n - 1. m) self-orthogonal code is completely orthogonalizable. If the Ji 's are unequal, it can also be shown that d171 ;11 = J + 1, where J ~ minco::::J:::n-2) Ji (see Problem 13.33). Hence, in this case also, the code is completely orthogonalizable. Robinson and Bernstein [40] have developed a procedure based on Theorem 13.4 for constructing (n, n - 1, m) self-orthogonal convolutional codes. Each of these codes has 1 1 = J = dmin - l, j = 0, 1, · · · , n - 2, and error-correcting capability fML = LJ /2J = tFB = L(d111 ;11 - 1)/2J. A list of these codes for n = 2, 3, 4, and 5 and various values of tML and 111 is given in Table 13.2. fo the table, each generator 1 polynomial 1 \D) is identified by the set of integers /1 , h,. · · , 11, that specifies the positions of its nonzero components. (Note that, in this case, the octal notation used in previous tables to specify polynomials would be quite cumbersome, since the polynomials are sparse, and 111 can be very large.)

gr

gj~

IEXAM!l'LIE B. U Or!:hl[))gJ
Consider the (2, 1, 6) self-orthogonal code from Table 13.2(a) whose generator polynomial il1l (D) is identified by the set of integers {O, 2, 5, 6}; that is, i{ 1l (D) = 1 + D 2 + D 5 + D 6 . This code has J = 4 and will correctly estimate the information error bit e6°l whenever the first nA = n(m + 1) = 14 received bits (more precisely, the 11£ = 11 received bits checked by the J = 4 orthogonal check-sums) contain tFs = lML = LI /2J = 2 or fewer errors. The positive difference set associated with this generator polynomial is l'1 = {2, 5, 6, 3, 4, l}. Because this positive difference set contains all positive integers from 1 to J(J - 1)/2 = 6, it follows that the memory order 111 = 6 of this code is as small as possible 9 for any rate R = 1/2 self-orthogonal code with lML = 2. (Note that the (2, 1, 6) self-orthogonal code of Example 13.10, whose generator polynomial is the reciprocal of the generator polynomial in this

Section 13.7

Code Construction for Majority-Logic

TABLE 13.2: Self-orthogonal codes. 'ML

m

1 2 3 4 5 6 7 8 9 10 11 12

1 6 17 35 55 85 127 179 216 283 358 425

{0,1) {0,2,5.6) {0.2,7.13.16,17) {0,7,10,16,18,30.31,35) {0.2, 14.21.29 ,32,45 ,49 ,54,55) {0.2.6,24,29 ,40,43.55 ,68. 75, 76.85} {0,5 ,28.38,41,49 ,50,68, 7 5 .92, 107. 121. 123. 127) {0.6,19,40,58.67 .78,83,109 .132,133.162,165,169 J 77,179} {0,2,10,22.53 .56.82.83 .89 .98,130, 148. 152,167.188, 192,205,216} {0,24,30,43,55 .71.75 ,89 ,104,125, 127. 162.167. 189,206,215,272.275 ,282.283} {0,3 .16,45 ,50.51.65 ,104,125 J 42,182.206.210.218.228.237 ,289 .300 .326.333 ,356.358) {0.22,41.57, 72.93.99 ,139 J 47,153.197,200,214.253 ,263 ,265 .27 6,283,308.367 ,368,372.396,425}

(a) R

1 2 3 4 5 6 7

2 13 40 86 130 195 288

=

1/2 CODES

{0.1) {0,8,9,12) (0,2,6,24,29,40) {0.1,27.30.61.73,81,83) {0,l.6,35,32,72,J00,108,120,130) {0,17,46,50.52,66,88,125,150,165,168,195} {0,2,7,42,45.117.163,185,195,216.229,246,255.279}

(b) R

=

{0.2) (0.6.11.13) (0.3,15.28,35,36) {0,18,23,37,58,62,75,86} (0,23,39,57,60,74,101,103,112,116} (0,26.34,47 ,57,58,112,121. 140.181.188.193} {0,8,12,27.28.64,113,131.154,160.208.219. 233,288}

CODES

m 1

2 3 4 5

3 19 67 129 202

(0.1)

(0.3.15,19) (0.5,15,34,35.42) (0,9 ,33,37,38,97 ,122,129) {0.7,27 .76.113.137 .155.156. 170,202}

(0,2) (0,8,17,18) {0.31,33,44,47 .56) {0.11,13,23,62,76,79,123} {0.8.38.48.59.82,111,146. 150,152)

{0,3} {0.6,11.13) {0,l 7.21.43A9.67} {0.19,35,50,71,77,117,125} {0.12.25,26.76.81.98.107. 143,197}

(c) R = 3/4 CODES tML

m

1 2 3 4

4 26 78 178

(4)

(0)

(41)

1/;z

1/;3

1/;4

(0,1) (0,16,20.21) (0,5,26,51,55,69) {0.19.59,68,85,88, 103.141)

(0.2) (0.2,10.15) (0,6,7,41,60,72) {0,39,87.117.138.148. 154.162)

{0.3) {0,14,17,26) {0.8,11,24.44,78) (0.2,13.25,96,118, 168,172}

(0,4) {0,11,18.24) {0,10,32,47.49 .77) {0,7,65.70,97.98. 144.178)

(d) R = 4/5 CODES Adapted from [40].

(4)

g]

682

Chapter 13

Suboptimum Decoding of Convolutional Codes

example, also has J = 4 orthogonal parity checks and tML = 2. This illustrates that taking the reciprocal of a generator polynomial does not change its associated positive difference set.)

EXAMPLE B.18 Orl:hlogomal Code

Consider the (3, 2, 13) self-orthogonal code from Table 13.2(b) whose generator 2 polynomials i?l (D) and \D) are identified by the sets of integers {O, 8, 9, 12} and {O, 6, 11, 13}, respectively. This code was previously shown to be self-orthogonal with 2 J = 4 in Example 13.12. The positive difference sets associated with J (D) and 2 ) (D) are ~o = {8, 9, 12, 1, 4, 3} and ~1 = {6, 11, 13, 5, 7, 2}, respectively. These positive difference sets are full and disjoint, as required by Theorem 13.4 for any self-orthogonal code.

gi

gi

gi

We can obtain a self-orthogonal (n, 1, m) code from a self-orthogonal (n, n 1, m) code in the following way. Consider an (n, n l, m) self-orthogonal code with generator polynomials gfi-l), ggi-l), · · · , g~~11> and let Ji be the weight of 11

1

1

g( + \D), j = 0, 1, · · · , n - 2. Then, we obtain an (n, 1, m) code with generator 1

polynomials lhi<1l(D), Jh/2 l(D), · · · , Jln(n-ll(D) from this (n, n - 1, m) self-orthogonal code by setting - (n-l)(D) , J . --0, 1 , .. · , n -2. lln (j+ll(D) -i·+i (13.109) 1 Because the positive difference sets associated with the generator polynomials 1 g;~ )(D), j = 0, 1, · · ·, n - 2, of the (n, n - 1, m) self-orthogonal code are full and disjoint, the positive difference sets associated with the generator polynomial JhiCi+ 1l(D), j = 0, 1, · · · , n-2 of the (n, 1, m) code are also full and disjoint.It is shown in Problem 13.35 that this condition is necessary and sufficient for the (n, 1, m) code to be self-orthogonal. Since Ji is the weight ofilJJ.Ci+ll(D), j = 0, 1, · · · , n -2, there are a total of J £ lo+ Ii+· · · + ]11 _2 orthogonal check-sums on the information error bit 0 e~ >, and the (n, 1, m) code has a majority-logic error-correcting capability of tML = LI /2J. Because the information sequence llll(D) = 1 results in the codeword v(D) = [1, ll.JI.C 1l(D), ... , Jln(n-ll(D)], which has weight J + 1, it follows that dmin::: J + 1; but fFB = L(dmin -1)/2J ::0: fML = LI /2J implies that dmin ::0: J + 1. Hence, dmin = J + 1, and the (n, 1, m) self-orthogonal code must be completely orthogonalizable. EXAMPUE B.19 Orthogonal Code

Consider the (3, 1, 13) self-orthogonal code with generator polynomials Jht(ll (D) = 1 + D 8 + D 9 + D 12 and Jhi.C 2) (D) = 1 + D 6 + D 11 + D 13 derived from the (3, 2, 13) self-orthogonal code of Example 13.18. This code has J = lo+ Ji = 4 + 4 = 8 orthogonal check-sums on the information error bit e6°l, and hence it has an errorcorrecting capability of tFs = t1v1L = LJ /2J = 4; that is, it correctly estimates e~Ol whenever there are four or fewer errors in the first nA - n (nt + 1) ==- 42 received

Section 13.

Code Construction for

the nE received bits (see Problem 13.36) checked the J = 8 In a similar fashion, every n - 1, m) self-orthogonal code in Table 13.2 can be converted to an (n. 1, m) self-orthogonal code. If a code is used with feedback decoding, each successive block of information error bits is estimated using the same decoding and the is the same as for block zero, assuming that the was illustrated in estimates have all been correct. This for the code of Example 13.10. On the other estimates have not been correct, postdecoding errors appear in the as illustrated in (13.62) for the same self-orthogonal code. In this case, the modified are still orthogonal, but errors as wen as trnnsrnission errors can cause further decoding errors. This results in the error effect of feedback decoders discussed in Section 13.5. VVe now demonstrate that self-orthogonal codes possess the automatic resynchronization with to error First, note that whenever error is e;Jl = 1, it is fed back and reduces the total a number of 1's stored in This observation follows from the bits that are used to estimate e}il rnust be fact that more than half the 1's in order to cause the decoding error, and hence when e;il = 1 is fed back it more 1's O's than O's to thereby the total in 1' fow, assume that there are no more transmission errors the at time unit I + m, only O's can enter the leftmost of registers. No matter what the contents of the are at time unit l + m, they must soon clear to all O's since only O's are and each estimate of 1 reduces the total number of l's in the registers. estimates of O have no effect on the register weights.) Once the to all O's, no further decoding errors are possible unless there are additional transmission errors; that error propagation has been terminated. cwcuvu~.

,C,VC>CCOvOAJC-'CW'i'.

1

fiL 7Ji

iQlrr1J:~(Q)(9J(Q)uuZJ!az31r(l)ie (iQloles

Completely orthogonalizable convolutional codes can also be constructed using a trial-and-error fo this case, some of the orthogonal parity checks are forraed from sums of bits, and these codes are not self-orthogonal. 13.11 contains J = 4 orthogonal checks The (2, l, 5) code of constructed trial and error, and hence dmin :::,, 5. Because the two generator polynomials have a total of 5, dmin must equal 5, and therefore the code is completely with error-correcting capability tFB = tML = LJ /2J = 2. 1. 4) code of Example 13.13 contains J = 6 orthogonal parity checks constructed by trial and error, is completely orthogonalizable, and has fFB = fML = LJ /2J = 3. A list of rate R = 1/2 and codes constructed Massey [6] is given in Table 13.3. generator polynomials are specified using the notation of Table 13.2. The notation used to describe the rules for the orthogonal check-sums in the rate R = 1/3 case 1s in the example.) Vh'JHCOH~uv,,-

684

Chapter 13

Suboptimum Decoding of Convolutional Codes TABLE 13.3:

Orthogonalizable codes.

m 2

5

3 4 5

11 21 35

{0,3,4,5) {0,6,7,9,10,11) {0,11,13,16,17,19,20,21 J {0,18,19 ,27 ,28,29 ,30,32,33,35}

6

51

{0,26,27 ,39 ,40,41,42,44,45 ,47 ,48,51}

(a) R

=

(1, 5)

1/2 CODES

m

g(l)

g<2)

3 4 5

4 7 10*

{0,1) (0,1,7) {0,1,9)

{0,2,3,4) {0,2,3,4,6) {0,1,2,3,5,8,9)

6

17*

{0,4,5,6,7,9,12,13,16)

{o,1,14,15,16J

7

22

{0,4,5,6,7,9,12,13,16, 19,20,21)

{0,1,20,22)

8

35

{0,4,5,6,7,9,12,16,17, 30,31)

{0,1,22,25,35)

lML

(1, 3, 10)(4,8, 11) (2, 3, 6, 19)(4, 14, 20)(1, 5, 8, 15, 21) (1, 9, 28)(10, 20, 29)(11, 30, 31) (13,21,23,32)(14,33,34)(2,3, 16,24,26,35) (1, 13,40)(14, 28,41)(15,42, 43) (17,29,31,44)(18,45,46)(2,3,20,32,34,47) (21,35,48,49,50)(24,30,33,36,38,51)

cor J
(b) R = 1/3 CODES Adapted from [6]. ''The actual value of III for these codes is less than shown, and the error-correcting capability IM L listed is achieved by adding O's to the generator sequences. ' (.r, ". · · ) indicates that the sums, + "" + · · · forms an orthogonal check sum on that require a sum of syndrome bits are listed.

"6° 1• Only those orthogonal equations

EXAMPlE 13.:W D"'lalizable Code

Consider the (3, 1, 7) code listed in Table ll.3(b) whose generator polynomials g< 1l (D) and gC 2l (D) are specified by sets of integers {O, 1, 7) and {O, 2, 3. 4, 6}; that is, it\D) = 1 + D + D 7 and rP\D) = 1 + D 2 + D 3 + D 4 + D 6 . The rules for forming J = 8 orthogonal parity checks are given by the set { (01), (02), (11), (22), (1232), (2142), (71), (31516162)},

sP)

where the notation (ki) indicates that the syndrome bit forms an orthogonal check-sum on and (ki /i) indicates that the sum + s/il forms an orthogonal check-sum on The J = 8 orthogonal parity checks on e6°> are thus given by the set

e6°l, e6°l.

st

Problems

®i~S

and has error-correcting

This code hss IFB = t1v1L =

:::an acJ:ueve a

majority-logic

o:;·de; m than self-orthogonal bits to disadvantage automatic resynchronization with feedback decoding.

BJl Consider the (2, L 3) encoder

-:21,

-!-

Draw the code tree for

ibJ, Find the codeword

-n,·,· 0 • 0 r,~,n0•.·;r,"_,,,,_, __

J13J; For a Jl.33 Consider the (2, 1, 3) encoder 21, For a BSC with p = lbi, Decode the received sequence rr·

1 -:- [)

+-

+

h = 4. to the information sequence nn = (100 1). Dh1C with equally likely input

metric table for the Fano metric.

= (1 L O 0,

l, 0 0, 0 1. l 0, 1 lJ

the number of decoding steps with the

r,,

.J

= (1

1, l 0, 0 0. 0 L l 0, 0 1. 0 0) .

the final decoded Nith the results of Problem 12.6, where the same received sequence is decoded the Viterbi algorithm. Consider the (2, l, 3) encoder of l?rnblem 21, For the binary-input, Dlv1C of Problem find an integer metric an appropriate factor table for the Fano metric. Scale each metric and round to the nearest fl-Po Decode the received sequence

final decoded path with the result Senne received sequence is decoded using the Viterbi algorithm. BoS Consider the (2, 1. 3) encoder of Problem 13.1. For a binary-input, continuousA WGN channel with = 1, use the stack algorithm and the A V\/GN channel Fano metric frorn 16) to decode the received sequence Il' = (+1.72, +0.93, +2.34, -3.42, -0.14, -2.34. -1.92, +0.23, +0.78, -0.63, -0.05, +2.95, -0.11, the final decoded path with the result of Problem 12.7, where the same received sequence is decoded using the Viterbi algorithm.

686

Chapter 13

Suboptimum Decoding of Convolutional Codes

13.6 Repeat parts (b) and (c) of Problem 13.3 with the size of the stack limited to 10 entrie~. When the st,wk is foll, e:ich :inrlition;:, 1 entry c;:,nsP~ tlw. p:ith nn the

13.7

13.3

13.9 13.10 13.11

B.U

13.13

13.H

13.15 13.16

B.17 1::U8 13.19 13.20

bottom of the stack to be discarded. What is the effect on the final decoded path? ID. Repeat Example 13.5 using the stack-bucket algorithm with a bucket quantization interval of 5. Assume the bucket intervals are . . . + 4 to 0, -1 to -5, -6 to -10, · · ·. lb>. Repeat part (a) for a quantization interval of 9, where the bucket intervals are ··· +8to0, -lto-9, -lOto-18, ···. Repeat Exampie 13.7 for the Fano algorithm with threshold increments of 1' = 5 and 1' = 10. Compare the final decoded path and the number of computations to the results of Examples 13.7 and 13.8. Also compare the final decoded path with the results of the stack-bucket algorithm in Problem 13.7. Using a computer program, verify the results of Figure 13.13, and plot p as a function of Eb/ No (dB) for R = 1/5 and R = 4/5. Show that the Pareto exponent p satisfies limR_,o p = oo and limR_,c p = 0 for fixed channel transition probabilities. Also show that 3R/3p < 0. ID. For a BSC with crossover probability p, plot both the channel capacity C and the cut-off rate Ro as functions of p. (Note: C = 1 + p log 2 p + (1 p) log2 (1- p).) l\:JJ. Repeat part (a) by plotting C and Ro as functions of the SNR E1,/ N 0 . What is the SNR difference required to make C = Ro = 1/2? ru. Calculate Ro for the binary-input, 8-ary output DMC of Problem 12.4. lh. Repeat Example 13.9 for the DMC of part (a) and a code rate of R = 1/2. (Hint: Use a computer program to find p from (13.23) and (13.24).) <e. For the value of p calculated in part (b ), find the buffer size B needed to guarantee an erasure probability of 10- 3 using the values of L, A, andµ given in Example 13.9. ID. Sketch Perasure versus E1,/ No for a rate R = 1/2 code on a BSC using the values of L, A,µ, and B given in Example 13.9. II}. Sketch the required buffer size B to guarantee an erasure probability of 10- 3 as a function of E&/ No for a rate R = 1/2 code on a BSC using the values of L, A, and f.l given in Example 13.9. (Hint: p can be found as a function of Eb/ No using the results of Problem 13.9.) Repeat Problem 13.4 using the integer metric tables of Figures 13.14( a) and (b ). Note any changes in the final decoded path or the number of decoding steps. For a BSC with crossover probability p, plot both the computational cutoff rate Ro from (13.27) and Rmax from (13.35) as functions of p. Find the complete CDFs of the (2, 1, 3) optimum free distance code in Table 12.l(c) and the (2, l, 3) quick-look-in code in Table 12.2. Which code has the superior distance profile? Show that for the BSC, the received sequencer is a codeword if and only if the error sequence e is a codeword. Using the definition of the ]H[ matrix for rate R = 1/2 systematic codes given by (13.42), show that (13.41) and (13.46) are equivalent. Draw the complete encoder/decoder block diagram for Example 13.11. Consider the (2, 1, 11) systematic code with g(l\D) !!lo

llJ,"

=1+

D + D 3 + D 5 + D 8 + D 9 + D 10 +Du.

Find the parity-check matrix ]H[. Vvrite equations fof the syndrnme bits so, st, · · · , s 11 in terms of the channel error bits.

Problems

G~7

'Write for the modified bits sf.s!+i• ··· ,s!+ii• assuming that the effect of error bits time unit l has been removed by feedback. ]JJ;J]_ Consider the (3, 2, 13) code of Example 13.12 and the 4) code of Example 13.13. 21o Find the 1natrix UJJo Find the parity-check matrix lHI(D). <e, Show that in each case G(D)IHIT (D) = QJJ. BJ!Z Consider the (3, 2, code of Example 13.12. !llo "\J;Vrite for the unmodified bits s1, s1+1, · · · , s1+ 13 that include the effect of error bits prior to time unit l l ?: lbio Find a set of parity checks for both e;:J°l and e}1J from the unmodified <eo

Determine the majority-iogic capability t1vn and the effective decoding length 11 E and compare these values with those in Example 13.12. mt Draw the block of the decodeL (Note that in this case the Uv'--VOsUH estimates are not fed back to modify the This alternative to feedback is called definite decoding.) Find a rate R = 1/2 nonsysternatic feedfonvard encoder with the smallest possible value of m such that J = 4 orthogonal parity checks can be formed on the error . (0) (]) bits e 0 and e 0 . Prove (13.65). Prove (13.66). Prove that if the weighting factors w;, i = 0, 1, · · · , J, are calculated using (13.64) and for a BSC 'With crossover probability p, the APP threshold decoding rule is equivalent to the majority-logic decoding rule only if all J orthogonal check-sums include the same number of bits, that is, only if n 1 = 112 = · · · = 11 1 • Consider an (11, k, m) convolutional code with minimum distance d 111 ;11 = 2tps + 1. Prove that there is at least one error sequence If; with weight tps + 1 in its first (m + 1) blocks for which a feedback decoder will decode un0 incorrectly. Consider the (2, 1, 11) code of Problem 13.20. !no Find the minimum distance d 111 ; 11 • !bi, Is this code self-orthogonal? I\'., Find the maximum number of orthogonal parity checks that can be formed on /DJ 0 ill\ Is this code completely orthogona!izable? Consider the (3, 1, 3) nonsystematic feedforward encoder with G 11 ,(D) = [1 + D +D 3 l+D 3 1+D+D 2 ]. filo Following the procedure in Example 13.14, convert this code to a (3, 1, 3) systematic feedforward encoder with the same d 111 ; 11 • lbio Find the generator matrix G, (D) of the systematic feedforward encoder. ro Find the minimum distance d 111 ;11 • Consider the (2, 1, 6) code of Example 13.15. ruo Estimate the bit-error probability P&(E) of a feedback decoder with errorcorrecting capability fFB on a BSC with small crossover probability p. fOlo Repeat (a) for a feedback majority-logic decoder with error-correcting capability fML· 2
1\\

13,23

Bo:Z,:IJ H25 12lo2<1£

BZ7 Jl3o2~

Bo2<1Jl

]33QJJ

12t2l1

B32

688

Chapter 13

Suboptimum Decoding of Convolutional Codes

a. the best rate R = 1/2 self-orthogonal code with d 111 ;11 = 9. lb>" the best rate R = 1/2 orthogonalizable code with dmin = 9. c. the best rate R = 1/2 systematic code with d111 ;11 = 9. llt the best rate R = 1/2 nonsystematic code with dtree = 9. 13.33 Consider an (n, n - 1, m) self-orthogonal code with 11 orthogonal check-sums on

et,

n-

j = 0, 1, · · · , 2. Show that d111 ;11 = J + 1, where J-£ minco:<'J:<'n-2) 11. 13.34! Consider the (2, 1, 17) self-orthogonal code in Table 13.2(a). a. Form the orthogonal check-sums on information error bit lb. Draw the block diagram of the feedback majority-logic decoder for this code. 13.35 Consider an (n, 1, m) systematic code with generator polynomials gCil(D), j = 1, 2, · · · , n - l. Show that the code is self-orthogonal if and only if the positive difference sets associated with each generator polynomial are full and disjoint. 13.36 Find the effective decoding length 11£ for the (3, 1, 13) code of Example 13.19. 13.37 Consider the (2, 1, 11) orthogonalizable code in Table 13.3(a). a. Form the orthogonal check-sums on information error bit lb>. Draw the block diagram of the feedback majority-logic decoder for this code.

e6°l.

e6°l.

BIBUOGRAIP'HY

1. J.M. Wozencraft, "Sequential Decoding for Reliable Communication," IRE Conv. Rec. 5 (pt. 2): 11-25, 1957. 2. J.M. Wozencraft and B. Reiffen, Sequential Decoding. MIT Press, Cambridge, 1961. 3. R. M. Fano, "A Heuristic Discussion of Probabilistic Decoding," IEEE Trans. Inform. Theory, IT-9: 64-74, April 1963. 4. K. Zigangirov, "Some Sequential Decoding Procedures," Prob. Peredachi Inform., 2: 13-25, 1966. 5. F. Jelinek, "A Fast Sequential Decoding Algorithm Using a Stack," IBM J. Res. Dev., 13: 675-85, November 1969. 6. J. L. Massey, Threshold Decoding. MIT Press, Cambridge, 1963. 7. J. L. Massey, "Variable-Length Codes and the Fano Metric," IEEE Trans. Inform. Theory, IT-18: 196-98, January 1972. 8. J.M. Geist, "Search Properties of Some Sequential Decoding Algorithms," IEEE Trans. Inform. Theory, IT-19: 519-26, July 1973. 9!. J.M. Geist, "An Empirical Comparison of Two Sequential Decoding Algorithms," IEEE Trans. Commun. Technol., COM-19: 415-19, August 1971.

10. J.E. Savage, "Sequential Decoding-The Computation Problem," Bell Syst. Tech. J., 45: 149-75, January 1966. llo I. M. Jacobs and E. R. Berlekamp. "A Lower Bound to the Distribution of

Computation for Sequential Decoding," IEEE Trans. !nfonn. Theory, !T-13: 167~74, April 1967.

Bibliography

Uo F, Jelinek, "An

IEEE Trans.

]_,t

Decoding Effort,"

Bound on Moments of IT-15:

and E. K. "A High-Speed Sequential Decoder: Design and Test," IEEE Trans. Commun. Technol., COM-19: 821-35, October 1971. Decoding with a Small 1972.

]_({jjo

(518!JJ

P.R. Chevillat and D. J. Costello, Jr., "An

" Tech of

Time-Invariant Convolutional IT-24: 443-51, 1978. li'!o P.R. Chevillat and D. J. Jr., "Distance and --,J~,u~," IEEE Trans. Commun., COM-24: 440-47, li~t J. A. Heller and t M.

"Viterbi Decoding for Satellite and Space Com" IEEE Trans. Commun. Technol., COM-19: 835-48, October

1971. ]_liJ\ P.R. Chevillat and D. J. Costello, Jr., "A Multiple Stack Algorithm for Era-

surefree of Convolutional 1460-70, Decernber 1977.

"IEEE Trans. Commun., COM-25:

Zilt D. D. Falconer. "A Hybrid Sequential and Algebraic Decoding Scheme," Sc.D.

MIT. Cambridge, 1967. Zlio F. Jelinek and J. Input _,,_.,, .. ,-·~,

Hybrid Decoding for Symmetrical Binary Control, 18: 261-98, April 1971.

'.22, D. Haccoun and M. J. Ferguson, "Generalized Stack Algorithms for Decoding Convolutional Codes," IEEE Trans. Inform. Theory, IT-21: 638-51. November 1975.

23lo A J.P. de A J. and J.P. M. Schalkwijk. "A Class of Binary Rate 1/2 Convolutional Codes That Allows an Improved Stack Decoder," IEEE Trans. Inform. Theory, IT-26: 389-92, 1980. 2tiL R. Johannesson, "Robustly Optimal Rate One-Half Binary Convolutional

Codes," IEEE Trans. Inform. Theory, IT-21: 464-68, July 1975. 2:i R. Johannesson, "Some Long Rate One-Half Convolutional Codes with Distance " IEEE Trans. Inform. IT-22: 629-31, "Some Rate 1/3 and 1/4 Binary Convolutional Codes with an Distance Profile,'" IEEE Trans. Inform. Theory, IT-23: 281-83, March

vu,cuu.1<..,o,;:,v1ll,

1977.

690

Chapter 13

Suboptimum Decoding of Convolutional Codes

27. R. Johannesson and E. Paaske, "Further Results on Binary Convolutional Codes with an Optimum Distance Profile," IEEE Trans. Inform. Theory, IT-24: 264-68, March 1978. 28. M. Cedervall and R. Johannesson, "A Fast Algorithm for Computing Distance Spectrum of Convolutional Codes," IEEE Trans. Inform. Theory, IT-35: 1146-59, 1989. 29. R. Johannesson and P. Stahl, "New Rate 1/2, 1/3, and 1/4 Binary Convolutional Encoders with an Optimum Distance Profile," IEEE Trans. Inform. Theory, IT-45: 1653-58, July 1999. 3@. R. Johannesson and K. S. Zigangirov, Fundamentals of Convolutional Coding. IEEE Press, Piscataway, 1999. 31. G.D. Forney, Jr., "Use of a Sequential Decoder to Analyze Convolutional Code Structure," IEEE Trans. Inform. Theory, IT-16: 793-95, November 1970. 32. P.R. Chevillat, "Fast Sequential Decoding and a New Complete Decoding Algorithm," Ph.D. thesis, HT, Chicago, 1976. 33. J. L. Massey and R. W. Liu, "Application of Lyapunov's Direct Method to the Error-Propagation Effect in Convolutional Codes," IEEE Trans. Inform. Theory, IT-10: 248-50, July 1964. 341. J. L. Massey, "Uniform Codes," IEEE Trans. Inform. Theory, IT-12: 132-34, April 1966. 35. D. D. Sullivan, "Error-Propagation Properties of Uniform Codes," IEEE Trans. Inform. Theory, IT-15: 152-61, January 1969. 36. J. P. Robinson, "Error Propagation and Definite Decoding of Convolutional codes," IEEE Trans. Inform. Theory, IT-14: 121-28, January 1968. 37. T. N. Morrissey, Jr., "A Unified Markovian Analysis of Decoders for Convolutional Codes," Tech. Rep. EE-687, Department of Electrical Engineering, University of Notre Dame, October 1968. 38. D. J. Costello, Jr., "Free Distance Bounds for Convolutional Codes," IEEE Trans. Inform. Theory, IT-20: 356-65, May 1974. 3§»o L. D. Rudolph, "Generalized Threshold Decoding of Convolutional Codes," IEEE Trans. Inform. Theory, IT-16: 739-45, November 1970. 40. J.P. Robinson and A. J. Bernstein, "A Class of Binary Recurrent Codes with Limited Error PropagaTion," IEEE Trans. Inform. Theory, IT-13: 106-13, January 1967.

algorithms reducused trellis-based 12. Convolutional used for error control to be used for many M,,,,vv'""'

"'-""un,.u·-'"

rn

with a

years to some. This

for linear block to be presented the Viterbi algorithm JS convolutional however. the on the sec:tionalization of a code based on a divideinto short which are one for each section. These metric tables are then combined form ,n.etric t::ibles for longer sections until a metric table for the entire code 1::; for:,i1ed. This final table then contains the rnost codevvord and its Metric tables for two adjacent code sections are the two sections. This received sequences to to be presented is subtreHis for a code.

to minimize the bit-error decoded bits. Also included

vvv~,"u"' linear block codes with the Viterbi 1s straightforward. If an 11-section bit-level trellis T for an (n, k) linear code is used for decoding, the decoding process is the same as that for a terminated convolutional code. The decoder processes the treHis frorn initial state to the final state serially, section section. The survivors at each level of the code trellis are extended to the next level the branches betvveen the two levels. The paths that

~92

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

enter a state at the next level are compared, and the most probable path (or the path with the largest correlation metric) is chosen as the survivor. This process continues until the end of the trellis is reached. At the end of the trellis, there is only one state, the final state sf, and there is only one survivor, which is the most likely codeword. Decoding is then completed and the decoder is ready to process the next incoming received sequence. The number of computations required to decode a received sequence can be enumerated easily. In a bit-level trellis, each branch represents one code bit, and hence a branch metric is simply a bit metric. The total number of additions required to extend the survivors from level to level is simply equal to the total number of branches in the trellis, which is given by (9.46). Let Na denote the total number of additions required to process the trellis T. Then, 11-l

N0

= L2P, · /;(a*)

(14.1)

i=O

where 2P; is number of states at the ith level of the code trellis T, a* is the current input information bit at time-i, and /; (a*) is defined by (9.45), which is either 1 or 2. If there is an oldest information bit a 0 to be shifted out from the encoder memory at time-i, then there are two branches entering each state s;+1 of the trellis at time-(i + 1). This says that there are two paths entering each state s;+1 at the (i + l)th level of the bit-level trellis T. Otherwise, there is only one branch entering each state s;+1 at time-i. Define (14.2)

where A/ is the state-defining information set at time-i. Let Ne denote the total number of comparisons required to determine the survivors in the decoding process. Then, 11-l

Ne= I:2Pi+l. J;(ao).

(14.3)

i=O

Therefore, the total number of computations (additions and comparisons) required to decode a received sequence based on the bit-level trellis T using the Viterbi decoding algorithm is Na+ N,. IEXAMP'LIE UU Consider the (8, 4) RM code whose 8-section bit-level trellis is shown in Figure 9.6. From Table 9.2, we find that Io(a*) = /1 (a*)= h(a*) = /4(a*) = 2. h(a*) = /5(a*) = h(a*) = h(a*) = 1, 0

Jo(a 0 ) = 11 (a ) = h(a 0 ) = J4(a 0 ) = 0,

Section 14. 1

dimension we find that Na = 2° · 2 + 2 1

N, = 2 · 0 +

1

·

2 +2

·0 +

The Viterbi

of the trellis is (0. 1, 2, 3, 2, 3, 2, 1, 0). 2

2+

·1 +

·2 +

· 1 + 22 · 1 + 21 · 1

·0 +

·1 +

·0 +

· 1 + 2 1 · 1 + 2° · 1

·

from

=lL 44 additions and 11

the bit-level trellis T is sectionalized into v sections with section boundary locations in A = {to, 1\. · · · , tv}, where O = to < t1 < · · · tv = n. This sectionalization results in a v-section trellis At location t;, the state space is ( C). The ith section of consists of the state space 1: 1;_ 1 the state space (C), and the branches that connect the states in 2: 1; _ 1 ( C) lo the states in l'. 1; A branch in this section t; - t; -1 code bits. Two states may be connected called ,w,w·w, we say that these parallel branches form a Viterbi decoding based on the v-section trellis T(A) is carried out as follmvs. the decoder has processed i trellis sections up to time-t;. There are I (C) I one for each state in I: 1, (C). These survivors together with their metrics are stored in the decoder memory. To process the (i + l)th the decoder executes the

JL Each survivor is extended to the next level at time-t; +l ·

the

branches diverging from it

::t

branch a state in :Er;+i the branch with metric is found. This metric is the branch metric. 2t branch is by the branch 'Nith the largest metric. ti!o The metric of a branch is added to the metric of the survivor from which the For each states E I:r;+i (C), the metrics of paths entering it are and the path with the largest metric is selected as the survivor terminating at states. the

The decoder executes these until it reaches the final state Sf. The sole survivor is the decoded codeword. The total number of ( additions and required to process a sectionalized trellis T(A) depends on the choice of the section boundary location set l\. = {to, t1, · · · , t 0 ). A sectionalization of a code trellis that gives the smallest total number of computations and comparison are considered to have the same is called an optimal sectionalization for the code. An for finding an optinial sectionalization has been devised Lafourcade and Vardy [3]. This algorithm is based on a simple structure: for any two integers x and y with O :':: x < y :':: n, the section from time-x to time-y in any sectionalized trellis T(A) with x. y El\. and x + 1, x + 2, · · · , y - 1 ~ l\. is identical. Let cp(x, y)

694

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

denote the number of computations required in steps 1 to 4 of the Viterbi decoding algorithm to process the trellis section from time-x to time-y. This number cp(x, y) is determined solely by the choices of x and y. Let (/Jmin (x, y) denote the smallest number of computations required in decoding steps 1 to 4 to process the trellis section(s) from time-x to time-yin any sectionalized trellis T(A) with x, y EA. The value (/Jmin (0, n) gives the total number of computations of the Viterbi algorithm for processing the code trellis with an optimum sectionalization. It follows from the definitions of cp(x, y) and (/Jmin(x, y) that . (O (/Jmzn

) _ { min{rp(O, rp(O, 1),

,Y -

y), mino<x
+ cp(x, y)}},

for 1 < y :S n (l 4.4) for y = 1.

For every y E (1, 2, · · · , n}, (/Jmin (0, y) can be computed as follows. The values of cp(x, y) for O :S x < y :S n are computed using the structure of the trellis section from time-x to time-y. First, (/JminCO, 1) is computed. The value (/Jmin(O, y) can be computed from (/Jmin(O, x) and rp(x, y) with O < x < y using (14.4). By storing the information when the minimum value occurs in the right-hand side of (14.4), we find an optimum sectionalization from the computation of (/Jmin (0, n). In general, optimum sectionalization of a code trellis results in a computational complexity less than that of the bit-level trellis. For example, consider the second-order RM code of length 64 that is a (64, 22) code with minimum Hamming distance 16. Using the Lafourcade-Vardy algorithm, we find that the boundary location set A = (0, 8, 16, 32, 48, 56, 61. 63, 64} results in an optimum sectionalization. The resultant trellis is an eight-section trellis. With this optimum sectionalization, (/Jmin(O, 64) is 101,786. If the 64-section bit-level trellis is used for decoding, the total number of computations required is 425,209. Therefore, optimum sectionalization results in a significant reduction in computational complexity. In general, optimum sectionalization results in a nonuniformly sectionalized trellis in which the section lengths vary from one section to another. This nonuniformity may not be desirable in IC hardware implementation of a Viterbi decoder [6, 8, 9]. Suppose the bit-level trellis T for the (64, 22) RM code is uniformly sectionalized into an 8-section trellis with the section boundary location set A = {O, 8, 16, 24, 32, 40, 48, 56, 64}. Each section consists of 8 code bits. Viterbi decoding based on this uniform 8-section trellis requires a total of 119,935 computations, which is larger than that based on the foregoing optimum sectionalization but is still much smaller than that based on the bit-level trellis. If a code trellis consists of parallel and structurally identical subtrellises without cross connections between them, then it is possible to devise identical, less complex Viterbi decoders to process these subtrellises in parallel. At the end, there is one survivor from each subtrellis. The survivors from these subtrellises are then compared, and the one with the largest metric is chosen as the final survivor. This parallel processing of subtrellises not only simplifies the decoding complexity but also speeds up the decoding process. For example, the 4-section trellis for the (8, 4) RM code shown in Figure 9.17 has two 2-state parallel and structurally identical subtrellises; hence, two identical 2-state Viterbi decoders can be devised to process these two subtrellises in parallel. For a large code trellis, parallel decomposition, presented in Section 9.7, can be carried out to decompose the trellis into a desired number of parallel and structurally identicc1l sllhtrellises.

Section 14.2

Recursive Maximum Likelihood

Because the trellis for a block code can be bidirectional. To facilitate the a codeword v = (vo, v1, · · · , v11 _1) is first into the sequence (vo, v11 _1, v2, v 11 -2. · · ·) before its transmission. The ing received sequence is (ro, r 11 _1, r1, r 11 _2, ···).From this received sequence, two (ro. r1, · · · . r 11 _1) and (r11 _1, r11 -2. · · · , ro). These two received sequences are received sequences are then fed into the code trellis from both ends for decoding. two decoders are used for decoding in both directions. When two decoders meet at the middle of the a decision is made. The with the n1etric is chosen as the decoded codeword. U the code trellis has mirrorthe two decoders are identical. This bidirectional decoding further and up the decoding process. A high-speed RM subcode has been based on both structures of the code [9, 10].

to be presented in this section is based on a divide-andconqueiAn (11. k) linear block code C is first divided into short sections. The distinct vectors in each section form a linear code. These vectors are processed based on the section of the received sequence. For each "'"""HJH, we eliminate the less probable vectors and save the vectors and their metrics in a table. we combine the metric tables of two neighbor sections to form a metric table for a longer section of the code. In the combining process, we eliminate the less probable vectors. The resultant table contains the vectors and their metrics for the combined sections. Continue the process until the full length the code is reached. The final table obtained contains one survivor and its metric. This final survivor is the most codeword with to the received vector and hence the decoded codeword. Because the table process is carried out such a MLD algorithm is called a recursive MLD (RMLD) The first such decoding algorithm was devised in [14, 15].

Let T be the minimal 11-section trellis diagram for an (n, k) binary linear block code C. Consider the trellis section from time-x to time-y with O :s x < y < n, denoted .y· For two states s, and Sv with sx E I:x(C) and Sy E I:y(C), if sx ands,. are connected, the set of paths connecting state sx to state Sy, denoted L(sx, sy), is a coset in the Px.y(C)/ .i" For convenience, L(sx, Si-) is regarded as a called a composite path. Because each composite path L(s,. si-) in T,.v is a coset in Pr.y(C)/ the number of distinct composite paths in T,.v is 2k(p,. 1 (C))-k(C, 1 ).

(14.5)

Based on the trellis structures in Section 9.4, it is possible to show that each coset in Px.y(C)/C.~\ appears as a composite path in .y 2k-k(Co.,)-k(C",,)-k(p,_ 1 (C))

times [8, 11].

(14.6)

696

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

For each distinct composite path L(sx, sy) in Tx,y, we find the path with the largest metric with respect to the corresponding section of the received sequence, (rx, rx+l, · · · , ry-1). Let l(L(sx, sy)) and m(L(sx, sv)) denote this path and its path metric, respectively. For convenience, we call l(L(sx, sy)) and m(L(sx, sy)) the label and metric of L (sx, Sy), respectively. We form a table that stores the label and metric for each distinct composite path L(sx, sy) in Tx,y (or a coset in Px,y(C)/C~'--y). This table is called the composite path metric table, denoted by CPMTx,y, for the trellis section Tx,y (or for the partition Px.y(C)/Crv). Because the partition Po. 11 ( C) / Co. 11 = C / C consists of C only, CPMT0. 11 contains only the codeword in C that has the largest metric. This is the most likely codeword with respect to the received sequence ll' = (ro, r1, · · · , r 11 _1). When CPMTo. 11 is constructed, the decoding is completed, and CPMT0. 11 contains the decoded codeword. A straightforward method for constructing CPMTx ,y is to compute the metrics of all vectors in the punctured code Px,y(C) and then to find the vector with the largest metric for every coset in Px ,y ( C) / C~': v by comparing the metrics of vectors in the coset. This direct construction method is efficient only when y -- x is small and should be used at the beginning of the recursion process. When y - x is large, CPMTx,y is constructed from CPMTx.z and CPMTz.y for a properly chosen integer Z With X < Z < y. Table combining is based on the trellis connectivity structure developed in Section 9.4 and shown in Figure 9.11. Consider two states sx and sy in the code trellis T with sx E bx(C) and Sy E by(C) that are connected by the composite path L(sx, Sy), Let (14.7)

denote the subset of states in I;z(C) through which the paths in L(sx, sy) connect state sx to state sy, as shown in Figure 14.1. Then, it follows from (9.28) and (9.29)

Time-x Time-z FIGURE 14.1:

Connection between two states.

Likelihood

Recursive

Section

©91

Algorithm

that o

L(sjil.

(14.8)

EZ:::(s. 1 .s,.)

(i)

·'ivhere lS the

, and L(s: , sv) () . and L(s:' , sy) It follows from

m(L(s,.

0

Ii S;

c(il\\ L).::: ;)

+ ('11

(14.9)

. s,.))}.

.,\\}

and (14.10)

• sYma\) )) o

takes its maximum. vvhere inu;x denotes 1 0 ·~r l~bel I "Umux))) and )) Because the metrics m ' S "\' / .._a .ILJL .... ~ a,e stored in CPI/ff,: ,mcl C:PMT:.,, (14.9) and (14.10) shovv that the CPMT,. 1. for T,. 1. can be constructed from CPMTc: and CPivIT:.y for the trellis sections T,.: and T::. 1 . for block codes m Section 9.,t ·vvs can s~t . sy) is given Ci.

•

,. , )-k(C,., )-k(C,. ,)

/,(=I

. sy))

,).:::

(14.11)

frrnm ( CPMT,,: and H follows from (14.5) that

(Ci)-k(C, ,J.

-1).

hovvever, if the metric we need to Because

u

(14.12)

from the

m

in L (sx, Sy),

er

I ,.1-1 comparisons. x - 1 additions, the total . s,.)) is

v,.) - 1 =

(v --

) -1.

(14.13)

which is much than - l l)) for y - x. Hence, constructing the metric table CPh1T,., from CPMT,.: and CPJ.\ij[T:., many fewer than direct construction of CPf
=

D.

E

JJ,. 1 (C)/C~r,..

a state

(s,. s1 ) such that

698

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

'.t Determine the state set :EJsx, sy). 3. Compute the metric m(L(s.r, and label l (L(sx, respectively.

from (14.9) and (14.10),

This process requires constructing the entire code trellis and examining the section from time-x to time-y. For a long code, it is difficult to construct a large trellis T and examine the section Tx,y· Furthermore, the total number of composite paths in Tx.y can be very large, with only a small fraction of distinct composite paths. Examining Tx,y to execute steps 1 to 3 can be very time-consuming and inefficient. Consequently, decoding implementation win be complex and costly. To overcome the complexity problem and to facilitate the computation of (14.9), we construct a much simpler special two-section trellis for the punctured code Px.y ( C) with section boundary locations at x. z. and y that has multiple "final" states at time-y, one for each coset in Px.y(C)/C~':v· This special two-section trellis contains only the needed information for constructing the metric table CPMTx.,· from CPMTx ~ and CPMT~.y· Let T({x. z. y)) denote this special trellis. Let :E: and z:;Y denote the state spaces of T ( {x, z. y}) at time-z and time-y. respectively. We require that T ( {x, z, y}) have the following structural properties: lo It has an initial state, s.r.O, at time-x. 2o There is a one-to-one correspondence between the states in I:- and the cosets in Px.~(C)/Ct~- Let D~ denote a coset in Px.JC)/Ct~· and s(DJ denote its corresponding state at time-z. Then. the composite path connecting s.r.O to s(DJ is L(sx.O, s(DJ) = D~. :?t There is a one-to-one correspondence between the states in the state space :E,. and cosets in Px.v(C)/C~':v· Let denote a coset in Px.,.(C)/C~':v• and s(D)') denote its corresponding state at time-y. Then, the composite path connecting the initial state sx.O to state s(Dy) is L(sx,O, s(Dy)) = Dv. Ha state s(DJ at time-z and a state s(Dv) at time-y are connected, then the composite path L(s(DJ, s(D_, )) is a coset in A.v(C)/ C2v· Every coset in P:.y (C)/ C~'.,· appears as a composite path between a state in t~ and a state in :Ey. · 4L For every state s (Dy) at time-y, there is set ofµ states in :E~ that connect the initial state sx.O to state s(Dy)- Let

(14.14)

denote this set of states. Then, Dy= L(s.r.O, s(Dv))

u

L(sx,O, s(D~i))) o L(s(D?l), s(D,)).

(14.15)

From the structural properties of the special two-section trellis for the punctured code Px,y(C) we see that (1) for every state s(DJ E :E:, its state metric (or composite path metric) m(L(s,:,o, s(DJ)) and label /(L(.1·x,O, s(DJ)) are given in metrir t::ible CPMT,,::, and (2) for each composite path betwee!"! a state s(DJ at tirn,c;:._"'7

0,nrl

<:ln

,..,,,-1~.,,.-,.nnt ,:,fn,h::.

... .A...11.JU.. "-'

.._......_..._..__~

-.... .... . 11.

i.t..__~jU'--v.li..n.,i.,

;,.. •

.._,,..,._~._...,

.,fn \ ,.._,,,.__,y,

.-.t +;,,Ytn.-u l'...u..

"-"--"--'-aV

_r,

;+ .. ..,....,..=1;-,,~~,.,. ,~,,fl f,..,fn \ .1.1..D

J..J...J1......,11....1L..11...._,

,,/n \\\ 111·\~\•'\L/::_.J, •J\JL/_VJJJ

..... ~--.A u..1...11..11...11

Section 14.2

A Recursive Maximum Likelihood

label l(L(s(DJ. s(D 1.))) are of the

ll1

.o. s(Dy)))

in CPMT:.v· H follows from (14.15) and the two-section trellis T ( {x. z. y}) that for each cos et o, s(Dy))) is

=

max -\-

{m.

(s(D:'() ), s(Dy)))}.

is to the metric table CPMT,. 1 can be constructed from CPMT,.~ and CPMT:.y the two-section trellis T({x. z. y}) for the code Px (C). For each state s(Dy) E the state set :E~ .o. s(Dy)) can be from the trellis T({x. z, y}). fo this treHis T ( {x, z, y}) is much than the section Tu of the entire code trellis T from time-x to the construction of CPMT,. 1 is much two-section trellis T z. y}) for the code Px.y(C) is constructed as follows 15]. We choose a basis {~1. g2, · · · . ~k(p, ,(C))} of p 11.(C) such that the first k(C~'.'y) = k(C: vectors form a basis of .• Let 1

cr

D.

/Jr.y = y - x

We form the

k(pu-(C)) ><

+ k(Px.y(C)) -

k(Cx.y).

(14.17)

n,.y matrix: g1

gk(C,.,)

(14.18)

CG(x, y) = ~k(C ,)+l

l '/l;k(p,_,(C)J

where I(]) denotes the k(C, 1 ) x (k(P,.v(C)) - k(C1 .y)) all-zero matrix, and l denotes the matrix of dimension k(Px.v(C)) - k(C,.v). Let y) be the linear code of n,.v by y). We construct a three-section trellis , z. y, x + llul) for C(x. y) with section boundary locations at x. z. y. and :'C + /1cy, as shown in Figure 14.2. the first two sections of T ({x. z, y. , + n,.y}) the desired two-section trellis T({x, z, y}) for JJ,.y(C). In we do not need to complete the trellis T( {x, z. y. x + llu·}). We need construct the first two sections from time-x to time-y. In from (14.16) and lhe structural properties of T({x. z, y)), we need the second section of T({x, z, y}) to carry out the computations of (14.16) and to form the metric table CPMTu. For we denote this one-section trellis (z. y). We assign each state s(D:) of T, _v) at a metric m(L o. s(DJ)) and a label l(L(sx.O, s(DJ)) based on CPMT,.:- Table CPMT~ 1 gives the composite

700

Chapter 14

2k(p._J C)) -k(

Trellis-Based Soft-Decision Decoding Algorithms

c,..J States

Time-z

Time-y

t

t

~~

/ / / /

/ /

/

2k(1>,.JCJ)-k(c, .., l states

in P,.J C)

A coset of - - A coset of C:'., inp,.,(C)

FIGURE 14.2:

Structure of the trellis diagram T({x, z, y, x

+ nx,y}),

path metrics of Tx(z, y) between time-z and time-y. With all this information, we can easily construct CPMT,. 1 by carrying out the computations of (14.16). EXAMPLIE 14.2

Let C be the RM(2, 4) code, a (16, 11) code, whose trellis structure we studied in Example 9.10. Set x = 4. y = 12, and z = 8. Then, P4, u(C) = RM(2, 3), Ci\ 2 = RM(l, 3), Ci'.s = q 1_12 = RM(O, 2), and 114_12 = 12. C(4. 12) is the (11, 9) linear block code generated by

G(4, 12)

I ~

1 0 0 0

1 1 0 0

1 0 1 0

1 1 1 0

1 0 0 1

1 1 0

0 0 0

0 0 0

0 0 0

1

0 0 0

0

0 0

l

1

1 0

1 0 1 1

1 1 1 1

0 0 0 0

0 0 0 0

0 0 0 0

0 0 1

1

1 1

1 0 0

0 1 0

0 0 1

By simple row operations, we can put G(4. 12) in trellis-oriented form. The onesection trellis T4 (8, 12) consists of two 4-state parallel and structurally identical subtrellises, called parallel components. One of the components is depicted in '1r1r1ina (()()()1'\ bhPl -FicmrP -o~--~ 141 - '"~·· ThP ~--~ nthPr . . . __ .._._ r,m .... ~--- hP __,. nhtciinPrl __,__,_~------'-- hv '-'J -·-----·b ,,_,'--...,,- · tn .,,__,, P'.>rh ---·--- .... h,-c,nrh .....,, __ _.._ ____ ...__,._, ___ Frnm - _._~--~

Section 14.2

A Recursive IVlaximum Likelihood

Algorithm

7iOJ1

Time y = 12

Time:= 8

P 1 £ 10000.111 Jl. P, £ IOOlUlOOI P, £ IOlOL 1010j.P 1 £ I0110J001l

FIGURE 14.3: A

parallel component of T4(8, 12) for the RIV/(2,

code.

14.3 we can identify the state set I: 8 (s4_0, s(D12)) for each state s(D12) at time-12. Each set consists of 4 states at time-8. 14.2.2

AW11 11MW Ji\!gi
Based on the method for constructing composite path metric tables, a RMLD algorithm can be formulated [15]. Let RMLD(x, y) denote the recursive algorithm for constructing the composite path metric table CPMTx.y for Px,y(C)/ This algorithm uses two procedures, denoted by MakeCPMT(x, y) and , y; z), to construct CPMT, ..,. The MakeCPMT(x, y) procedure constructs CPMTx.y directly from vectors in Pu(C). The CombCPMT(x, y; z)

102

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

procedure constructs CPMTx.y by combining tables CPMTx.z and CPMT;;,_v using (14.9) (or (14.16)) and (14.10). where x < z < y. Because tables CPMTx.; and CPMT;;,y are constructed by executing RMLD(x, z) and RMLD(z, y), we can express the procedure CombCPMT(x, y; z) in recursive form: CombCPMT(RMLD(x, z),RMLD(z, y)). Algorithm RMLD(x, y) is used as follows: Construct CPMTx.y using the less complex one of the following two options: 1. Execute MakeCPMT(x, y). 2. Execute CombCPMT(RMLD(x, z),RMLD(z, y)), where z with x < z < y is selected to minimize computational complexity. Decoding is accomplished by executing RMLD(O, n ). To start the RMLD algorithm to decode a code, we first divide the code into short sections. The metric table for each of these short sections is constructed by executing the MakeCPMT procedure. This forms the bottom (or the beginning) of the recursion process. Then, we execute the RMLD(x, y) algorithm repeatedly to form metric tables for longer sections of the code until the table CPMT0, 11 is obtained. Figure 14.4 depicts such a recursion process. We see that the RMLD algorithm allows parallel/pipeline processing ofreceived words. This speeds decoding process. As pointed out earlier, the simplest and most straightforward way to construct the CPMTx._v directly is to compute the metrics of all the vectors in the punctured code Px.y(C) independently and then find the vector with the largest metric for each coset in Px.y(C)/ by comparing the metrics of vectors in the coset. Each surviving vector and its metric are stored in the CPMTx,)" Let MakeCPMT-I(x, y) denote this make-table procedure. The computational complexity of this procedure can be easily evaluated. There are 2kCv,.,(C))-k(C, ,l cosets. The number of computations required to determine the metric of each coset is given by (14.13). Then, the total number of computations required by the MakeCPMT-I(x, y) procedure is

c;r,,

wi\x, y) = 2kCv,.,(Cll-kCC,,l. [(y-x)2k(C,,) -1].

FIGURE 14.4:

(14.19)

Illustration of the recursion process of the RMLD algorithm.

Section 1

1(())3:

of (14.9) (or one-section trellis (z, y). A the conventional add-compareFor each coset D,. in . s(D,.))) is computed

.o. s(D 1.))) is found by is called the CombCPMTby this procedure to construct CPMT,.r .,(C)J-kCC.,J

, )-k(C, ,J-k(C ,l+l

-1].

(14.20)

y) and CombCPMT-

The

V) algorithm . Because both MakeCPMT . y) and Com_bCPI\/lT. y: z) are very simple, the RMLD-(I, either software or hardware. A""""'~,,,· can be devised if detail structme of the one-section trellis , v) is used. Such a procedure has been . y; z)

in [15].

From and we can if y - x > 2, then for any requires fewer choice of z with x < z < _v, the CombCPlvJTto form the CPlVff,. 1 irhan the y) procedure: and (2) if y - x = 2, the . y; x + 1) and that of MakeCPMT says that the MakeCPMT-I(x, y) is efficient when v - _, is sn121H and when used at the beginning to form metric tables to start the recursion process. Vvhsn _v - x is large, the CombCPMTV(x. y; z) should be used iG construct metric tables. At the beginning of the recursion process . y - x is small and few are done by the MakeCJFMT-I(x. v) the entire process. Therefore, the . y; z) procedure.

The overall

depends on the A sectionalization that called an optimum

results in the smallest overall sectionalization [15]. Let 'lYmin (x. y) denote the smallest number of ( additions and to construct the CPlVIT.cr· An addition operation and a comparison are assumed to have complexity). H foHows from the RMLD(x. section that ,,,

"'min

(

X. V

.

) Ii

!

= I,

if y =

1V1 (x, y).

X

(I)

min{WM

+ 1,

(14.21)

where (x. z)

+

y)+

(x, y; z).

(14.22)

104

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

TABLE 14.1: Computational complexities of the RMLD(I,V) algorithm and the

Viterbi algorithm for

some RIVI codes.

Vnteirll:»ii ih>ID§e<W l{])Jlll 64l·§tedfollll tlireHiln§

JRMJLI[J).09V)

Vntle1ribn ibJID§ei!li l{])Jlll lLIDfounrrteraiidle- V IDrrirlly IDilgl{])rci.ftllnll1l11

RM(2, 6) (64, 22)

425,209

78,209

101,786

RM(3, 6) (64, 42)

773,881

326,017

538,799

RM(4, 6) (64, 57)

7,529

5,281

6,507

Cl{])idle

Then, the total number of computations required to decode a received word is given by lllminCO, n). Using (14.19) through (14.22) we can compute IJlmin(x, y) for every (x, y) with 0 :S x < y :S n as follows: We compute the values of \l'inin (x. x + 1) for O :S x < n using (14.19). For an integer i with O :S x < x + i :Sn, we can compute IJlmin (x, x + i) from 'Vmin(x'. y') with y' -x' < i, (14.19), and (14.20). By keeping track of the values of z thus selected we can find an optimum sectionalization. The RMLD-(I.V) algorithm with optimum sectionalization is more efficient than the Viterbi algorithm based on the Lafourcade-Vardy's optimum sectionalization [3]. A few examples are given in Table 14.1. From Table 14.1 we see that both the RMLD-(I,V) and Viterbi-Lafourcade-Vardy algorithms are more efficient than the Viterbi algorithm based on the bit-level trellis. The RMLD-(I,V) algorithm not only is more efficient than the Viterbi-Lafourcade-Vardy algorithm but it also allows parallel/pipeline processing to speed up decoding, which is important in high-speed communications. 14.3

A SUIBOPTIMUM mEIF!.AlN'IE IDIECOrnNG AlGOmTHM i8AS1EID ON A lO'W-'WHGHl S11Ji8llF!.1ElUS

Trellises for long block codes have very large state and branch complexities. These complexities grow exponentially with code length. Consequently, maximum likelihood decoding of long codes based on full-code trellises would be very difficult if not impossible to implement. For this reason it is very desirable to devise trellisbased suboptimum decoding algorithms to provide an efficient trade-off between error performance and decoding complexity. This section presents a suboptimum iterative decoding algorithm based on low-weight subtrellis searches that was proposed in [17]. This algorithm consists of three simple steps. First, a simple method is used to generate a sequence of candidate codewords, one at a time, based on the reliability information of the received symbols. When a candidate codeword is generated, it is tested based on an optimality condition. Hit satisfies the optimality condition, then it is the most likely codeword, and decoding stops. If the optimality test fails, then the smallest region centered on the tested candidate codeword that contains the most likely codeword is determined. H this region is small. a search is conducted through a low-weight

Section 14.3

Iterative

A

subtreHis for the such as Viterbi or RMLD. If the search is too large. a new candidate codeword is generated. and the test and search are renevved. This process continues until either codeword is found or candidate codewords are exhausted ( or a and the process is then terminated.

Let C be a (11, k) linear code with mmmmm distance dmin to be decoded. Let ~c == (ro, r1, · · · , r 11 _1) be the soft-decision received sequence and 12 = (zo. z1, · · · . z,,_i) be its hard-decision vector. A method for candidate codewords is to use a set of test error the hard-decision received vector ZI and then decode each modified vector determines the and effectiveness of the ~"~~~·~rn,~ should be used as the test error be used in likelihood order to the candidate codewords for test. Let p be integer no than 11. and QP denote the set of the p least reliable of the soft-decision received sequence n· as discussed in Section 10.2. Let E denote the set of 2P error of 11 with errors confined to the in Qp. The error in this set are more to occur than the other error fo Chase decoding [18], as in 10.4.2, p is chosen to be Ldmin/2J, and E consists of 2Ld",; 2J test error fo the following Chase is used to generate candidate codewords for the 11

1~3Ui

/

i(j)~ru:imc!?l!i'it)J l!®s;'1: aJ[(JID1 SeZJn:lu ~egi©[(J The conditions in Section 10.3 are used to test the candidate codewords. Let Wk be the kth nonzero in the weight profile W = {wo = 0, w1 • ...• w111 } of code C. Let Y be a candidate codeword for test. We define 6.

-

Pk

G

6.

Wk)

u)k -

ID1 (w)I.

I:

Ir; I,

iEDi/'1"l(V)

and R(w.

w,J

!:,.

I

I

= {w EC: d(w, v) < Wk},

(14.25)

where sets D1 and vt\w) are defined in Section 10.3 ((10.23) and (10.29)), and d(v', v) denotes the Hamming distance between w' and v. Then, we can restate the condition given by (10.33) (Theorem 10.1) as follows: for a candidate codeword w E C and a nonzero weight Wk E if the correlation >..(r,, v) between Y and the received sequence Ir satisfies the condition >..(Ir, v) .=::: G(w:

w,J.

(14.26)

then the most codeword 'YlVJL is in the region R(v, w,J. The condition given by (14.26) simply defines a region in which the most likely codeword v ML can be

706

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

found. It says that VML is among those codewords in C that are at distance less from the candidate codeword v; that is,

Wk-1

or

(14.27) If wk-1 is small, we can search in the region R(v,

wk) to find VML· If wk-1 is too large, then it is better to generate another candidate codeword v' for testing in the hope that the search region R(v', wk) is small. From (14.26) we readily see that

JL If k = 1, the candidate codeword vis the most likely codeword '¥ML· 2, H k = 2, the most likely codeword VML is either the candidate codeword v or a nearest neighbor of w. Therefore, we have the following two optimality conditions: L If )c(ir, v) =::: G(w, w1), then w = "'ML· 2, If G(w, w1) < A(ir, v) =::: G(v, w2), then minimum distance dmin = w1 from w.

'VML

is at a distance no greater than the

The first condition is a sufficient condition for optimality of a candidate codeword. The second condition is a sufficient condition for v ML to be a nearest neighbor of a tested candidate codeword. We call G(w, w1) and G(v, w2) the optimality and nearest-neighbor test thresholds, respectively. They are used for testing a candidate codeword to determine whether it is the most likely codeword or a search for the most likely codeword among its nearest neighbors is needed. 14.3.3

An iterntive Derndin11g A!gmithm Based «:m a Min11imum-Weight Trems Search

Suppose a candidate codeword w is tested, and the most likely codeword VML is found in the region R(v, w2). Then, '¥ML can be found by searching through the minimum-weight subtrellis Tmin (v) centered on v using either the Viterbi or the RMLD algorithm [17]. The decoding algorithm is iterative in nature and consists of the following key steps: l, A candidate codeword v is generated by using a test error pattern e in the set E. 2, The optimality test or the nearest-neighbor test is performed for each generated candidate codeword v. 3, If the optimality test succeeds, decoding stops. If the optimality fails, but the nearest-neighbor test succeeds, a search in the region R(v, w2) is initiated to find the most likely codeword vML. 4, If both tests fail, a new candidate codeword is generated.

Suppose the most likelihood VML has not been found at the end of the (j - l)th decoding iteration. Then, the jth decoding iteration is initiated. Let 'Vbest and )c(r, 'Vbest) denote the best codeword and its correlation discrepancy that have been found so far and are stored in the decoder memory. The jth decoding iteration consists of the following steps:

Section 1Ll.3

Iterative

§1leJID JL Fetch ei from E and decode ei

go to 2,

w1),

YML

+ ;z

into

2; = v; stop the

'-'"'"U'L«,u"'

E

C. H the

process.

search T,11 i 11 (v) to find VML and stop the go to step 4. §1lrBJID I!:,, A(il', '¥!Jest), '¥/Jest with V and A(rr, "!best) by A(Il', go to 5. §, U j < IE I, go to the codeword with the least correlation r,,,,,.v'"''''"''" w2),

The process is depicted the flovvchart shown in Figure 14.5. It at most one trellis search. The case for which the decoded codeword is the most codeword wML is the output from the search of the trellis centered at 'ilbest· For this decoding algorithm, Uv',,,Vo,,rnc
z = hard decision on received vector n· compute syndrome § of z. A1w,, = 08

IGenerate first lest pattern e in iE I ',,,,~ >:~

I Algebraically decode (7l + e) to iiiiiJ I yes ,v , , ·~-(Decodmgliilli1)

·~ Compute index sets D 0 (v), D 1(v)

V Sort elements of D 0 (v) Compute A(r, v), G(v; w 1), G(v; w 2 ) V (IsA([,v)sG(v;w 1) )

yes ~ ·

...

~-,'>-~>JStop\

J

I > Search 'JI',,,,,, (v) centered around v ~ ~ to obtain optimum v,1n If A(r, v) < A1,,,,. then set v1i('\, = v and A-tiec;f = A(n. v)

Have all test patterns in E been used ?

yes Search 'JI',,, 1,, (v1,cs,) centered at > v1,,,, and return codeword having · ..>\Stop\ minimum discrepancy with n·

Generate next test pattern e from E

FIGURE 14.5:

Flowchart for the iterative decoding algorithm.

108

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

10() ,,..._-----.-----r------,,-------,-------,-----,,

*ChaseA!g II + RM Iter w 1-weight search 1'RMw2 oRM Opt

10-7~---~1---~-----'----~---~-----' 0

2

4

6

8

10

12

E 1,IN11 ,dB

FIGURE 14.6: Error performances of the iterative decoding ( w1 -, w2-weight trellis search) for the (64, 42, 8) RM code.

Because this iterative decoding algorithm uses the Chase algorithm-2 to generate candidate codewords for the optimality test and minimum-weight trellis search, it may be regarded as an improved Chase algorithm-2 and hence achieves asymptotically optimal (MLD) error performance with a faster rate. Consider the (64, 42) RM code with a minimum distance of 8. This code can be decoded with majority-logic decoding. Therefore, with the iterative decoding described here, the algebraic decoder is very simple. The error performances of this code with various decoding algorithms are shown in Figure 14.6. We see that it has a 0.5-dB loss in coding gain at BER of 10- 6 compared with the optimum MLD, but it outperforms the Chase algorithm-2 by more than 1 dB from 10- 3 BER down. 14.3.4

(l(llmp1.1ru:arltio1111al Complexity

Assume that the computational complexity of the algebraic decoding to generate the candidate codewords is small compared with the computational complexity required to process the minimum-weight trellis. The computational complexity of the decoding algorithm can be analyzed easily from the flowchart shown in Figure 14.5. Let N, denote the fixed number of computations required in sorting the components of the received sequence Jr in increasing order of reliability, and let N1oop denote the number of computations required in 1. computing the index sets D 1 (v) and Do(v);

Section 14.3

Iterative

test threshold

Let N (T,nin) denote the number of to process the rninimunntreBis (w) ( or (vr,est) ). The number of iterations to on SNR. The maximum number of uv;.,vo,HUcF, decode a received sequence the worst-case maximum number of a received sequence with the

llfmax =

+ 2Ldmn,/2J ><

Ntoop

+ N(T,11i11).

(14.28)

consider the (64, 42) RM code with dmin = 8. The unifonn eight128, 8192, section fuU trellis for this code has 1024, 8192, 1024, 1). The state 45,157,717, iterations required to decode a received sequence of 64 with the iterative is 16. From (14.28) we find that Nmax = 495. In the can be much smaller than this worst-case as shown in Figure 14.7. For at SNR = 5 uocannw needed is about 400, and the average number of

,o 10"

~

r.::

g :5"'

a.

s0 u

10

3

e--

0,.., (])

,£,

s::, C:

102

~

(])

on

"',.., (])

~

101~

"'w 1-weight search + w 2-weight search

100 '------~1_ _ _ _~1_ _ _ __,__1_ _ _~1_ _ _ __,_1_________, 1 2 3 4 5 6 7

E 1,IN0 ,dB

FIGURE 14.7: Average number of

w2-weight trellis search) of the (64,

the iterative

111)

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

101

f: ~

.s'ci, >-;

.~ Of)

"

'8 0 u

Q)

"O 4-<

0

10()

>-; Q)

..0

8;:,

" Q)

bO

"

ow 1-weight search * wrweight search

>-; Q)

~

10- 1 ~ ~ ~ ~ ~ ~ ~ - ' - ~ ~ ~ - - ' - - ~ ~ ~ . . _ _ ~ ~ - - ' ~ ~ ~ - '

1

2

3

4

5

6

7

E1,IN0 ,dB

FIGURE 14.8: Average number of decoding iterations for the iterative decoding ( w1 -,

w2-weight trellis search) of the (64, 42, 8) RM code. iterations required to complete the decoding is 1, as shown in Figure 14.8; however, to decode this code with the Viterbi algorithm based on the uniform eight-section full trellis requires 554, 640 computations. With optimum sectionalization of the fullcode trellis the number of computations required is 538, 799. We see that the iterative decoding algorithm results in a significant reduction in computational complexity, and the performance degradation is only 0.5 dB at BER of 10-6 compared with the optimum MLD.

14.3.5

impmviemenrfcs

The error performance of the iterative decoding algorithm can be improved by using a larger search region. For example, we may use the region R(v, w3), which consists of all the codewords at distances w1 (minimum distance) and w2 (next to the minimum distance) from the current tested candidate codeword for searching for the most likely codeword wML. In the case that

the decoder searches the purged trellis Tw 2 (v) centered on v for finding VML, where Tw 2 (v) consists of v and all the paths of the overall trellis of the code that are at distances w1 and w2 from v. We call this search a w2-weight trellis search. It is clear that the improvement in error performance with a larger search region is achieved at the expense of additional computational complexity. The error performance of the (64, 42) RM code and the computational complexity with a w2-weight trellis search are shown in Figures 14.6, 14.7, and 14.8,

Section 14.4

The MAP

The trellis search achieves a 0.25-dB coding over the trellis search with some additional fo the foregoing iterative ~,v__,v,-HUCC:. to generate candidate codewords. The decoder may fail to decode the modified received sequence and hence results in a failure. This decoding failure can be by the first--orcler based on the ordered statistics in Section 10.8.3 to candidate codewords. This decoding is very this decoding results than the in better candidate codewords algebraic in many cases. As a converges at a faster rate. With first-order statistic ~v·v~•.uu of candidate codewords to be is k for an (n. k) Another current one and several tested ones, to determine the search region. One such can be found in [19], which in error for codes. Of course, this achieved at the expense of additional

Maximum likelihood minimizes the block error probability; minimize the bit error probability. it does not MLD delivers harcl--clecoded bits hard outputs) without their reliability values ( or In many error-control coding schemes it is desirable to provide both decoded bits and their values soft outputs) for further decoding to improve the system"s error performance and to reduce the information loss. For in iterative decoding (see Chapters 16 and 17), the soft of one decoder can be used as the soft inputs to another and the two decoders process the received sequence iteratively until the limit of the code is achieved. A decoding algorithm that processes soft-decision inputs and produces softis called a soft-in/soft-out (S][SO) algorithm. The most well decision known SISO decoding is the MAP a posteriori probability) decoding algorithm that was devised Bahl, Jelinek, and Raviv in 1974 [28]. This algorithm, called the BCJR algorithm, is devised to minimize the bit-error probability and to provide values of the decoded bits. The MAP algorithm ( or its suboptimum is the heart of turbo or iterative decoding [29, 30].

u~.. ~. u

l1r1c;, MA!f» ,R,JgJ«:DWi"1clrum 18asev1 m,1 a 18h"c-L'2"5'1Eli l~ie!fo;

Again, assume BPSK transmission. Let w = (vo, v1, · · · , v11 _1) be a codeword and I\: = (co, c1; · · · , c 11 _1) be its bipolar sequence, where for 0 :s i < n, c; = 2v; - 1 = ±1. Let R' = (ro, r1, · · · . r 11 _1) be the soft-decision received sequence. In decoding, the MAP estimates each transmitted code bit v; by computing its log-likelihood ratio (LLR), which is defined as L( V; )

1c.

= 1og

p(v; = p(v; =

llrr) , Olrr)

(14.29)

712

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

where p(v; Ir) is the a posteriori probability of v; given the received sequence Ir. The estimated code bit v; is then given by the sign of its LLR as follows: v;

=

1, { 0,

if L(v;) > 0, if L(v;) :S 0.

(14.30)

From (14.29) we see that IL ( v;) I is a measure of the reliability of the estimated code bit v;: the larger the IL(v;)I, the more reliable the hard decision of (14.30). Therefore, L ( v;) represents the soft information associated with the decision on v;. Computation of L(v;) is facilitated by using the trellis for the code C to be decoded. In the n-section bit-level trellis T for C, let B; ( C) denote the set of all branches (s;, s;+1) that connect the states in state space I:; (C) at time i and the states in state space I\+1(C) at time-(i + 1). Each branch (s;, s;+1) E B;(C) represents a single code bit v;. Let (C) and (C) denote the two disjoint subsets of B; (C) that correspond to code bits v; = 0 and v; = 1, respectively. Clearly,

Bf

B;

for O :Si < n. Also, based on the structure of linear codes, IBf(C)I (s', s) E B; ( C), we define the joint probabilities !:,.

I

I

A;(s, s) = p(s; = s, s;+1 = s, ll')

=

IB/(C)I. For

(14.31)

for O :s i < n. Then, the joint probabilities p( v;, ll') for v; = 0 and v; = 1 are given by

I:

p(v; = 0, ll') =

A;(s', s),

(14.32)

A;(s', s).

(14.33)

(s',s)EB?(C)

I:

p(v; = 1, ll') =

(s',s)EB/(C)

The MAP algorithm computes the probabilities A; (s', s) 's, which are then used to evaluate p(v; = Oill') and p(v; = llll') in (14.29) from (14.32) and (14.33). In fact, the LLR of v; can be computed directly from the joint probabilities p(v; = 0, ll') and p(v; = 1, ll'), as follows: L(v;)

For O < j sequence ll':

:s

= l og

p(v; = 1, ll') p(v;

= 0, Ir)

.

(14.34)

< n, let ir;,1 denote the following section of the received !:,.

'nJJ

=

(r;, Tj+l, · · · , r1-1)

where for l = j, rr;.; denotes the null sequence. For any states E I:;(C), we define the probabilities a;(s) ,B; (s)

!:,. !:,. -

p(s; =S,ll'O_i),

(14.35)

p(ir;,,, !s; = s).

(14.36)

Section 14.4

H follows from the definitions of ot; (s) and oto(so)

=

= 1.

j3,,(s1)

(14.37)

I:;(C) ands

E

~

I

7B

/3; (s) that

states s 1 ands with s 1

For any tvvo

The MAP Decoding Algorithm

E

I:;+ 1(C), we define the

=

(s • s)).

I

Y;(s. s) = p(s;+l = s. r;Js; = s)

(14.38)

= p(s;+l =sis;= s')p(r;!

. s;+1)

1

For a it follows from the definitions of A;(s 1 • s), ot;(s). j3;(s). 1 and y;(s • s) that for O:::: i < n, A;(s 1 • s)

= ot;(s )y;(s 1

1

s)/3;+1(s).

•

and (14.39) that we can express

it follows from (14.32). follows:

L(s' .s)EB 1(C) a; (s

L(v;)

=log~

'

L(s'.s)EB 0 (c)

as

1

1 )

Yi (s , s) /3; +1 (s)

Ol;(s')y;(s'. s)/3;+1Cs)

.

(14.40)

I

n;~

Q;~

For a state s E (C), let 1(s) and 1(s) denote the sets of states in and in I:;+1 (C). respectively, that are adjacent to s, as shown in Figure 14.9. Then a; (s) and /3; (s) can be expressed as follows: -1 (C)

]_o

For O ::::: i ::::: n.

L

a;(s) =

p(s;-1 =

s'. s; =

s, r;-1. iro.;-1)

s'EQ:'_\ (s)

22;_ 1(C)

22;(C)

22;+ 1(C)

0

0

0

0

0

0

n

0 s

lr/J

n\01(s)

n;+1Cs)

0 0

0 FIGURE 14.9:

States connected to states

0 E

I:;(C).

114

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

2. For O ::, i ::, n, f 'I ' p,ri,•"i+l.n,si+l=SSi=SJ

V L

C/.fe\ __

p,,~J-

s'Erl;t1(s)

L

(14.42) Yi(S, s')/3i+1Cs').

s'Erl;t 1(s)

From (14.41) we see that probabilities <Xi (s )'s, 0 < i ::: n, can be computed recursively from the initial state so to the final state Sf of the 11-section bit-level trellis T with initial condition ao(so) = 1 once the state transition ( or branch) probabilities Yi (s', s) 's are computed. This computation process is called the forward recursion. The probabilities f3, (s)'s, 0 ::: i < n, can be computed from (14.42) recursively from the final state Sf to the initial state so of T with initial condition /3 11 (s) = l. This computation process is called the backward recursion. The state transition probability y, (s', s) depends on the probability distribution of the information bits and the channel. Assume that each information bit is equally likely to be O or 1. Then, all the states in z:;, (C) are equiprobable, and the transition probability I 1 p(s,+1 =sis,= s) = -(~1~1- lr2,~1 Cs')I For an A WGN channel with zero mean and two-sided power spectral density No/2, the conditional probability p(r;l(s,, s,+1) = (s', s)) is given by

1 ~exp{-(r, -c,) 2 /No}. (14.43 ) ....;nNo where = 2vi - 1, and v, is the code bit on the branch (s', s). Because in (14.29) (or (14.34)) we are interested only in the ratio of p(vi = llir) to p(vi = Olir) (or p(vi = 1, r) to p(v; = 0, ir)), it follows from (14.32) and (14.33) that we can scale A1(s', s) by any factor without changing the LLR of each estimated code bit. Consequently, we can use p(r,l(s,,s,+1) = (s I ,s)) =

c,

I

2

/:,

w;(s, s) = exp{-(ri - Ci) /No}

(14.44)

to replace Yi(s', s) in computing <Xi(s), f3i(s), A,(s', s), p(v, = 1, r), and p(v; = 0, r). We call w; (s', s) the weight of the branch (s', s). To carry out the forward recursion set ao(so) = 1 to initiate the recursion process. ]_. Assume that the probabilities ai (s')'s have been computed for all states s' in ~i(C). 2. In the trellis section from time-i to time-(i + 1), assign the weight w,(s', s) to each branch (s', s) E B,(C). 3. For each states E z:;,+1 (C) compute and store a,+1(s) =

L

a,(s')w;(s', s).

1 s'Er2;' (s)

4., Repeat the process until a 11 (sf) is computed.

(14.45)

Section 14.4

The IVIAP

To carry out the backward recursion from the state s.r to the initial state so as set /3 11 (sf) = 1 to initiate the recursion. :TI.,

/3;+1(s 1) for all states s' E l:,+1CC) have been

Assume that the

2, In the trellis section from time-i to + 1). each branch s') E (C). and store ,6;(s)

L

=

w;

the

w;

to

s')/3;+1

s'EQ;:\ (s)

to carry out foe MAP decoding

Tc, Perform the forward recursion process to ot;(s)'s. for O:::: i:::: n.

use the

three the forward state

'.t Perform the backward recursion process to the backward state vu,,uvo, /3; (s)'s, for O :::: i :::: 11. 2t Decoding is also performed the backward recursion. As soon as the /3; +1 (s )'s for an states s E l:; +1( C) have been A;(s', for an branches (s'. s) E (C). From the p(v; = 0. ir) and = 1. the LLR of v; from (14.34) and decode v; based on the decision rule (14.30).

Because the forward and backward recursions are of each other, can be executed from both directions of the code trellis. This bidirectional process reduces the decoding To achieve bidirectional decodthe codeword v = (vo, v1, · · ·, v11 _1) to (vo. vn-1, v1, v 11 -2, · · ·) we before its transmission. Let li' = (ro, r 11 _1, r1, r 11 _2, ···)be its received sequence. Before decoding, we r to obtain two received sequences, ir(l) = (ro, r1, · · · , r 11 _1) and n'C 2l = (r 11 -1, r 11 _2, · · · , ro), which are called forward and backward received sequences, respectively. Then, we shift n,(1) and irC 2l into the code trellis from the initial state so and the final state Sf, to forward and backward recursions simultaneously for computing a and j3 probabilities in directions [33, When the recursions meet at the middle of the trellis, the decoding As soon as both ot;(s') and /3;+ 1 (s), withs' E :E;(C) and s E I:;+1 (C), have been the LLR of the estimated code bit v; is evaluated from (14.39), (14.32), and (14.34). Even though the same number of is required, the bidirectional process roughly reduces the decoding time by half. If the code trellis has we can devise two identical circuits to compute ot 's and /3 's. H the code trellis T for a code is too for IC uu ,_1n.,u1c., we identical subtrellises without the trellis into parallel and 1

716

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

exceeding the maximum state complexity of T. Each of these subtrellises has much smaller state and branch complexities than the full-code trellis T. This p,uallel decomposition allows us to devise identical smaller MAP decoders to process the subtrellises in parallel independently without communications between them, which simplifies the IC implementation and speeds up the decoding process. In Chapter 9 it was shown that it is possible to decompose a trellis T into parallel and structurally identical subtrellises without exceeding the maximum state complexity of T. Suppose T can be decomposed into µ subtrellises, denoted by T(l). r< 2l, .. · , r<µ,). Based on each of these subtrellises TUl, we compute Q(j)(l) = (s',s)EB/ (C)j

and Q
=

where Bf(C)J and B;1(C)J denote the sets of branches labeled with O and 1, respectively, in subtrellis rCil. Then, the LLR L(v;) of v; is given by [33, 34] (14.47)

for O::: i < n. From the standpoint of decoding speed, the effective computational complexity of decoding a received sequence is defined as the computational complexity of a single parallel subtrellis plus the number of computations required to compute the LLR of each estimated code bit from (14.47). The time required to compute L(v;) from (14.47) is relatively small compared with the time required for processing a subtrellis. Because all the subtrellises are processed in parallel, the speed of decoding is therefore limited only by the time required to process one subtrellis.

14.4.3

Comp1uri:2rtio1nal Complexity

The computational complexity of the MAP algorithm can be analyzed by enumerating the numbers of real operations required to compute Yi. a;. (3;, A;, p( v;, r), and L(v;) for each trellis section. We assume that exp(,) and log(,) are computed by using a read-only memory (ROM) (i.e., table lookup). Because w; (s', s) is used to replace Yi (s', s) in all computations, this value can be read out from ROM for each received symbol r;. Consider the trellis section T; between time-i and time-(i + 1) for O::: i < n. For any states E 'E;+1 (C), lr.l;c\s)I is simply the number of branches entering s ( or incoming degree), which is the same for all states in 'E;+1 ( C). The term Ir.i}c\s) I is either 1 or 2 depending on whether there is an oldest information bit a 0 to be shifted out from the encoder memory. Then, the total number of branches in T; is II..__./,.._,,/ n.rr,1I

-

--

2Pi+11o(clre\l l"'""j \'-'} I,

(14.48)

Section 14.4

The IVIAP

.Algorithm

?'H

where Pi+l is the dimension of the state space I:i+l (C) at time-(i + 1). For any state s E :Ei(C), IS1;~1 (s)I is simply the number of branches s (outgoing which is the same for all states in I:; (C). The term IS1;~1 (s)I is either 1 or 2 depending on whether there is a current information bit a''' at time-i. H is clear that the total number of branches in T; is (14.49) ·where p; is the dimension of the state space LJ; (C) at time-i. It follows from and that to 01;+1Cs) for all states in L;+1CC) (C)I and I (C)I - 2Pi+ 1 additions. follows from (14.46) and that to (3; (s) for all states in "2;; ( C) IB;(C)I (C)I - 2P; additions. H follows from (14.32), (14.33), and (14.39) that of p(v; = 0. rr) and = 1, rr·) 21 (C)I and (C) I - 2 additions. the ratio p( v; L n·) / p( v; = one division. A divisim1 to a multiplication operation. Based on the of a received sequence rr N 111 = 4£

+ 11

(14.50)

multiplications and 1V0 = 3£ - 2V -

- 1)

(14.51)

where 11~1

[=

L IB;(C'JI,

(14.52)

i=O

and 11

V

= 2)Pi.

(14.53)

i=O

Note that [ and V are the total number of branches and states in the trellis T, (14.50) and (14.51) we see that the major factor in the of the MAP is the number of multiplications. Because for each trellis y;(s'. s) for the distinct or (31 (s) for all (s'. s) E B;, and LLRs for all code bits must be requires a memory to store 1Vf =

211

+ (V

- 1)

+n =

3n

+V

- 1

real numbers. As an example. consider the 16) RM code with a minimum distance of 8. The bit-error performances of this code based on MAP and Viterbi decodings are shown in Figure 14.10. We see that the MAP algorithm performs just slightly better than the Viterbi algorithm. The 32-section bit-level trellis for this code has a total of 4798 states and a total of 6396 edges. MAP decoding of this code requires 25,616 multiplications, 9530 additions and 4893 storage units.

718

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

10- 1 c - - - - - , - - - - - , - - - - - r - - - - r - - - ~ - - - ·

l

Dashed line: MAP Solid line: MLD

10- 5 ~ - - ~ - - - - " - - - - ~ - - - " - - - - - - ' - - - - - - ' 2 2.5 3 3.5 4.5 4 5 E1,IN11 (dB)

FIGURE 14.10: Bit-error performances of the (32, 16) RM code with MAP and Viterbi decoding algorithms.

14.5

MAP DECODING BASED ON A SECTIONAUZIEID TRIElUS

The MAP decoding algorithm can be carried out based on a sectionalized trellis [33, 34]. Proper sectionalization of a code trellis may result in a significant reduction in computational complexity and storage requirements. 14.5.1

The Algorithm

Let T (A) denote a v-section trellis for an (n, k) linear code C with section boundary locations in A = {to, t1, · · · , fv}, where O = to < t1 < · · · < fv = n. For two adjacent states s' E ~r; (C) and s E ~r;+i (C), let L(s', s) denote the set of parallel branches connecting state s' to state s, called a composite branch. Each branch b(s'. s) E L(s', s) consists of t;+1 - t; code bits,

For each branch b(s'. s) in L(s', s), we define the following probability: yr;(lb(s'. s))

6

p(sr,+ 1 = s, lb>(s', s), ll't;J;+ 1 1sr; = s') p(sr;+i = s, ib>(s', s)lsr; = s') · p(fl'r;.t;+i l(sr;, sr,_1_ 1 ) = (s', s), b(s', s)), (14.54)

which is called a branch probability connecting states' to states. Then, the probability of the composite branch L(s', s) connecting states' to states is given by I

[l

Yr; (!bi(s'. s)).

Yr; (L(s. s)) =

h(s'.s)EL(s',.1·)

(14.55)

Section 14.5

n9l

MAP Decoding Based on a Sectionalized Trellis

For the MAP based on the sectionalized trellis T(A), the fon,vard and backward recursions are to the following probabilities:

L

Oir; (s) = s

1

otr;_ 1 (s )Yr;_ 1 (L(s',

'EQlcl ( ) 'i-1 s

L

,Bt;(s)=

1

Jlt;(L(s,s ))f\+ 1 (s

1 ),

(14.57)

s' El:1;d 1 (s) 1+1

wheres is a state in

(C),

c:{'~

1

(s) denotes the set of states in I\_ 1 (C) that are

to s, and s-;{'~ 1 (s) denotes the set of states in :Et;+i ( C) that are The initial conditions for the recursions are ao(so) = 1 and /3 11 (s1) = 1. For the LLR of an estimated code bit v1 with ti :S I < define L.

Yt; (L~~l (s', s)) -

E

to s. ti+1,

we

Yr; (lbi(s', s)),

(14.58)

Jlt; (lbi(s', s)).

(14.59)

ib(s'.s)EL(s',s)

v1=0 Jlr; (L~~\s'. s))

L.

-

E lb(s'.s)EL(s'.s)

v1=l

H follows from

(14.58), and (14.59) that (14.60)

Jlt,

for every code bit

v1

with

t; ::::

l < t;+l· Then, the LLR of v1 is given L(s'.s) Ott; (s')Yt; (L~}\s', s))f3t,+i (s)

) = log

v1=l

(Ol , Lcs1.s) Ott; (s')Yt; (Lv, (s'. s))/3t,+J (s) 111=0

where the summations are over all the adjacent state pairs (s', s) with s' E I:t; ( C) ands E I:t;+i (C). To carry out process we must first compute the composite branch fo a sectionalized trellis a section may consist of many composite the number of distinct composite branches is small. In computing Ott; (s), f:3t; (s), and L(v1), we need the probabilities of the distinct composite branches. Thus, we may perform a preprocessing step to (L Yt, (L~~\s'. s )), and Jlt; (L~}\s', s)) for each distinct composite branch L , s) and each code bit v1 and store them in a table called the Yt; -table. From (14.56) we see that Ott; (s)'s can be computed along with Yt; (L(s', s))'s. The MAP based on a sectionalized trellis T(A) can be carried out in two stages. At the first stage, the parallel branches of each distinct composite branch are to obtain the probabilities JI;, (L(s', s)), Yt, (L~~\s', s)), and

720

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

Yt, (L~;) (s', s) ). Then, the MAP decoding is performed with parallel branches viewed as a single branch. Bidirectional and parallel MAP decoding based on a sectionalized trellis can be performed in exactly the same manner as that based on the bit-level trellis T. In bidirectional decoding, ot 11 (s)'s and /Ji, (s)'s can be computed simultaneously along with y 11 (L(s', s))'s. Consider the computation of a branch probability Yt, (lb(s', s)) given by (14.54). If all the states at any section boundary location are equiprobable, then

= s , lbi(s /, s) Ist,. = s /) = - - -1-IL( - I- ")I '\'

P( s 11+1

L....s" Erl.id I (s')

S , S

1

i+I

H has been proved that Ir.i;dl 1 (s') I and IL (s', s") I remain the same for all states •+

s' E ~

1,

(C). Therefore, p(s 11 + 1 = s, b(s', s)js11 = s') is constant in the trellis section Ti

from time-ti to time-ti+l · Because we are interested only in the ratio given by (14.61), we can scale the branch probability Yt, (b(s', s)) by any factor without changing the LLR of an estimated code bit. Therefore, in computing a, f3, and L ( v1), we can use (14.62)

p(rr1,,11_,_ 1J(s1,, s11 + 1) = (s'. s), lb>(s', s))

to replace Yt, (b(s', s)) to simplify computations. For an A WGN channel with zero mean and variance No/2, p(rr1,,11+1l(s1,, si,+ 1)

=

(s'. s). lb>(s', s))

1

l

117

2 1 ,~l r. · - 2v -- (n No)111,/2 ex P - .~ ( t,+1 ( t,+J· - 1)) I N o ,

where mi

= ti+l

- ti.

Because 1/(n N 0 )"' 112 is constant in

w1, (lhi(s', s))

l

2

= exp N

Ti,

117

L

,~1 l"t,+j · v1,+f

I

(14.63)

we can use (14.64)

0 j=O

to replace Yt, (lbi(s', s)) in computing L(v1) of (14.61). To construct the Yt,-table for the trellis section Ti from time-ti to time-ti+l with 0.::: i < v, we can use the following procedure-y to save computations: 1,

Compute w 11 (b(s', s)) using (14.64) for all distinct branches in

Ti.

2. Compute y 11 (L~~l(s'.s)) and y 11 (L~;\s'.s)) from (14.58) and (14.59) with y11 (b(s', s)) replaced by w 11 (b(s', s)) for each code bit v1 with ti ::: l < ti+l · 3. Compute Yt, (L(s', s)) for all distinct composite branches from (14.60). From (14.56) we see that D!t, (s)'s can be computed along with the construction of y-tables from the initial state so to the final state s 1 of the code trellis in the

MAP

Section 1L!.5

Based on a Sectionalized Ti"ellis

721

forward direction. H bidirectional decoding is it follows from (14.57) that /3 1; (s) 's can be with the construction. of from the final state sr to the initial state so of the code trellis in the backward direction. As soon as a 1, and (s)'s at the boundaries of the treHis section T; have been the LLRs of code bits vr with t; ::: l < t;+1 are evaluated from

To of the MAP based on a sectionalized code trellis T we need to use some trellis structural n,a,c,n,~,·i-,"Q 9. Consider the section T; of T (A) from t; + 1 · branch . s) in this section is a cos et in the ")/Ctr d . , Pt;J;+1 ( L /;J;+I an .COI1SlSlSOJ. £

branches. There are

the total number of branches in T; is (14.68)

B; =Br· Bf.

The number of composite branches entering a state s of s) is by d[eg (S ) in

:.=

2

k(Co.,. )-k(Co.,. I )-k(C, I

it follows from the definitions of

Q);2

/-

1

(C)

E

/-

I·'·) I

•

(s) and deg(s); 11 that

(c)

1Qr;_ 1 (s)I = deg(s); 11 •

The number of vv,uc,~unv branches leaving a state s of s) is deg(s)o!I/

(14.70) E

(C)

= 2k(C1;.11)-k(C,,+I ·" )-k(C,i·'i+I).

it follows from the definitions of

c::?~ (s) and deg(s) 1

1nf1+1> (s)I = deg(s)o!lt· 1

0"

the

(14.71) 1

that (14.72)

122

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

Again, we assume that exp(-) and log(·) are computed by using table lookup. To form the Yt; -table for the ith trellis section T; based on the procedure-y presented in the previous subsection, the computations required at each step are listed:

JI.. Step 1 requires Bf · Bf · (m; - 1) additions and m; multiplications. 2. Steps 2 and 3 require Bf · (B( - 2) · m; and Bf additions, respectively. For the case of Bf = 1, since there is only one branch in each composite branch, we need to compute only w 1; (b(s', s)) for all distinct branches. Therefore, construction of the y1, -table for trellis section T; requires a total of N'· (y) a

=

{ Bd , · BPz · (m , - 1) Bd · (m·l - 1) , l

+ Bd, · (BP,

- 2) · m ,·

+ Bd, ,

for for

Bf > 1, Bf= 1,

(14.73)

additions and a total of (14.74)

N~1 (y) = m;

multiplications. Computation of cx's and /3's from (14.56) and (14.57) requires a total of N~ (a)+ N}, (/3) additions and N,\, (a)+ N,\1 (/3) multiplications, where

N~(/3)

= (deg(s1); -1) · 2PI; = Bf_ 1 - 2Pr;, = (deg(s1)out -1) · 2Pr; = Bf - 2PI,,

N,;,(a)

=

Br_

N 1~1 (/3)

=

Bj.

N:,(a)

11

(14.75) (14.76) (14.77)

1,

(14.78)

The last step of MAP decoding is to compute the LLRs of the estimated code bits v1 fort; :S l < t;+1 with O :::: i < v. We define s(i)

~

1

Ol1;(s')Y1;(L(s , s))/31;+1 (s),

L

(14.79)

(s',s)

sgl(v1)

6

1

LOl1;(s')Y1;(L~~)(s ,s))f31;+ 1 (s),

(14.80)

(s' ,s)

Vt=O

sflcv1)

6

Olt; (s')Yt; (L~7\s', s))/31;+1 (s).

L

(14.81)

(s' ,s)

v1=l

H follows from (14.60) and (14.79) through (14.81) that sCil

=

sg>cv1)

+ sf\V/)

(14.82)

for any /, with t; :::: l < t;+l· To compute the LLR L(vz)'s from (14.61), we need to compute sg\vz)'s and s?\V/)'s. The sg\v1)'s and s?\v1)'s can be computed efficiently by using the following procedure-S:

Section 14.5

Based on a Sectionalized Trellis

IVIAP

5U) from

723

- 1 additions and

cations.

s?\vz)'s from (14.81) for i;

Z,

additions and 3, Compute t; :::

l <

· m;

sg\v1)'s from

t; +l ·

(14.82)

This step

m;

sfl

Once sg\vz)'s and vz fm t; ::: l < t;+1 can be

the differences 5Ci) -subtractions to the LLRs of estimated code bits which m; divisions to of the LLRs of estimated code bits in the

-1+

(14.83)

·m;

additions and a total of

+

+ 1) · m;

(including m; fo summary, execution of the MAP T(A) requires a total of

(14.84)

based on the sectionalized trellis

v-l

N 0 (1\) = L{Nj,(y)

+ Nj,(L) + N;,(a) + N; (/3)}

(14.85)

+ N,\,

(14.86)

1

i=O

additions and a total of v-1

lV111(A)

=

L{N,'.,(y) i=O

of

The numbers N" (A) and N 111 (A) give a measure of compuof the MAP based on the sectionalized trellis T(A). During the decoding process the must be stored for the /3's, and LLRs of the estimated code which require v-1

M(y) = L; v Bf' (2m;

+ 1)

(14.87)

storage locations ( or The a's and f3's for the of LLRs of the estimated code bits also need to be stored. Thus, for bidirectional decoding, M(a,

/3)

i:,

=

M(a)

+ M(/3)

(14.88)

2

storage locations are required, where \!-l

M(a) = M(/3)

=

L JI:1; (C)I = i=O

V - 1.

(14.89)

124

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

If the LLRs of estimated code bits are to be used for further decoding process, they

must also be stored, which requires another M(L)

=n

(14.90)

storage locations. Therefore, the total storage requirement for MAP decoding based on the sectionalized trellis T (A) is M(A) = M(y)

+ M(a, fJ) + M(L).

(14.91)

The computational complexity and storage requirement of MAP decoding of a linear block code very much depend on the sectionalization of the code trellis. A sectionalization that minimizes both is desirable; however, in general, such a sectionalization does not exist. If there is no severe constraint on the size of memory storage, we may choose a sectionalization that minimizes the computational complexity. Based on the foregoing analysis, decoding computation of the MAP algorithm involves two kinds of real-number operations, additions and multiplications, in every decoding step. A multiplication operation is more complex than an addition operation, and the operations cannot be treated the same (have the same weight) in the minimization of computational complexity. This makes it difficult (if not impossible) to find a sectionalization that minimizes both the number of additions and the number of multiplications. Because the number of multiplications required in decoding is much larger than the number of additions required, and a multiplication operation is much more complex than an addition operation, we may just find a trellis sectionalization to minimize the total number of multiplications with a constraint that the number of additions may not exceed a times the number of additions based on the bit-level trellis; or we can weight an addition operation as a fraction of a multiplication operation, say, l/b of a multiplication. Then, we find a trellis sectionalization to minimize the total number of multiplications and weighted additions. The Lafourcade-Vardy algorithm [3] presented in Section 14.1 can be used to find such an optimum trellis sectionalization to minimize the total number of multiplications and weighted additions. Optimum trellis sectionalizations (in terms of minimizing the number of multiplication operations with constraint a = 1.2) of some RM codes and the (24, 12) Golay code for MAP decoding are given in Table 14.2. For comparison, the computational complexity based on the bit-level trellis for each code is also included. From this table we see that a bit-level trellis requires much greater computational complexity than an optimum sectionalized trellis.We also see that the optimum trellis sectionalization in terms of minimizing the number of multiplication operations may not reduce the number of addition operations. Thus, there is a tradeoff between the number of multiplications and the number of additions. Optimum trellis sectionalizations (in terms of minimizing the total number of multiplications and weighted additions with 1/5 (b = 5)) of some block codes for MAP decoding are given in Table 14.3.

Tt-\BLE 14.2:

trellis secHonalizations of some block codes for MAP IB\nlt-Ileveil ·/rreTIIli§

!Cl[)(rile

RM(8,4)

IV!Ii1!1Illtnrrnllk~(iill{],il!J

192

Ai!l
with consvraint a= 1.2.

CCDJP)lillnnnllllnmt §te1I:ttfi(J))rm21Il:1i222ittii(J))n11 IV!Iennwry

50

57

iR
!Mhnll!ii]lilili,mltfo1111

A1TI
{0. 1,2,4,6,7,8)

128

58

!Mie,nn<()rry

41

720

[86

197

(0, l, 2. 4, 7, 8, JO, 12, 14, 15, l6}

400

218

119

RM(16, 11)

1,040

426

197

(0, 2, 4, 6, 7, 8, 10. 12, 13, 14, 15, 16)

808

504

139

RM.(32, 6)

2,800

714

733

{O. 2, 4. 8, 16, 20, 24, 26, 28, 30, 31. 32)

872

856

251

RM(l6, 5)

V1

25,648

9,530

4,893

(0, 4, 8, 9, 12. 16, 17, 20, 23, 24,27,28,29,30,31,32)

ll,128

l l,404

1,295

RM(32, 26)

4,784

2,202

733

{0,4,6,7,8, 10, 12, 13,14, 16, 18, 19, 20, 21. 22, 24, 25, 26,28,29,30,31,32)

4,088

2,639

591

{0,2,4,8, 12, 16,24,32, 44,48,52,58,60,62,63,64}

2,216

{0,8, 16,24,28,31,32,34, 40,46,47.48,54,55,56,59, 60,61,62,63,64)

348,536

RM.(64, 22)

RM.(64, 42)

RM(64, 57)

Golay(24, 12)

11,056 1,500,272

2,197,616

20,464

14,368

2,794 475,258

998,266

9,850

5,322

2,829 325,053

325,053

2,829

2,757

'" 6' F<"

::i :I'>, V1

RM.(32, 16)

RM.(64, 7)

m

~ )>

v

0

(j)

r,

3,344

627

0

Q.

:J

I.Q

569,812

37,795

00

ru

"'(j)Q.

{0, 1, 8, 12, 14, 16, 17, 20, 22, 24,26,28,30,32,34,36,38, 40,42,44,46,48,50,52,54 56,58,59,60,61,62,63,64}

1,454,720

{0,4,6,7,8,9, 11, 12, 13, 15, 16, 18, 19,20,21,23,24,25, 26,27,28,29,30,32,34,36, 37,38,39,40,42,43,44,45, 46,48,49,50,51,52,53,56, 57,58,60,61,62,63,64}

19,128

(0,3,8,9, 12, 15, 16, 19,20, 21. 22, 23, 24)

6,742

1,196,690

139,453

0

:J QJ

Vl

fl) r,

~0

11,811

2,303

::l

ru

N

ro

Q..

-i

;;,

6,372

8.13

"' "'-.l

1',J

Y'i

726

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

TABLE 14.3: Optimum trellis sectionalizations of some block codes for MAP

decoding with b = 5.
14.6

Optimmm sectfom1fiizllltforr1

Bit-level trellis operntiorr1s

Jlfolllmlllllry iocationi

Operntim1s

RM(8, 4)

202

(0, 8)

RM(16, 5)

757

(0, 16)

247

74

RM(l6, 11)

1,125

(0,4,6.8,10, 12, 16)

796

RM(32, 6)

2,942

(0, 8. 16, 24, 32)

697

RM(32, 16)

27,554

RM(32, 26)

5,224

RM(64, 7)

11,614

{0,3,8, 12. 16,20,24,29,32)

11,970

(0,4.6, 7,8, 10, 11, 12, 13, 14, 16. 18. 19 20,21,22,24,25.26.28,32}

4,532

{0,8, 16,32.48,56,64)

1,881

RM(64, 22)

1,595,323

(0.3,8, 16.24.32,40.56,61.64)

RM(64, 42)

2,397,269

(0, 1.4.8, 12. 14, 16.20.22.24,26.28.32, 36,38.40,42,44.48.50,52,56,60,63,64)

RM(64, 57)

22,434

(0,4.6. 7.8. 10. 11. 12,13, 14, 15, 16. 18, 19.20.21.22.23.24.25.26.27,28,29.30.32 34,35,36,37,38.39.40,41.42.43.44,45.46 48,49,50.51.52.53.54.56,57,58,60.64)

Golay(24, 12)

15,432

{0,3.8. 12, 16.21.24)

412,598 1,652,378 21,013

6,742

MAX-LOG-MAP DECODING AlGORrfHM

To achieve optimum bit-error performance, the MAP decoding algorithm requires a very large number of real computations, especially multiplications. For long codes with large trellises, this large computational complexity makes implementation of this decoding algorithm very difficult and impractical. If we do not insist on attaining optimum error performance, a suboptimum MAP decoding algorithm can be devised to provide a very efficient trade-off between error performance and decoding computational complexity. This suboptimum algorithm is known as the Max-log-MAP algorithm and is based on a very simple approximation for the logarithm of a sum of real numbers [30]-[32]. 14.6.1

Max-iog-MAIP' Dewdling Based m11 a IBiit-llevel lrems

For a finite set of real numbers {81 , 82, · · · . 8q} the following approximation holds: I q

log

\

(Le°;):::::: (max {8; }). \i=l

(14.92)

l:':7:':CJ

This approximation is called the maximum logarithm (max-log) approximation.

Section 14.6

7/27i

the LLR of an estiniated code bit l(v;)=

max

{logot;(s')+logy;

.s)+log/3;+ 1 (s)}

(s',s)EB/ (C)

{log ot; (s') +logy; (s'. s) + log /3;+ 1 (s)}.

(14.93)

(s'

From

we have max {logot;-1(s

1 )

+

Jli-1

. s)}.

(14.94)

s' EQ:~ 1(s)

,6; (s) =

max {logy; (s, s') + log /3; + 1(s')}.

(14.95)

s'Es:.i:t 1(s)

-c;)2.

logy;(s'.s)=-

·where c; = 2v; - 1. (Note that since No and 1/ ,,/rr No are constants, we use - c; ) 2 as the branch The metrics log ot; (s) and log ,6; (s) are the forward and backward metrics of state s. and can be with initial conditions logoto(so) = 0 and log/311 (s1) = 0. The the branch metric. The sum log ot; (s') + logy; . s) + metric corresponding to the code bit v; on the branch The Max-log-MAP decoding is to the LLR l(v;) for each the forward and backward recursions are carried ot; (s) and log ,6; executing the Bidirectional to uvo,~•,,rncl', time. Again, assume that log(·) is MAP additions Because a operation is as regarded as an addition-equivalent operation. of the Max-log-MAP is straightforward, (14.96). c;

For each trellis logy;(s', s) has only two to = ±1 (or v; = 1 or 0). Computing logy; (s', s) 's in the entire decoding process 2n additions and 211 multiplications. ot's and ,B's, Na(ot)

= l'(,(/3)

=

E

(14.97)

additions each and Nc(0t)

= N,(/3)

= [ - V

+1

(14.98)

728

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

comparisons. To compute L(vi) for O::: i < n requires a total of Na(L)

= 2[

(14.99)

additions, a total of (14.100)

comparisons, and Ns(L) = n

(14.101)

subtractions. Therefore, the Max-log-MAP decoding requires a total of Na=7[-2V+n+2

(14.102)

addition-equivalent operations and 411 multiplications. Because [ » n, the number of multiplications required in the decoding is very small compared with the number of addition-equivalent operations and hence can be ignored in measuring complexity. To store y's, a's, {J's, and L's requires 2V + 3n - 2 storage locations. The error performance of Max-log-MAP decoding is very close to that of MAP decoding while reducing the computational complexity significantly. Figure 14.11 depicts the error performances of the (32, 16) RM code using both MAP and Max-log-MAP decodings. We see that Max-log-MAP decoding results in only a very small performance degradation, less than 0.1 dB. The Max-log-MAP decoding algorithm is actually equivalent to the soft-output Viterbi decoding algorithm (SOVA) [35, 36].

- Dashed line: MAP Solid line: Max-log-Map 10--1

10-s ~ - - ~ - - - ~ - - ~ - - - - " - - - - - - ' - - - - - - - ' 2 2.5 3 3.5 4 4.5 5 E 1,IN0 (dB)

FIGURE 14.11: The bit-error performances of the (32, 16) RM code using MAP and

Max-log-MAP decodings.

Section 14.6

16!.,6,2

MioJ}{~i©gJ-MA~ rr:»eiwirfo111QJ IBioiS(B(Oj rnril

Decoding

§e[iti©u'11ioila:i:ell:l1 rwems

si

we can the LLR of an estimated code (14.61) based on a sectionalized trellis T(A) [34]: L(VJ)

=

.s) {log Olr;

(s')

+

f (1)

Yt; ( iL.JVf

, s))

+ log,811+ 1 (s)}

v;=l .s)

+

{log ot 11 (s')

Yr, (L~~\s'. s))

+ log/3 1;+i (s)},

(14.103)

v;=O

that the state and

for ti ::: l < t;+l· H follows from (14.55) branch metrics are given by log ct; (s) =

max

ot11_ 1 (s')

+ log Y1;_ 1

, s))},

(14.104)

s'EQ;;~, (s)

/3; (s)

= =

log Yi; f (0) ( I Yi; ( ~V/ s.

max

s'EQ;d 1 (s) 1+!

{log Yt,

max

+

(14.105)

{log Jlr (ibJ(s', s))}.

(14.106)

(log Yi; (ibJ(s', s))},

(14.107)

{logy11 (lbi(s 1 .s))}.

(14.108)

1

U](s',s)EL(s',s)

max

s'))

U](s'.s)EL(s'.s)

v;=O 1

Jli;(L~7l(s ,s))=

max U](s'.s)EL(s',s)

V;=l

for

ti :::

l < t;+l·

To carry out the decoding based on T(A), we preprocess the parallel branches of each distinct branch L (s'. s) to obtain the composite branch metric logy1;(L(s', s)) and the composite branch metrics logJ1t;(L~~\s 1 , s)) and log Yr; (L~7\s 1 • s)) corresponding to code bit vr fort; ::: l < t;+l· These metrics are stored in a branch metric table. The branch metric tables are then used to compute state metrics log ot 1; (s) and log /3 1; (s ), and LLR i( vr ), for all the states in T (A) and all the code bits. fo the decoding process each composite branch L(s'. s) is as a y 11 (L(s'. s)). and m; pairs of metrics, single branch, which is assigned a branch (Y1,(d~\s 1 , s)). (y1;(L~7\s, s')). corresponding tom; code bits on the branch where m; = t;+1 t;. Computations of state metrics are based on (14.104) and (14.105). Computations of the LLRs of code bits are based on (14.103) using the ACSS process. If bidirectional decoding is performed, the forward and backward state metrics are computed along with the branch metrics in both directions. When the forward and backward state metrics at the two locations of a trellis section have been computed, computations of the LLRs of the code bits corresponding to the trellis section begin. For A WGN and BPSKsignaling, we can use (14.63) to compute log Yr; (lbi(s'. s)). 111 12 ) Because No and ; are constant in the trellis section T;, we can use

(i., 1 0

1;+1-l

I og Y1, (lbi(s I . s))

6

=

~ ~ 1=11

rr · (2vr - 1)

(14.109)

730

Trellis-Based Soft-Decision Decoding Algorithms

Chapter 14

as the metric of branch b(s', s) in computing log Yr; (L(s', s))'s, log Yr; (d~\s', s))'s, -(1)( s I , s ) ) ' s. an dl og Yr; (Lv 1

v/+

_ ) be the branch that has the largest branch Let ll:»*(s', s) ~ (v;, v;'+l' · · · , 1 1 metric among the parallel branches in L(s', s). From (14.106) we find that

log Yr; (L(s', s))

= log Yr, (b*(s', s)).

(14.110)

Then, it follows from (14.106) through (14.108), and (14.110), that log Yr; (L(s', s))

_I

= max{log Yr; (L~~\s', s)), log Yr; (L~7\s', s))} log Yt; (L~~\s', s)),

l og Yr;

if v;'

= 0,

1'f v1*

1,

(14.111) (L (l) v (sI , 1

s , ))

=

fort; ::: l < t;+l· Based on (14.111), we can construct the branch metric table for the trellis section T; using the following procedure-B:

lo We compute log Yr; (L(s', s)) from (14.106) for each distinct composite branch

in 1;. '.t For each code bit v1 in T;, based on b*(s', s) and from (14.111), we first check whether v;' = 0 or 1 (a logic operation) to determine which of log Yt; (L~~) (s', s)) and log Yr; (L~;\s', s)) is equal to log Yt; (L(s', s)). Then, we need to compute only the one between log Yr, (d~l (s', s)) and log Yr; (s', s)) that is not equal to log Yr; (L(s', s)).

cd/l

At the second stage of the Max-log-MAP decoding, the state metrics log a 1; (s) 's and logf'\(s)'s are computed from (14.104) and (14.105) recursively with initial conditions log a 10 (so) = 0, and log /3 1,. (sf) = 0. For bidirectional decoding, log otr; (s) 's and log f3r; (s) 's are computed simultaneously from both directions of the trellis T (A) while the branch metric tables are being constructed, section by section. Once the state metrics log a 1, (s')'s and log /3 1,+ 1 (s )'s at the section boundary locations t; and t;+l have been computed and the branch metric table for trellis section T; has been constructed, we can compute the LLRs of the estimated code bits v1 fort; ::: I < t;+l from (14.103) by executing the ACSS process. To compute the LLR L(vi) efficiently, we define R(i)

6

max{loga 1;(s')+logy1,(L(s',s))+log/31;+i(s)},

(14.112)

max{logot 1;(s') + log Yr, (L~~\s', s)) + log/3r,+ (s)},

(14.113)

(s' ,s)

R~i)(v1)

6

1

(s' ,s)

V/=0

Rf\ v1)

~ max{log a 1; (s') + log Yt; (L~;) (s', s)) + log /3i;+i (s)} }.

(14.114)

(s' ,s)

v1=l

for O ::: i < v. From (14.60), (14.112) through (14.114), and by using the max-log approximation, we readily see that t; ::: I < t; + 1, I 1 A 1 1 i:::\

\i'""t,..LJ..J

J

Section 14.6

Max-log-MAP Decoding Algorithm

1311

It follows from (14.111) through (14.114) that

if v/

= 0, if v/ = L

(14.116)

Then, an efficient follows: JL VVe based on (14.112). ::t For each code bit v1 in T;, based on lbi*(s', s) and from (14.116), we first check = 0 or 1 to determine which of Rg) ( v1) and Rf l ( v1) is to ""'"''"'""'' the one that is not equal to RU).

21c We

the difference Rf\v1) - Rg\v1), fort;::: l < t;+I·

i(v1)

the branch metric tables for procedure-R We readily find that construction the trellis sections can be of the branch metric table for the treHis section T; requires a total of Ni(y)

=

Bf1 · {B/1 -1

+ (B/1 /2 -1) · m;}

Ni( (I y ) = Bd. l

. (m; -1)

(14.117)

(14.118)

additions. From and (14.105) we find that the computation of metrics of states at the boundaries of T; a total of Nj, (ot) + N;, (/3) additions and NJ (ot) + (f3) where Nj,(ot) = N:1 (/3) = Bt,

(14.119)

N; (ot) = BC

(14.120)

C

1-l

-

2Pt;

'

(14.121)

the complexity of computing the LLRs given by (14.103), we follow and find that a total of N{(i)

=

(Bt -1). (m;

+ 1)

(14.122)

and a total of (14.123)

additions subtractions) are required to compute the LL Rs of the estimated code bits in T;. Because a comparison operation has the same complexity as an addition, it is regarded as an addition-equivalent operation. Therefore, to decode a received

732

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

sequence, the Max-log-MAP algorithm based on the sectionalized trellis T(A) requires a total of v-1 Nae(A)

=

I)N~(y)

+ N~(y) + N~(i) + N~(L) + N~(a) + N~(a) + N~(f3) + N~(f3)} (14.124)

addition-equivalent operations. Nae(A) is used as a measure of the computational complexity of the Max-log-MAP decoding algorithm based on a sectionalized trellis. The storage requirement for the Max-log-MAP algorithm is the same as that for the MAP algorithm. The computational complexity NaeCA) depends on the sectionalization A of the code trellis T. The sectionalization that minimized Nae CA) is called an optimum sectionalization. The Lafourcade-Vardy algorithm presented in Section 14.1 can be used to find such a sectionalization. Table 14.4 gives optimum sectionalizations of trellises for some RM codes with the Max-log-MAP decoding. For comparison, the computational complexities of these codes based on bit-level trellises are also included. We see that proper sectionalization reduces computational complexity significantly. TABLE 14.4: Optimum trellis sectionalizations of some RM codes for Max-log-MAP

decoding. OJl)fonmm sectiolll!llilizllltioHll

Bit-level trellis RM(r,m)

Operntfrms

B<0lllmillllry ioc!lltiolll

Memory

01Jerntfonns

Memory

{O .. 4, 8)

156

81

195

{0.2,4,8. 12, 14. 16)

486

77

109

RM(l, 3)

230

55

RM(l, 4)

886

RM(2, 4)

1,446

195

(0.2,4,6,8, 10. 12, 14, 16)

1,222

RM(l, 5)

3,478

731

(0,2.4.8, 16.24.28.30.32)

1,510

165

RM(2, 5)

35,142

4,891

22,078

919

RM(3, 5)

6,950

731

(0,2,4,6. 7.8, 10. 11, 12. 13, 14, 16, 18. 19,20,21,22,24,25,26,28,30,32}

6.446

549

RM(L 6)

13,782

2,827

{0,2,4,8, 16.24,32,40.48,56.60, 62,64)

5,094

469

RM(2. 6)

1,975,462

325,051

(0. 1,3,5,8. 10. 16, 18,24,30, 32, 34. 40, 46, 48, 54. 56, 59, 61, 63. 64)

905,974

37,839

RM(3, 6)

3,195,814

325,051

{0.2.4.8. 10.12. 14. 16. 18.20.22,24. 26,28.30,32,34,36,38,40.42,44. 46.48.50.52.54.56,58,60,62,64}

2,646.566

141,925

RM(4, 6)

30,246

2,827

29.174

2,453

(0, 1.3.5.8. 12. 16.20,24.27.29. 31, 32)

{0,2,4,6, 7.8. 10. 11. 12. 13. 14. 15. 16, 18, 19,20,21.22.23,24.25.26.27,28. 29,30,32.34.35,36.37,38,39.40,41. 42.43.44.45.46.48.49.50,51,52. 53.54,56,57,58,60.62,64}

Algorithm

733

realization of the MAP algorithm. very close to that of the MAP algorithm, inferior to those of the MAP when vve use the Maxvalue degradation

h, -

)=

. 82}

+ fc(\82

- 81\).

(14.125)

where we can

is g correction functiorL V~.U'""H•V

log(e 61

+

0

''

+

a finite set of real numbers {81, · · · , 8q}, The rncursion is initialized with the + ... + e8;-1) with 1 < i .'.': q is

trellis sectionalizations of some RM codes for log-MAP

TABLE 14.5:

decoding. m-i-Ilev2Il frellllns

RM(r,m)

Orr:verrtiJtfi<0R11s

On}erntti1n§

Men,11m7

RM(l. 3)

330

55

RM(l. 4)

1.258

195

RM(2.4)

2,298

195

RM(l. 5)

4.906

731

Rll/!(2, 5)

54.202

4.891

Rll/!(3. 5)

11.354

73!

Memory

{O. L!, 8)

244

81

{O. 1. 2. 4. 8. 12. P. l'.i. 16)

866

83

(Q. J. 2, 3. LI, 6. 8. 10, 12, [3.14. JS. 16)

2,242

115

{0. l. 2.
2.946

171

43,906

1,415

11.210

555

10.690

475

{O. 1. 2. 3. 5. 8. 9. 12. 16. l7. 20. 24. 28. 31, 32. 33, 36. LJ.O, 44 . .::].7, 43. 52. 55. 56. 59. 61. 62. 63. 64)

1.933.682

63,819

{O. 1.2.3.4.6.8. 10. 12. 14. 16. 18.20.

5,009,346

141,931

49,618

2,459

{O, l.2.3.5.8.9. 12. 15. 16. 17. 20. 23. 2'1. 27. 29. 30. 31. 32}

{O. 1.2.3.4.6.7.8. 10. 11. 12. !3. 14. 16. 13. 19. 20. 21, 22. 24. 25. 26. 28. 29. 30. 31. 32)

J J

Ril1(l. 6)

19.370

2.827

Rll/f (2. 6)

2.925.978

325,051

]

R ll/f (3, 6)

SJ 92.346

325,051

{0. 1.2.4.8, 16.24.32.40.48.56.60. 62.63.64)

22. 2-L 26. 20. 30. 32.

36. 33, 40, 42,

L\4, 46, 48, 50. 52, )4. 56, 58. 60, 61. 62, 63, 64]

RM( 4, 6)

49,946

2.827

{0. l. 2. 3. 4, 6, 7, 8. 10, l l, 12, 13, 1.,t 15, 16. 18. J 9. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 32. 34, 35. 36. 37. 38. 39. 40. 'fl. 42, 43. 44. L!-5, 46. L]8. 49. 50, 5}. 52, 53.54.56.57.58.60.61.62.63.64)

134

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

known. Hence, log(e6 1 + · · · + e°i)

= log(L:. + e°;) = max{log 6, Oi} + fc(l log 6

- 8; I),

(14.126)

with 6 = e0 t + ... + e0i-t. Based on this recursion, we modify the Max-log-MAP algorithm by using simple correction functions. This algorithm, called the log-MAP algorithm [31 ], gives the same error performance as the MAP algorithm but is easier to implement. Each correction term needs an additional one-dimensional lookup table and two additions based on (14.125). Consequently, the log-MAP algorithm requires only additions and comparisons to compute the LLRs. The storage requirement for the log-MAP algorithm is the same as those for the MAP and Max-log-MAP algorithms, assuming the storage of the lookup tables is negligible. Consider the computational complexity of the log-MAP algorithm. Because two extra additions are required per comparison to calculate fcO in (14.125), a total of N!,(y) + 3Nj.(y), Nj,(a) + 3N/(a), Nj,({3) + 3Nj,({3), and N!,(L) + 3Nl.(l) addition-equivalent operations are required to compute logy 's, log a 's, log f3 's, and LLRs in Ti, respectively, where Nj,()'s and Ni()'s are the numbers of additions and comparisons evaluated for the Max-log-MAP algorithm. Table 14.5 gives optimum sectionalizations (in terms of minimizing the number of addition-equivalent operations) of trellises for some RM codes with the log-MAP decoding. For comparison, the computational complexities and storage requirements of these codes based on bit-level trellises are also included. We also see that proper sectionalization reduces computational complexity and storage requirements for the log-MAP algorithm. PROBLEMS

14!.1 Suppose the (8, 4) RM code is decoded with the Viterbi algorithm. Determine the number of real operations ( additions and comparisons) required for the following trellises: !11. The eight-section bit-level trellis. lb. The uniform four-section (two-bits per section) trellis shown in Figure 9.17. c. Optimum sectionalization based on the Lafourcade-Vardy algorithm. 14.2 Suppose the (8, 4) RM code is decoded with the differential Viterbi decoding algorithm based on the uniform 4-section trellis of the code. Determine the number of real operations required to decode the code. 14!.3 The first-order RM code of length 16 is a (16, 5) linear code with a minimum distance of 8. Decode this code with the Viterbi algorithm. Determine the number of real operations required for the decoding based on the following trellis sectionalizations: m. The 16-section bit-level treilis. !bi. The uniform eight-section trellis. c. The uniform four-section trellis. irlt Optimum sectionalization based on the Lafourcade-Vardy algorithm. 141.4! Decode the (16, 5) first-order RM code with the differential Viterbi decoding algorithm based on the uniform four-section trellis. For each section, determine the parallel components, the set of branches lezcving a state at the ieft e11d uf a

parallel component, an
Bibliography

Jl.41o5

UAii

Jl4L7

Jli:IJJ'!

Jlt:L(!J)

14LJ1@

ll6loH

735

complementary structure. Determine the total number of real to decode the code. Decode the (8, 4) RM code with the trellis-based recursive MLD beginning of the the code is divided into four and each section consists of 2 bits. The metric table for each of these basic sections is constructed Devise a recursion to combine these metric tables to form metric tables for longer sections until the full of the code is reached (i.e., a for combining metric For each combination of two tables using the . y; ::) construct the two-section trellis T({x, y; z)) for the code P,.i-(C). Determine the number of real required to decode the code with the RMLD-(I,V) algorithm. Decode the (16, 5) Rl\1 code with the RMLD-(I,V) algorithm using uniform sectionalization. At the the code is divided into of 2 bits path metric tables. For each. Devise a rncursion procedure to combine each combination of two adjacent metric construct the special two-section trellis for the corresponding code. Determine the total number of real required to decode the code. Repeat Problem 14.6 dividing the code into four 4 bits per at the beginning of the recursion. Compare the computation complexity of this recursion with that of the recursion devised in Problem 14.6. Devise an iterative decoding algorithm Lased on a minimum-weight trellis search using the ordered statistic with order-1 reprocessing (presented in Section 10.8.30) to generate candidate codewords for optimality tests. Analyze the computational complexity of your algorithm. To reduce decoding computational complexity, the order i should be small, say i = 0, 1, or 2. The advantage of ordered statistic decoding over the Chase-H decoding is that it never fails to generate candidate codewords. Simulate the error performance of the iterative algorithm devised in Problem 14.8 for the (32, 16) RM code using order-1 reprocessing to generate 17 candidate codewords for testing and search of the ML codeword. Determine the average numbers of real operations and decoding iterations required for various SNR. Decode the RM code with MAP and Max-log-Map algorithms based on a uniform four-section trellis. Simulate and compare the error performances for two algorithms, and compare their computational complexities. The 16) RM code can be decomposed into eight parallel and structurally identical four-section subtrellises. Decode this code with the parallel Max-logMAP algorithm. Compute the number of real operations required to process a single subtrellis and the total number of real operations required to decode the code. Also determine the size of the storage to store the branch metrics, state metrics, and the likelihood ratios.

""Error Bounds for Convolutional Codes and an Decoding Algorithm," IEEE Trans. Inform. Theory, 13: 260-69, 1967. '.2o G.D.

1973.

Jr., ''The Viterbi Algorithm," Proc. IEEE, 61: 268- 78, March

136

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

3. A. Lafourcade and A. Vardy, "Optimum Sectionalization of a Trellis," IEEE Trans. Inform. Theory, 42: 689-703. 1996. 4. R. J. McEliece, "On the BCJR Trellis for Linear Block Codes." IEEE Trans. Inform. Theory, 42: 1072-92, July 1996.

5. R. J. McEliece, "The Viterbi Decoding Complexity of Linear Block Codes," Proc. IEEE Intl. Symp. Inform. Theory, p. 341, Trondheim, Norway, June 1994. 6. H. Thirumoorthy, "Efficient Near-Optimum Decoding Algorithms and Trellis Structure for Linear Block Codes," Ph.D. dissertation, Dept. of Electrical Engineering, University of Hawaii at Manoa. November 1996. 7. B. Honary and G. Markarian. Trellis Decoding of Block Codes, Kluwer Academic. Boston. Mass., 1997. 8. S. Lin, T. Kasami. T. Fujiwara, and M. P. C. Fossorier, Trellises and Trellis-Based Decoding Algorithms for Linear Block Codes, Kluwer Academic. Boston, Mass., 1998. 9. H. T. Moorthy, S. Lin. and G. Uehara, "Good Trellises for IC Implementation of Viterbi Decoders for Linear block Codes." IEEE Trans. Commun., 45: 52-63, January 1997. 10. E. Nakamura, G. Uehara, C. Chu, and S. Lin. "A 755 Mb/s Viterbi Decoder for RM(64, 35, 8)," Proc. IEEE Intl. Solid-State Circuit Conf, San Francisco, Calif., February 1999.

n.

T. Kasami, T. Takata, T. Fujiwara. and S. Lin, "On Structural Complexity of the L-Section Minimum Trellis Diagrams for Binary Linear Block Codes," IE/CE Trans. Fundamentals. E76-A (no. 9): 1411-21, September 1993.

12. N. Deo, Graph Theory with Applications to Engineering and Computer Science, Prentice Hall, Englewood Cliffs. N.J .. 1974. 13. M. P. C. Fossorier, S. Lin, and D. J. Rhee, "Differential Trellis Decoding of Convolutional Codes," IEEE Trans. Inform. Theory. 46: 1046-53, May 2000. 14. T. Fujiwara, H. Yamamoto, T. Kasami, and S. Lin, "A Recursive Maximum Likelihood Decoding Procedure for a Linear Block Code Using an Optimum Sectionalized Trellis Diagram," Proc. Thirty-Third Annual Allerton Conf on Commun. Control and Computing. pp. 700-709, October 4-6.1995. 15. T. Fujiwara. H. Yamamoto. T. Kasami . and S. Lin. "A Trellis-Based Recursive Maximum-Likelihood Decoding Algorithm for Binary Linear Block Codes," IEEE Trans. Inform. Theory, 44: 714-29. March 1998. 16. T. Kasami. H. Tokushige, T. Fujiwara, H. Yamamoto, and S. Lin, "A Recursive Maximum Likelihood Decoding Algorithm for Some Transitive Invariant Bimiry Block Codes," IEICE Trans. Fundamentals, E81-A (no. 9): 1916-24, Qr-,,nto1n,,hc,.r 1 OOQ

\.JVJ!l.V.1.J.J..VV.ll

_l_//V,

Bibliography

1'371

17, H. Moorthy, S. Lin. and T. Kasami, "Soft-Decision Decoding of Linear Block Codes Based on an Iterative Search " IEEE Trans. Inform. 43: 1030-40. M[ay 1997. Measurement 1972.

for Decoding of Block Codes with Channel 18: 170-81.

n.c,,0wcu,,

and S. Lin, "A Trellis Based for Linear Block Codes," IEEE

"Multilevel Codes and Multistage March 1989. S. T. and S. Lin. Block M-PSK Modulation Codes," IEEE Trans. 1993.

::lrn, T.

z::t

Decoding of Multilevel Theory, 39: 1024-18.

S. T. T. and T. Takata, "An Upper Bound on the 5ffective Error Coefficient of Two-Stage Decoding and Good TwoLevel Codes," IEEE Trans. Commun., 42: 813-18, Febru1994.

J.

Z2lo T. Y. T. Fujiwara. T. Kasami, and S. Lin, "Suboptimum Decoding of Decomposable '' IEEE Trans. Theory. 40: 1392-1405, 1994. Z-4\o G. Schnabl and M. "Soft-Decision Decoding of Reed-Muller Codes as Generalized Concatenated Codes," IEEE Trans. Inform. 41: 304-8, January 1995. 1,§,

D. Stojanovic. M. P. C. Fossorier, and S. Lin. "Iterative Maximum Likelihood of Multilevel ·• Proc. Coding and Cryptograph, pp. 91-101. Paris, France. 11-14, 1999. R. F H. Fischer, and J. B. Huber, "Multilevel Codes: Theoand Practical Design Rules," IEEE Trans. Inform. Theory, 45: July 1999.

u~,,.,u.muu,

27, N. Seshadri and C. W. "List Viterbi Decoding Algorithms with Applications," IEEE Trans. Commun., 42: 313-22, February/March/April, 1994. Zit L. R. Bahl. J. F. Jelinek, and J. Raviv, "Optimum Decoding of Linear Codes for Minimizing Symbol Error Rate," IEEE Trans. Inform. Theory. 20: 284-87, March 1974. 1,<9),

C. A. and P. Thitimajshima, "Near Shannon Limit ErrorCorrecting Coding and Decoding: Turbo Codes," Proc. IEEE Intl. Conf Commun.. Geneva. Switzerland, pp. 1064-70. May 1993.

138

Chapter 14

Trellis-Based Soft-Decision Decoding Algorithms

30. J. Hagenauer, E. Offer, and L. Papke, "Iterative Decoding of Binary Block and Convolutional Codes," IEEE Trans. Inform. Theory, 42: 429-45, March 1996. 31. P. Robinson, E. Villebrun, and P. Hoeher, "A Comparison of Optimal Suboptimal MAP Decoding Algorithms Operating in Log Domain," Proc. Intl Conf Commun., Seattle, Wash., pp. 1009-13, 1995. 32. A. J. Viterbi, "An Intuitive Justification and a Simplified Implementation of the MAP Decoder for Convolutional Codes," IEEE J. Selected Areas Commun., 16: 260-64, February 1998. 33. Y. Liu, M. P. C. Fossorier, and S. Lin, "MAP Algorithms for Decoding Linear Block Codes Based on Sectionalized Trellis Diagrams," Proc. IEEE GlobeCom. Conf, Sydney. Australia, pp. 562-66, November 1998. 34!. Y. Liu, M. P. C. Fossorier, and S. Lin, "MAP Algorithms for Decoding Linear Block Codes Based on Sectionalized Trellis Diagrams," IEEE Trans. Commun., 48: 577-587. April 2000. 35. J. Hagenauer and P. Hoeher, "A Viterbi Algorithm with Soft-Decision Outputs and Its Applications," Proc. IEEE GlobeCom. Conf, Dallas, Tex .. pp. 1680-86, November 1989. 36. M. P. C. Fossorier, F. Burkert, S. Lin, and J. Hagenauer, "On the Equivalence between SOVA and Max-log-MAP Decoding," IEEE Commun. Lett.. 2: 137-49, May 1998.

[Q) ®~@ Uuu [Pl~)) [Q)®~@(d]~

used as the outer codes. Concatenation caa ais,) be with multiple outer inner codf:s. Ivfultilevel concatenation provides error-control systems more codes 21nd in with different rates for vanous corrnnunicc1don e1nironments. Using multilevel many coGstructed from very short inner and outer codes. The mul.tilevel concatenated codes from short inner and OPtef codes is long codes as multilevel concatenated codr::s fm Hhid1 the codes are short and simple. Multilevel concatenated codes can be decoded in ,,vu"·''"'''"'~ with reduced decoding coma;-e decoded one at a time in

and multilevel concatenations for error

A and an are

concatenated code is formed from two codes: an (111. k1) binary code C1 k2) code with GF(2k1). The symbols of C2 (or k1-tuples).

(112,

740

Chapter 15 Outer encoder (n 2 , k 2 ) code over GF(i'1)

Concatenated Coding, Code Decomposition Inner encoder (n1, k1) binary code

~----Encoder-----

Channel

~----Decoder-----

FIGURE 15.1: Communication system using a concatenated code.

Usually, an RS code is used for C2. Encoding consists of two steps, as shown in Figure 15.1. First, the k1k2 binary information digits are divided into k2 bytes of k1 information digits each. These k2 bytes are encoded according to the rules for C2 to form an n2-byte codeword. Second, each k1 -digit byte is encoded into a codeword in C 1, resulting in a string of n2 codewords of C 1, a total of n2n 1 digits. These digits are then transmitted, one C 1 codeword at a time, in succession. Thus, the resultant code is an (n1n2, k1k2) binary linear code. The component codes C1 and C2 are called the inner and outer codes, respectively. Uthe minimum distances of the inner and outer codes are d1 and d2, respectively, the minimum distance of their concatenation is at least d 1 d2 (see Problem 15.1). This type of concatenation of two codes is known as one-level concatenation. The concatenated code of C1 and C2 is also decoded in two steps, as shown in Figure 15.1. First. each C1 codeword is decoded as it arrives, and the check digits are removed, leaving a sequence of 112 k 1 -digit bytes. These bytes are then decoded according to the decoding method for C2, to leave the final corrected message. Decoding implementation is the straightforward combination of the implementations for codes C1 and C2, and the amount of hardware required is roughly that required by both codes. Concatenated codes are effective against a mixture of random errors and bursts, and the pattern of bytes not correctable by the C1 code must form a correctable error pattern for C2 if the concatenated code is to correct the error pattern. Scattered random errors are corrected by C1. Burst errors may affect relatively few bytes, but probably so badly that C 1 cannot correct them. These few bytes can then be corrected by C2. IEXAMIPlE 15.1

Consider the concatenation of the (15, 11) RS code with symbols from GF(2 4 ) and the (7, 4) binary Hamming code. Each code symbol of the RS code is represented by a byte of four binary digits, as in Table 2.8. Then, each 4-bit byte is encoded into a codeword in the (7, 4) Hamming code. The resultant concatenated code is a (105, 44) binary code. Because the minimum distance of the (7. 4) Hamming code is 3, and the minimum distance of the (15, 11) RS code is 5, the concatenated code has a minimum distance of at least 15. Uthe code is decoded in two steps. first the inner code and then the outer code, the decoder is capable of correcting any error pattern such that the number of inner codewords with more than a single error is less than 3. ftJ,. simple generalization of the concatenated coding is to interleave the outer code. Let the outer code C2 be an (n2. k2) linear block cocle wi.:h symbols ii'Oi,,

Section 5.1

Concatenated Codes

1~~.1

i=~k,--/1,->I~>I ~--i,-

m-bit b~te

k 1 bits

0

C:,

-<'-.-

0

First outer codev;,rord

I

0 0 0

_,_ Mh outer codeword

~

~

First inner codeword

flGURE 15.2:

11 2th

inner codeword

An interleaved code array.

linear code with k1 = Am, where ), and the inner code C1 be an (n l · kt) integer. A message of k2 m-bit (or k2m is first encoded into an 112-byte codeword in This codeword is then stored in a buffer as a row in an array, as shmvn in 15.2. After 'A outer codewords have been the buffer stores a A >< 112 array. Each column of the array consists of A (m-bit) )cm and is encoded into an 11 1-bit codeword in C1 . Each encoded column is transmitted Therefore, the outer code is interleaved by a depth of A. When an inner codeword with possible errors is received, it is decoded and the parity bits are removed. The decoded A bytes ( or Am bits) are stored in a receiver buffer as a column in a A x n2 array. After 112 inner code decodings, the receiver contains a A x 112 decoded array. Then, each row of this array is decoded based on the outer code Therefore, there are a total of >.. outer code ~~~vuu,;c,~ The overall interleaved concatenated coding system is shown in Figure 15.3. H is possible to construct a concatenated code from a single outer code and multiple inner codes. For one may use an (n2, k2) code C2 with from GF(i'l) as the outer code and 112 (111, k1) codes, Ci 1), c?l, · · · , , as inner codes. Again, the encoding consists of two steps. First, the lqk2 information digits are divided into k2-bytes of lq -digits each. These k2-bytes are encoded according to the rules for C2 to form an 112-byte codeword, (ao, a1, · · · , a112 _i), where each byte a; is regraded as an element in ). Second, the ith a; for O < i < 112 is encoded into a codeword in the ith inner code l. The overall encoding again 'A is a

cf

742

Chapter 15

~

Concatenated Coding, Code Decomposition

Outer encoder (n 1, k2 ) code over GF(2"')

~·----~

Interleaver (buffer)

Inner encoder (ni, k1)

binary code

Channel

-

Outer decoder (11 2 , k 2 ) code over GF(2"')

Deinterleaver (buffer)

Inner decoder (111, k1) binary code

FIGURE 15.3: An interleaved concatenated coding system.

results in an (111112, /qk2) concatenated code. By concatenating one outer code with multiple inner codes, Justesen [2] was able to construct a class of asymptotically good concatenated codes. Concatenated coding has been widely used in digital communication and storage systems to achieve high reliability (low bit-error probability) with reduced decoding complexity. The inner code is usually short and is decoded with softdecision decoding. An RS code is commonly used as the outer code and is decoded with hard-decision decoding using an algorithm presented in Chapter 7 (the Euclidean algorithm is the most popular one).

F!G URE 15.Ll: A snbtrellis of the 8-s':'ction trellis for the (64. 40) RM sn bcode

Section 15.2

Multilevel Concatenated Codes

71£!3

100 ~ - ~ - ~ - - ~ - ~ - ~ - - ~ - ~ - ~ - - ~ - = - - Uncoded BPSK -- RM(64. 40) & (255,223) concatenation -- NASA standard 64-state convolutional code """' (255,223) RS code -,~ (64, 40, 8) optimum decoding

2

3

4

5

6

7

8

9

10

11

E 1,IN11 (dB)

FIGURE 15.5: The bit-error performances of the (64, 40, 8) RM subcode and the concatenated coding scheme with the 8) RM subcode as the inner code and the NASA standard 223, 33) RS code as the outer code.

As an we consider a concatenated error-control scheme in which the outer code is the 223, 33) RS code with symbols from GF(2 8 ), and the inner code is a (64, 40) linear code with a minimum distance of 8. The outer code has a minimum distance of 33 and is capable of correcting 16 errors. The generator matrix of the inner code is obtained removing two rows from the generator matrix of the 42, 8) RM code. The trellis of this inner code consists of 32 parallel and structurally identical subtrellises, each with 64 states. One such subtrellis is shown in Figure 15.4. The parallel structure allows us to devise 32 identical subdecoders to process the subtrellises in parallel to achieve high decoding speed. fo concatenation with this (64, 40) inner code, the (255, RS outer code is interleaved to a depth of 5. The overall rate of the concatenated coding system is 0.545, and its bit-error performance is shown in Figure 15.5. H achieves a BER of 10- 6 at 3.1 dB SNR. 1§.2

Ml\JLlllLIEVIEL (ON(AllEi,!AllEIDJ (OIDJIES

Single-level concatenation can be generalized to multilevel concatenation [5, 6]. In a multilevel concatenated coding system, multiple inner and outer codes are used to form concatenated codes, and these concatenated codes are then combined to form an overall concatenated code. Multilevel concatenation provides an even more powerful coding technique for error control than single-level concatenation and offers more flexibility in designing error-control systems for various communication environments. Furthermore, the multilevel structure allows the use of multistage decoding to reduce decoding complexity.

744

Chapter 15

Concatenated Coding, Code Decomposition

+ ---

- - l > ~ . _ _ _ __:_J,_,,,(_~_,,,_i)_=_a_;"_'1~

~_

FIGURE 15.6:

[A,,,/(0)]

An encoder for an m-level concatenated code.

An m-level concatenated code is formed from a set of m inner codes and a set of m outer codes, as shown in Figure 15.6. Them inner codes are coset codes formed from a binary (n, k) linear block code A1 and a sequence of m linear subcodes of Ai, denoted by A2, A3, ... , A 111 +1 = {O}, such that (15.1) where A111 +1 contains only the all-zero codeword 0. For 1

:s: i :s: m, let

1. k; and dA; be the dimension and minimum distance of A;, respectively. 2. [A;/ A;+1] denote the set of representatives of cosets in the partition A;/ A;+l · [A;/A;+i] is a linear code with 2k;-k;+1 codewords, and hence its dimension is k; - k;+l· This code is called a coset code. For 1 :s: i :s: m, let q; ~ [[A;/ A;+1][ =

2k;-k;+1,

(15.2)

and [A;/A;+1] ={a;°: 1 :S: j :S: q;}.

(15.3)

Then, a coset in A;/ A;+1 is given by (i)

.

(a.J +a. a

E

A;+1L

and Cfi

A;

=

(i)

U(ai

+ A;-1-1)

j=l

=

[A;/A;-1-1] EB A;+l·

(15.4)

Concatenated Codes

Section

v,1e

From

145

see that EB ... EB

(15.5)

from the codes 15.6 . . is an (N. Ki) linear block where q; is defined (15.2). the ith level

For 1 :s i :Sin~ the code over vvith a In the m··level concatenation concatenated code has as the outer code denote this concatenated cede

I A;+tl·

o

over GF(q 1 ) is encoded

a rnessage of

each -···--·-,,.,,_ into a codevvord

J -

-

in the ith outer code Bi for 1 1 )

r1·)') 'O

:s i :s

bUl of fo/il is encoded

m.

1

in the ith inner code /Ai+iJ, where f 1 (,) denotes the ith inneT code encodeT. The of the ith inner code sequence: \

!,

"(lJi), 1 1

.Ii

1

"(l(i) 7

·· · · · • }i

N-1

))

·

,vhich is a codeword in the ith level conca,tena1ecl code Bi o [A;/Ai+l] and is the output a sequence of coset fror.o I A1+ilsequence of the overall encoder of the m-lc:vel1 concatenated system shown in 15.6 is the su.m: (15.6) =".(I)+

_L

_I_ I • • •

I

where for O :s j < JV, lil

~·. ']

h

1·

.Ii

(15.7)

j.

/=I

of is a codeword in A1 . Therefore,

From (15.7) we see that each rc is a sequence of N codewords in The collection of sequences, 1/l

C

==

F

{(~o, ~1- ... , ~N-1): tJ

"/_(i),

Ji\u

1

!

l

(i)

l

(i)

J "lno (7 0 , '7 1 , ... ,

b(i)

iv-i) E Bi

i=l

for 1 :Si

:s m

and O :s j < N).

(15.8)

746

Chapter 15

Concatenated Coding, Code Decomposition

forms an m-level concatenated code. For simplicity, we denote this concatenated code with C

6

=

(15.9)

{B1, B2, ... , Bm} o (A1, A2, ... , And·

Note that for 1 ::: i, j ::: m, and i i= j, [Ai/Ai+ll and [Aj/Aj+i] have only the all-zero codeword OI in common; that is,

(15.10) It follows from (15.6), (15.8), and (15.10) that the m-level concatenated code C is the direct-sum of m-component concatenated codes; that is,

(15.11) The minimum distance of C is lower bounded as follows:

(15.12) (see Problem 17.2). The dimension of C is Ill

I( =

L IC (ki -

ki+1).

i=I

Let n = 8 and N = 8. We choose A1 and its subcodes as the following RM codes of length 23 = 8: A1 = RM(3, 3), A2 = RM(2, 3), A3 = RM(l, 3), A4 = RM(O, 3), and As = {OI). A1 is simply the (8, 8) universal code with minimum distance dA 1 = 1; A2 is the (8, 7) even parity-check code with minimum distance dA 2 = 2; A3 is the (8, 4) first-order RM code with minimum distance dA 3 = 4; and A4 is the (8, 1) repetition code with minimum distance dA.i = 8. Because RM(3, 3) => RM(2, 3) => RM(l, 3) => RM(O. 3) => {©),

we can form the following coset codes: [Ai/A2]

= [(8. 8)/(8, 7)],

[A2/ A3]

= [(8, 7)/(8, 4)],

[A3/ A4] = [(8, 4)/(8, 1)], [A4/{®}]

= [(8, 1)/{0i}].

The dimensions of these four coset codes are 1, 3, 3, and 1, respectively. Therefore, q 1 = 2, q2 = 23 , q3 = 23 , and q4 = 2. fo the construction of a 4-level concatenated code with the preceding coset codes as the inner codes, the four outer codes Bl, B2, B3, and B4 must be codes over GF(2), GF(23 ), GF(2 3), and GF(2), respectively. Suppose we choose the following four outer codes: L B1 is the binary (8, 1) repetition code with minimum distance ds 1 '.20 B2 is the extended (8 . 5) RS code over GF(23 ) with ds 2 = 4:

= 8;

Section 15.2

is the extended (8, 7) RS code over is the (8, 8) universal code with

Multilevel Concatenated Codes

with dB-1

= 2; and

= 1.

The 4-level concatenated code constructed from the codes is

inner and outer

which is a linear block code. From of C is lower bounded dmin(C)

the minimum distance

>< 8, 2 >< 4, 4 >< 2, 8

:::0::

7.ll.7

X

J_}

==

8.

= 8. This cod,s has the same cede BCH code obtained

as bit

BCHcode.

we choose the of a 3-level concatenated code:

codes as the coset inner codes in the construction

3)/ RM(l, 3)]

= =

[(8, 7)/(8, 4)], [(8. 4)/(8,

[A3/{@}] = [RM(O, 3)/{11Jl}] = [(8, 1)/{@}]. and must have from GF(2 3 ), and we choose the as the outer codes: B1 is 1) extended RS code over with a minimum distance of 8; (2) the (8. extended RS code over with a minimum distance of 4: and is the (8, 7) binary even code with a minimum distance of 2. Then, the 3-level concatenated code

C=

0

3)] EB

o

[RM(l, 3)/ RM(O. 3)] EB

0

is a linear block code with a minimum distance of 16. This code has one information bit more than the 24) extended BCH obtained an overall bit to the BCJH1 code. The two show that the multilevel construction of concatenated codes results in good codes. Good multilevel concatenated codes and their trellis can be found in [9]. An m-kvel concatenated code can be decoded in m stages. The first-level concatenated code B1 o [Ai/ A2] is decoded and the mth level. concatenated code Bm O [ Am I{@)] is decoded at the last 3 codeword (C ""'"'""' 0'"'"~,·li, to m outer codewords lbJ(l). il:D< 2 l •...• lb)(m l is transmitted. Let ir = (iro. rr 1, ... , irN ~ 1) be

748

Chapter 15

Concatenated Coding, Code Decomposition

the received sequence, where each received component r.i is the sum of a transmitted f"nrlP'PVnrrl -in .._,.._,,....._....,:;·;.._,._._ ...... .!...ti.JI.

A. .l.l

9\nrl CT.LI. ,;i,n Prrra'l\r V.l...d..._..,..J..

At

'lJPf'>tnr

G.1...1_1..l

l'V'l,.;1.,>J.l.o

thP. t...LlV

ll..l.

tirct 0l.

.L.1.

,1,::,.,-,.r,.,,--l;nirr .-:_,t0.n-e , UVVV\...l.1.lll..i:, .:,t.Ut::,

T'.(l) J1

~Tl"~("'

-

J1

into a codeword ilJ,(ll = (b6ll, b?>, ... , br~ 1) in the first-level outer code modified received sequence is formed: i/2)

=

r1Dr-A'>.rlnr1

.lU \...l\,..,\.,VU\,.;U

B1.

Then, a

(11'62), ll'i2), ... , ll'~~l)

_

-ll'

(1)

(1)

.

(1)

(1)

(15.13)

-(f1(bo ),.fi(bl ), ... ,fl(bN-1)).

If there are no errors in r< 1l = r, each component i 2l is a codeword in A2; otherwise, J

J:j2l of r<2l is the sum of a codeword in A2 and an error vector. At the second decoding stage, r< 2l is decoded into a codeword b< 2l = (b62 ), \b)?l, ... , b~;) in the second-level outer code B2. Then, another modified received sequence is formed: ll'

(3) _

(3)

(3)

(3)

-(ll'o ,r1 , ... ,ll"N-1)

= 11'(2 ) - (h(b62 )), Again, if there are no errors in r< 1l 3

f2(b;

2 \

(15.14) ... ,

h(h~~l)).

= r, each component / J3l is a codeword in

A3;

otherwise r_j ) is the vector sum of a codeword in A 3 and an error vector. At the third-stage decoding, r< 3 l is decoded into a codeword in the third-level outer code B 3 . This decoding process continues until r< 111 l is formed and decoded into a codeword Jbi< 111 l in the mth level outer code Bm· Each decoding stage consists of two steps, the inner and outer decodings. At the ith decoding stage, the modified received sequence ll'(i)

=

1

(Ir6)' ... ' 11'~ -1)

is first formed based on the decoding results of previous decoding stages. The inner code decoder decodes each component /il into a codeword in Ai and identifies J the coset in Ai/ Ai+l that contains the decoded codeword. Let .fi (hjil) denote the representative of this coset. By inverse mapping, we obtain ,;
=

(l-ul l-uJ 7 '70 '

l-ui

1 '· · ·' 7N-I ).

This vector is then decoded based on the ith level outer code Bi into a codeword llJiCi) in Bi. This completes the ith decoding stage. The decoder proceeds to the next decoding stage. In the foregoing multistage decoding, the decoded estimate at one stage is passed to the next stage for decoding the next component concatenated code. If a decoding error is committed at a stage before the last stage, error propagation is likely to occur. 15.3

A SOIFT-DIEOS!ON MULTISTAGE DIE(ODING

The multistage decoding presented in the previous section can be modified for soft-nP,cisi')TI rJpr-('lrling nf

rn11Jti]P'1P] C'OTIC-::JtPn::,tprJ ror,npc ,:,nrJ rJpro0rnprwc,hjp N>rJpc

Section 15.3

A Soft-Decision Multistage Decoding

74~

complexity, each level is decoded with a trellisConsider the multilevel concatenated shown in Figure 15.6. For convenience we reintroduce some notations that were defined in the section:

JL rr = (ll'o, :1, . . . .

0 is the soft-decision received sequence at the of the matched filter, where for O ::=: j < N, Fj = (rj.O, rj.l· .... rj.n-1) is the received

2"

n·~/_

=

(rtgi. . .... ) is the modified received sequence for the ith stage 1 1 wlv"'··e for O < 1· < N o 1/il l )' U\/VV''-'""~" J,A C ' j = (rCil j.0' rU) j.1' '' '' r
2t ~(il

=

(bgi, biil, .... b~/_ 1) is the decoded codeword in outer code

-'.·,

=

; (i) (i) ) 1s . th \\\:o . ir 1(i) ..... irN-l •e

jl

nated code

o

at the ith

1 di d ecooe . ~1 .. h leve_1 conca . t eco d ewor d m l1e 1c [A;/ A;+iJ, where for O ::=: j < N, (i) (i) (i)

=(c.o,c·1, .... c.J,11- 1> ]. ].

= .f;(b(.i\ J the (i - l)th stage of has been completed. the received sequence rrUl = (lr~il. for the ith stage of u v v ~ · u u , ~ is obtained from ) 1 ,rCi-ll and 1rU-ll as follows: for O ::=: l < n and O ::=: j < JV, the /th of the jth section rl l of li'(i l is by

il'i J, ... ,il't~

.I

.(i) __ (i-1). (1 _ I j.1 _

I j.i -

.(i-1)) 2lj.l

(15.15)

consists of two the inner and outer decodings. the inner decoder processes the N sections of the independently and forms N metric tables for the outer code

~vv~o,u"'~

At received sequence us~ov,A.HH,;e.,.

coset

Let {:m(il} ~ {:m(il + :Bl : :m E A;+d be a coset in / A;+1 with :m(i) as the The metric of {:mU l} with respect to rl] l is defined as the largest

correlation metric between i/il and the vectors in {:mUl}, denoted by M({:mUl}). Let J

+ 21* be the vector in {ta1Ulj ,vith the largest metric. This vector is called the label of {ta1(i)} with respect to rrjl· A metric table, denoted by MTyi. is formed that for each coset {ta1Ul} E A;/A;+l• stores its metric il1({21Ulj) and its label ta1Ul + 21*. This table is caHed the metric table for rr(il. The inner decoding at the ith stage is to form .I

N metric one for each section of the received sequence rr·Ul. These N metric tables are then to the outer decoder. For a codeword 11:DUl in the ith outer code B;. the metric of 11:DUl is defined as the following sum: N-1

M(ll:D(i)) ~

L M({f;(bji))}),

(15.16)

j=O

where M({f;(b(.il)}) is the metric of the coset {f;(b<.il)} with f;(b(.iJ) as the coset J

J

]

leader that is stored in table MT(il. The outer code decoding is to find the codeword J

150

Chapter 15

Concatenated Coding, Code Decomposition

l!J,Cil E B; that has the largest metric among all the codewords in B;, by using a

trellis-based decoding algorithm, such as the Viterbi algorithm or RMLD, to search through the trellis for B;. Owing to possible error propagation from one decoding stage to the next, the foregoing multistage decoding is not optimum (i.e., it does not achieve MLD performance), even though each stage of decoding is MLD. It is a suboptimum decoding algorithm; however, it provides an efficient trade-off between error performance and decoding complexity for multilevel concatenated codes and decomposable codes. The performance of multistage decoding can be improved by passing a list of L best estimates from one stage to another and then choosing the estimate with the largest metric at the last decoding stage as the final decoded codeword, as with the list Viterbi algorithm [20] at each stage. The improvement and the additional decoding complexity is determined by the size of the list. 15.4

IDIECOMPOSITION OIF CODES

The opposite of multilevel construction of concatenated codes presented in the previous section is decomposition of codes into component codes. A code is said to be µ-level decomposable if it can be expressed as a µ-level concatenated code. Such a decomposable code can be decoded in multiple stages, as described in the previous section. Multistage decoding of long decomposable codes reduces decoding complexity significantly, because after decomposition, component codes are shorter in length and smaller in dimension, and require less decoding complexity. The most well known class of decomposable codes is the class of RM codes. In Section 4.4, it was shown that a RM code can be decomposed in many ways using squaring construction; however, a RM code also can be decomposed as a multilevel concatenated code [10, 11, 14-17]. Consider the rth-order RM code of length 2 111 , RM(r, m). Let k(r, m) and q(r, m) denote the dimensions of the RM(r. m) code and the coset code, [RM(r, m )!RM(r- l, m )], respectively. (Note that RM(-1, m) = {@}). From Section 4.2 we find that k(r, m)

=

t (7;) 1=0

and q(r,m)=

For O .::: l .:::

r,

(m)r .

(15.17)

we can readily show that /-1

k(r, m) = k(r -!, m)

+ Lq(r -

i, m).

(15.18)

i=O

Let (n, k)" denote the linear block code over GF(2'") obtained by interleaving the binary (11, k) code to a depth of).. with each group of).. interleaved bits regarded as a symbol in GF(2'"). Then, from (4.71), it is possible to decompose the RM(r, !11) code

of Codes

'I.

15'1

concatenated code as follows v)q(r.111-v).

RM:

o{RM(r. m - v). RJl!f.(r

;n -- v). · . · .

(15.19)

-/_t,m-v)},

where 1 :::: v :::: m - 1. t,L = v for > ,J, and ,u == ,. othen;vise. (15.19) shows thcr1 as a multilevel concatenated code in many ·Nays also note that in any Uvv~Hc"),:,G,Si',·,~.~ of a RM code. aH the r,~,~--.~, inner and outer codes are also RM

Let m = 6 and minimum di.stance of 8. Set RM(3. 6) code can be 6)

fol

=3

6) code that is a (64. 42) code with a it follows from (15.19) that the into a t!.-level concatenated code as follows:

= 3) }.

3) is the (8, 1) 3) = t minimum distance of 4; RM(2, 3) 8) universal code. Hence, 3) is the

RM(3 6) = {(8.

= (8.1) 0 [(8. 8)/(8.

3)}o{(8.8). EB (8.

o

EB

8)

EB(8. 7)3 o

7). (8. 4). (8, 1)}

WL 7)/(8. 4)] 0

[(8, 1)/(®}].

(15.20)

A tricliis can be ccns1tmctsd for each leve]l of concatenated code. The (8. 1) code has a trivial 2-state 8-section trellis. the first-level concatenated code (8. 1) o 8) / (8, 7)] has and each section is 8 bits The (8., code has a "1-state t!.--section ::is shown in Figure 9.17. By Cartesian interleaved codF; has a 4-section trellis with 64-states and and each has 8 states. 4) 3 o [(8, 7)/(8, 5)] has a 4section 64-state each section is 16 bits and the 16 bits of a branch label represent two codewords in the inner code [(8. 7) / (8. 4)]. l[f the 8-section bit-level trellis as shown in 9.6 is used for the (8, 4) then the (8. 4) 3 code has an 8-section trellis with 512 states at time-3 and time-5. the second-level concatenated code (8. o [ (8, 7) / (8. has an 8-section trellis, each section is 8 bits long. and each branch label. is codeword in [(8, 7)/(8, 4)]. Now, consider the third-level concatenated code. The 7) even code has an 8-section 2-state as shown in 15.7. the (8. 7) 3 interleaved code has an 8-section 8-state trellis. The third--level concatenated code 7) 3 o [(8, 4)/(8, 1)] has and each 8-bit branch label is an 8-section 8-state each section is 8 bits a codeword in the inner code [(8. 4)/(8, 1)]. the fourth-level concatenated code (8. 8) o [(8. 1)/{@}] has a trivial 1-state 8-section each section is 8 bits 1) code. long, and each 8-bit branch label is a code NO"d in the 1

752

Chapter 15

Concatenated Coding, Code Decomposition

FIGURE 15.7: The 8-section bit-level trellis for the (8, 7) even parity-check code.

The preceding example shows that when we decompose a code into an m--level concatenated code, we also decompose its trellis into m trellises, one for each level of concatenated code. The trellis for each level of concatenated code has much smaller state and branch complexities than those of the original code. In multistage decoding of a decomposable code, each level can be decoded by using a trellis-based decoding algorithm, which reduces decoding computational complexity significantly. As an example, consider the fourth-order RM code of length 128, RM(4, 7). This code is a (128. 99) code with a minimum distance of 8. This code has a uniform 16-section trellis with maximum state complexity of 2 19 states. Implementation of any decoding algorithm based on this full-code trellis is practically impossible; however, this code can be decomposed into a 3-level concatenated code as follows [17]: RM(4, 7)

= {(16, 5)(16, 11). (16.11)2, (16. 15) 3 (16. 16)} o {(8, 8). (8, 6), (8. 4))

where (16, 5) and (16, 11) are first- and second-order RM codes of length 16. Both (16, 5) and (16, 11) RM codes have 16-section bit-level trellises with maximum state complexity of 16 states (see Figures 9.7 and 9.8). The first-level outer code B1 = (16, 5)(16, 11) is an interleaved code and is regarded as a code of length 16 over GF(2 2 ). This code has a 16-section trellis with maximum state complexity of 256 states. The second-level outer code B2 = (16, 11) 2 has a 16-section trellis with maximum state complexity of 256 states. The third-level outer code B3 = (16. 15) 3 (16, 16) has a 16-section trellis with 8 states. The binary (8, 6) code is a subcode of the (8, 7) RM code. The preceding decomposition results in relatively small trellis state complexity compared with the 2 19 states of the original undecomposed code. Therefore, three-stage decoding of this code requires relatively little decoding complexity. The bit-error performance of this code with the 3-stage soft-decision decoding presented in the previous section is shown in Figure 15.8. Performance is degraded by 0.9 dB compared with optimum MLD at BER of 10- 5 , but there is a 1.7-dB coding gain over the majority-logic decoding of the code. As another example, consider the third-order RM code of length 128, 7), which is a (128, 64) code with a minimum distance of 16. The 16-section trellis for this code has a maximum state complexity of 226 states! This code can be decomposed into a 3-level concatenated code as follows [17]: RM(3, 7)

= {(16, 1)(16. 5) 2 , (16. 5)(16. 11). (16, 11)2(16, 15)) o {(8. 8). (8, 5), (8, 3)).

With this decomposition. the third-level outer code B 3 = (16, 11) 2 (16. 5) has the largest trellis complexity. 512 states, which is very sm;ill comp;irerl with 726 . Thtc bit-·error performance of thi~ code ~nith 3··Stage decoding is ~hcrvvn in Figure 15.9.

Section 15.4 100

of Codes

753

I I ,, Majority-logic decoding x Multistage decoding o Optimum decoding ~ BPSK uncoded

10-1

10-2

g JS

"'

.D ,_,0 0..

10-3

1-a

0,_,

,_, 10-4 -

.~

iii 10-<

10-(;

10

-7

l

2

3

4

5

6

7

8

10

9

E1,IN0 (dB)

of the

x Majority-logic decoding ,., Multistage decoding ~ - ·-List-5 decoding o Optimum decoding

uncoded

10- 7 '---~-"~~-'-~~--'-~~-'--_____J_~~--'--~~-~~~-'--1 2 3 4 5 6 7 8 9 £ 1/Nu (dB)

10

FIGURE 15.9: The error performance of the (128, 64) RM code with various uvvv""""'"'u·

754

Chapter 15

Concatenated Coding, Code Decomposition

There is a 1.3-dB loss in coding gain compared with the optimum MLD; however the reduction in decoding complexity is enormous. H a list of five best estimates is passed from one decoding stage to the next stage, a 0.8-dB coding gain loss versus the optimum MLD can be recovered, as shown in Figure 15.9. 15.5

AN fflERATME MULnSTAGE MW ALGOmTHM

In fact, MLD can be achieved with the multistage decoding presented in Section 15.3 in conjunction with decoding iterations between stages and an optimality test at each stage. Iteration is initiated only when the optimality test fails. In the following an iterative multistage MLD (IMS-MLD) algorithm [17-19]. For simplicity, we first describe this algorithm using a two-level concatenated code, then we generalize it tom-level codes. Let l!J>Cll = (bill.bill, · · · , bi)) be the decoded codeword at the first stage. Then, ii1l has the best metric with respect to the received sequence rCll = ir. The metric of bPl is given by (15.16): M(lb>(l))

~

N

L M({,f, (b?))}). j=l

1

where M((f1 (b;1l)}) is the metric of the coset U1 (bt)} E A1/ A2. Let Ji (bj )) be the label of coset {ft (bf)} with respect to the jth section 1 M({f1 (b; l)}) = M(f1 (bt)

r; l of rC l. Then, 1

+ aj

1

N

+ rn';>,

and M(lb(lJ) =

L M(f1 (byi) + aj).

(15.21)

j=l

If we replace each component

sponding coset label

1 fi(b<. )) J

bcn 1

of the decoded codeword

]b(ll

with its corre-

+ m*, we obtain the following coset label sequence: (15.22)

From (15.21) and (15.22) we see that the metric of l!J>Cll is the metric of its corresponding coset label sequence. If the coset label sequence L(l!J>C 1l) is a codeword in the overall concatenated code C = {B1, B2} o {A1, A2}, then it is the codeword in C that has the best metric and hence is the most likely codeword. In this case the decoding is completed and the sequence a* = (m 1, m2, · · · , m)v) is a codeword in the second-level concatenated code C2 = B2 o A2. The second outer codeword lb( 2l = (bi21 , bi21 , · · · , bi)) can be recovered from m* as follows: !bP1 = (h- 1 (m7). f 2- 1 (212), · · ·, f 2- 1 (mt)), where f 2- 1 0 is the inverse mapping of .f2(·). Alternatively, for systematic encoding, the information bits are obtained directly from the coset label sequence. If the coset label sequence L(b( 1l) is not a codeword in C, then it is the sum of a codeword (1) . [ Cf1(b 1(1) ),.fi(b 2(1) ),···,fi(bN))mC1=B10 Ai/A2 ] andasequenceofcodewords in A 2 that is not a codeword in C2 = B2 o A2. In this case, decoding continues. Therefore, the condition that L(b,Cll) is a codeword in C is an optimality condition. which will be used in the IMS-MLD ;cilgorithm.

Section 15.5

An Iterative Multistage IVILD Algorithm

155

At the second stage of decoding, the received sequence ir< 2l = (rrfl, n·;2J, ... , ir~l) for is obtained the effect of the decoded codeword fo/ll at the first from irC 1l = ir, as given (15.15). The inner decoder forms N metric one for each section of rrC 2l. These tables are then passed to the outer decoder. The outer decoder finds the codeword lbJC 2l = (b?), bfl, · · · , b
M'(JbPl)

=

L M
(15.23)

j=l

For 1 < j :::: JV, let h(b\fl) = 1illj l, which is a codeword in A2. Then, 1ill(2l ~ (1<1f1, 2

2 <21) 1s · a co d ewo1··d.m C2 = B2 o A 2, an d 1il 2C l , · · · , mN N

M(!bP))

= M(1ill(2 )) =

L M(1ill)

2

)),

(15.24)

j=l

where M (1ill ]c.2l) is the metric of 1ill ]c2l with respect to ir<J2l. Now, we compare metric M(Jb/ll) and metric llff(lbJC 2 l). Recall that M(f1(\bJ?))

+ mj)

=

rEaA~ MCf1(\bJ;1)) I

Then, for any filJ

E

M(fi(lbiyl) 2 l E A2,

Because z1~

+ 1illj),

-

+ 211)::: M(f1(1b>j1l) + 1illj).

it follows from (15.25) that 1 M U1 (ftJJ) ))

+ m1) :::

1 M U1 (ibJj l)

+ m;2l).

(15.26)

Let corr(·,·) denote the correlation function (any other metric can be used as well). H is easy to show that (15.27)

for any 1illj

E

A2. Then, it follows from (15.21), (15.24), (15.26), and (15.27) that

(15.28)

where holds if and only if the sequence 21* = (@i', m2, · · · , 21t) is a codeword in C2. For the iterative two-stage MLD algorithm, decoding iterations are based on the generation of a sequence of estimates at the first stage. The estimates are generated in decreasing order of metrics, one at a time, as by using the list Viterbi algorithm [20]. At the ith iteration, the first-stage decoder generates the ith best estimate, denoted by ibJ(l).i = (b?),i, hil).i, · · · , b~).i ). Let ibJC 2l.i denote the decoded codeword at the second stage based on !bJ(l).i and Ir(2). The algorithm is based on the following two theorems. Theorem 15.1 is a direct consequence of (15.28).

756

Chapter 15 THEOREM 15Jl.

Concatenated Coding, Code Decomposition

For i > 0, M(b< 1l.i) :::: M(b<2l,i), where equality holds if and

only if the coset label sequence for !iJPl.i, L (!!](ll.i) ~ Cf1 (b?l.i) + ~i, Ji (bill.i) + a 2,, .. ,fi(b~l.i) +a)v), is a codeword in C; that is, (a{,a2,··· a'F,,) is a codeword in C2 = B2 o A2. Let io be the integer such that 1 ::: io < i and M(IJ/2),io) = max M(b<2l,i),

(15.29)

l:5cj
Then, Jo( 2l.io is the best decoded codeword at the second decoding stage during the first i - 1 iterations. Theorem 15.2 follows from Theorem 15.1 and the fact that for i < j, M(b(ll,i):::: M(lb(ll,i), 15.2 For i > io, if M(b<2l,io) :::: M(U-i 1l,i), then the codeword in C that corresponds to IJ(l).io and b< 2l.io is the most likely codeword with respect to the received sequencer. If M(bC2l,io) < M(lli1l.i) and the coset label sequence L(Jl:»(l),i) is a codeword in C, then L(Rii
THEOREM

In fact, the optimality conditions of Theorem 15 .2 are also necessary conditions for the iterative decoding algorithm.

Stell]! 1. Compute the first (best) estimate lli1l,l of the first decoding stage and its metric M(till.l ). Check whether the coset label sequence L(l!J/ll,l) is a codeword in C. If it is, L (b(l), 1 ) is the most likely codeword and

the decoding stops; otherwise, go to step 2. Step 2. Perform second-stage decoding and obtain the estimate L(b( 2),l) with metric M(&iPl, 1 ). Set io = 1, and store b(ll.1 and !IJPU. Go to step 3. Step 3. For i > io, ll)Cll,io and b< 2l,io are currently stored in a buffer register together with the metric M(b(2l.i0 ). Determine the ith best estimate l!J/ll,i of the outer code B1 , and its metric M(lli 1l,i). If M(l!:ll<2l.io) :::: M(llJPl.i), then i:iPl.io and tPl,io together give the most likely code in

C, and decoding is finished; otherwise, go to step 4. Step 4. Check if the coset label sequence L(l!J/ll,i) is a codeword in C. If it is, L(ii 1l,i) is the most likely codeword in C and decoding is finished; otherwise, go to step 5. §tep 5. Generate !bJC 2l.i. Update io, liJPl.io, Jbi( 2 l,io, and M (lt/ 2 l,io). Go to step 3.

The decoding process iterates until the most likely codeword is found. Therefore, the maximum number of iterations is qf 1. This is the extreme case. In generai, the number of iterations required to obtain the most likely codeword is very small compared with qf 1 • We may limit the number of iterations to keep decoding computational complexity down and decoding delay small. In this case, the decoding algorithm achieves near-optimal error performance. Theorems 15.land 15.2, and the two-stage iterative MLD decoding can be generalized tom stages. In m-stage decoding, new decoding iteration cm he initi::1tPd

Section ·155

An Iterative

MLD

Decoding iteration begins with the of 1 a nevv estimate at the stage, say stage l. H aH the estimates at stage l (resulting from a sequence of codewords from above stage l) have been and the decoder moves up to the (l - 1)th stage and starts a new iteration with a new estimate. Decoding iterations continue until the decision is NIL codevvord is found. Just as in made at the first stage. Suppose the process is at the ith of the iteration. Let ibJUl.J denote the decoded codeword in the outer code B;. Let L(fuPl-i) denote the coset label sequence corresponding to ibJCil.i. The metric of L(lb/il.i) or fOJCi).i is denoted M(fOJUl.J). Let ibJUol.io denote the codeword vvhose metric M(ibJUol.io) is the best among the codewords that have been before the ith uv;..,v,.nu."' stage of the jth iteration and whose coset label sequence L(ibJUol.io) is a codeword in the overall m-level code C. Then, the buffer contains M(ibJCiol.io), L(ibJCiol.i0 ), the estimates of the 1, 2, .... (io - 1) from which the estimate ibJUol ..fo resulted, and the estimates of the L 2. · .. , (i - 1) from 1Nhich the estimate ibJUl.J resulted. At the completion of the ith decoding stage of the the decoder makes one of the three moves:

q/(

:TI., H M(!bJUl.i)

:s

M(ibJUol.io), the decoder moves up to stage (i - 1) and starts a

new iteration with a new estimate. if M (!bJUl.J) > M (ibJCiol-io). the cosetlabel sequence L(!bJU >-i) is tested. Hit is a codeword in C, then the buffer is updated (M(ibJUol.io) = M(fOJUl.i), The decoder moves up to (i - 1) and starts a new iteration with a new estimate. 3, H M(!bJUl.i) > M(ibJUol.io). and the coset label sequence L(!bJUl.i) is not a codeword in C, then the decoder moves down to the (i +1)th stage of the jth iteration.

'.t

When the decoder reaches the last stage, it must move up to the - l)th and start a new iteration the label sequence of the estimate !bJ(m).J 1s a codeword in C). Whenever the decoder reaches the first stage at the beginning of an decision is made at the completion of the first-stage decoding whether the uv~~·~u,.rc. is to be terminated or to continue. the decoder has reached and completed the first stage at the jth iteration. The decoder makes one of the following moves: :Il., It M(lbi(l).i) :s M(ibJUol.io), then the decoding is finished. The ML codeword is formed from !bJCiol.io and the codewords above ioth stage that resulted in the generation of ibJUo>-io.

if M(lbi(l)·i) > M(flDCiol-io), and the coset label sequence L(fOJCll.i) is a codeword in C, then L(ltD(l).J) is the ML codeword. Decoding 21, H M(ibJ(l).i) > M(ibJUol.io), and L(fOJ(ll,i) is not a codeword in C, then the decoder moves down to the second stage and continues the for the jth iteration.

2;,

758

Chapter 15

Concatenated Coding, Code Decomposition

Thus, the tests performed at the first decoding stage are actually the optimality conditions. In m-stage decoding, a new decoding iteration can be initiated at any stage above the final stage. Decoding iteration begins with the generation of a new estimate at the starting stage, say stage l. From there, the decoder can either move down to the next stage, or if the coset label sequence is a codeword in C1, move up and start a new iteration at stage (l -1). If all the q1K1 estimates at stage l (resulting from a particular sequence of codewords from stages above stage l) have already been generated and tested, the decoder moves up to the (l - l)th stage and starts a new iteration with a new estimate. Decoding iterations continue until the ML codeword is found. Just as in two-stage decoding, the final decoding decision is made at the first stage. In decoding with the IMS-MLD algorithm, the number of iterations and the computational complexity depend on the SNR; they decrease as the SNR increases. At a certain point the computational complexity of the IMS-MLD algorithm may become even smaller than that of the conventional multistage decoding presented in Section 15.3. To reduce the worst-case computational complexity, we may set a limit on the number of iterations to be performed. Of course, this limitation results in a suboptimum version of the IMS-MLD algorithm. If the limit on the maximum number of iterations is chosen properly, performance degradation will be small. Consider the third-order RM code RM(3, 7), which is a (128, 64) code with a minimum distance of 16. Suppose this code is decoded with the IMSMLD algorithm based on the 3-level decomposition given in Section 15.4. Its bit-error performance is shown in Figure 15.10. We see the performance curve of

10-1 - - Uncoded BPSK -IMS-MLD ~~~~ - Union bound for MLD """".,,,~~ "*'" Suboptimum IMS decoding ~~,, .g. Conventional MSD

10-2

;E'

10-3

13

"

.0 0 h P.. h

10-.j

0

h h

!

r:o

10-0

10-(;

2

3

4

5

6

7

8

9

10

E1,IN0 (dB)

FIGURE 15.10: Bit-error performance of the (128, 64) RM code with various multistage decodings.

MLD

Section 15.5

the IMS-MLD agrees -vVith the union bound for MLD. Also included in the are the bit of the code with sion of the HvIS-MLD and the conventional Section 15.3. We see that the IMS-MLD 1.35 dB at the BER = the maximum number of iterations is set IMS-MLD at the over conventional

conventional

~·~~~~,u,""u are shown in Figure 15.lL The lfM[S-MLD conventional 0.75 dB; however the uv-..v•,,u,,"' with a limit of 7 iterations (5 for the first stage and 2 for the second stage) achieves almost optimum performance. The of the three Uv'-'VUHUeCC) are in Table 15.2.

of various

TABLE 15:1:

of the

(128, 64) RM code. A wennge mnmlb,er ohe>iill OiPJerntfon,§ mi §NKl liJleeo
2.1[])

2R c10:::J

3JD

3.5

ill.I[])

ill.§

§.ID

U(Qijjle,·lbloun!lllIlelflliy

0111

Conventional MSD

[10 4 ]

2.57

1.88

1.42

1.03

0.80

0.65

0.55

4.26

Suboptimum IMS

[10-l]

6.25

3.33

1.82

1.15

0.83

-

-

36.6

4.0

2.60

-

-

-

9.29

9.28

9.23

9.3

Optimum IMS-MLD [10~] Viterbi

12.6

45.0

[109]

10.0 9.3

TABLE 15.2: Computational complexities of various multistage decodings of the

(128, 99) RM code. A,ernge mmmn!tDel' o[ li'e21Il OiPJerntfonn§ :ait §NJPs. ][l)er
2.5

3.IJI

3.5

4,1[])

4.5

5.ID

5.5

1Up~iJl>U lb,ounmll mn rnm]!lllexnity

Conventional MSD

[104]

1.2

1.1

0.9

0.7

0.56

0.4

0.3

1.5

Suboptimum IMS

[104]

3.3

2.1

1.3

0.8

0.56

0.4

-

7.8

Optimum IMS-MLD [10~]

22.0

7.9

6.3

1.0

0.56

0.4

-

-

4.61

4.56

4A4

4.20

3.82

3.29

4.63

Viterbi

[109]

4.62

160

Chapter 15

Concatenated Coding, Code Decomposition

- ~ Uncoded BPSK ...,.JMS-MLD - Union bound for MLD ..,,.. Suboptimum IMS decoding - Conventional MSD

10-7 ~ - ~ - ~ - - ~ - ~ - - L - ~ - ~ - _ J

1

2

3

4

5

6

7

8

9

10

E 1,IN11 (dB)

FIGURE 15. 11: Bit-error performance of the (128, 99) RM code with various multistage decodings.

15.6

CON(AllENATIED CODING SCHIEMIES WITH (ONVOU.mONAL INNER CODIESi

So far only concatenated coding schemes with block inner codes have been considered; however, convolutional codes can be used as inner codes for constructing concatenated codes as welL Consider the interleaved concatenated coding system shown in Figure 15.L Suppose we replace the block inner code with an (111, ki, v) convolutional code. We obtain a concatenated convolutional coding system. At the first stage of encoding, A consecutive outer codewords are formed and stored in the interleaver buffer as a A x n2 code array. At the second stage of encoding. the code array is read out column by column in binary form (Am bits per column), and the binary sequence, regarded as an information sequence, is then encoded by the inner convolutional code encoder continuously. For inner code encoding, we set Am = l k1. In this case each column of the code array in the interleaver buffer is encoded into l n 1-bit code blocks. The encoding results in a terminated convolutional code sequence. At the receiving end, the received sequence is first decoded by a 2 '' -state truncated Viterbi decoder. The decoded information bits are then grouped into m-bit bytes as symbols in GF(2111 ) and stored by column as a A x n2 array in a deinterleaving buffer. The array in the deinterleaving buffer is read out row by row, and each row is decoded by the outer decoder. In applications, a rate-1 / n 1 convolutional code (or its punctured version) is often used as the inner code in a concatenated convolutional coding system. One such concatenated convolutional coding system is the error-control coding scheme used in the NASA Tracking Data Relay Satellite System (TDRSS). In this system, the (255,223, 33) RS code over GF(2 8 ) is used as the outer code. and the 64-state rate-1/2 convolutional code generated by the polynomials g1 (D) = 1 + D + D 3 + D 4 + D 6

Binary Concatenation

Section 15.7

1(51

10-1 •

~~ ~

_, 10

10- 1

\~------

~~\ \1\\

-

/UncodedBPSK

\

~,

\'\ \

64-state conv. code

\ / ' \ 5 . 223, 33) RS code

ii\;, 1 '\

\

''

II

'

\

j

'\·

\\

RS + conv. wit. h m . t·mite . interleaving

1 I

\

\

\

\

\ /\

\

I

!(\

10-7 .

·#

\

\

\

.

.

l 1

\

.

I

\ ~

10-111

1~-'----lC.L~-'-'-·--"'----'-~i~w..,___.......L~~--''----'-~-'---"---'

0

1

2

3

4

5

6

7

8

9

10

11

12

13

Et/N0 (dB)

::1GUR.E 15. 12: Error

NASA

of the error-control coding scheme used in the SateHite System.

,~n•·1,'),~~';l,,Au,,~

and 1§2(D) = 1 + D 3 + + + D 6 is used as the inner code. The convolutional inner code has free distance dtree = 10. The overall rate of the system is 0.437. The bit-error performance of this concatenated convolutional coding system is shown in Figure 15.12. This system (with infinite interleaving) achieves a BER of 10- 6 at 2.53 dB of SNR. this concatenated convolutional coding system with the concatenated block coding system presented in Section 15.1, we see that the concatenated convolutional coding system has a 0.57 dB coding gain over the concatenated block coding system presented in Section 15.1; however, it has a lower rate, 0.437 with 0.545. H finite interleaving is used in the concatenated convolutional system, then the coding gain over the concatenated block coding in Section 15.1 will be reduced.

concatenation schemes in which the outer codes So far we have considered and the inner codes are binary; however, concatenation can also are

762

Chapter 15

Concatenated Coding, Code Decomposition

be achieved with binary outer and inner codes. Let (n1, k1) and (n2, k2) be two binary codes, denoted C1 and C2, respectively. Suppose C1 is used as the outer code, and C2 is used as the inner code. In encoding, an information sequence k1k2 bits long is divided into k2 subsequences, each consisting of k1 information bits. These h subsequences are encoded into k2 codewords in the outer code C1 and are stored as a k2 x 11 1 array in an interleaver buffer. Then, each column of the array is encoded into a codeword in the inner code C2, which is transmitted as it is formed. In decoding, each received sequence of 112 symbols is decoded based on the inner code C2. The decoded k2 information bits are stored as a column in a deinterleaver buffer. After 111 inner code decodings, a k2 x n1 array is formed in the deinterleaver buffer. Then, each row of the array is decoded based on the outer code C 1 . At the completion of each outer decoding, the k1 decoded information bits are delivered to the user. Again, to achieve good error performance while maintaining low decoding complexity, we use a short inner code and decode it with a soft-decision decoding algorithm, and we use a long powerful binary code as the outer code and decode it with a hard-decision algebraic decoding algorithm. The binary concatenation scheme described here is actually the product coding scheme presented in Section 4.7. If the inner code has a simple trellis, it can be decoded with a soft-input and softoutput decoding algorithm, such as the MAP algorithm ( or its simplified version), discussed in Chapters 12 and 14. The reliability information of each decoded symbol provided by the inner code decoder can be used as soft-input information to the outer code decoder for soft-decision decoding of the outer code. Using soft-decision decoding for both inner and outer code decoders significantly improves the error performance of the concatenated coding system, but the decoding complexity also increases significantly. To reduce the decoding complexity, a simple reliabilitybased algebraic soft-decision decoding algorithm presented in Chapter 10, such as the Chase-2 algorithm or the ordered statistic decoding algorithm, can be used for decoding the outer code. If decoding complexity and decoding delay are not critical issues, the error performance of a binary concatenated coding scheme can be further improved by using soft-input and soft-output decoding for both inner and outer code decoders and performing the inner and outer decodings iteratively. During each decoding iteration, the inner code decoder provides the outer code decoder with reliability information of the decoded symbols; this reliability information is then used as the soft input for the outer code decoder. Based on this soft-input information (together with the channel information), the outer code decoder performs softoutput decoding. The reliability information of the decoded symbols is then fed back to the input of the inner code decoder as a soft input to start the next decoding iteration. Decoding iterations continue until a certain stopping criterion is satisfied. To prevent successive decoding iterations from becoming correlated, the interleaver at the encoding side pseudorandomly permutes the bits in the code array after the outer code encoding. The permuted array is then encoded by the inner code encoder. At the decoding side, after inner code decoding, the decoded symbols must be permuted inversely (inverse permutation) to allow the outer code decoder to perform outer code decoding. To start the next decoding iteration, the decoded symhols at the end of the outer code decoding are permuted back to the

Problems

form for the inner code Mv·v~·~"" is in each ,,_,vv~o~u,,I',

763

and :inverse very and

The tvm-dimensional codes without the checks on ::hecks suitable for the described of iterative ~,~~-.~...,.~ the information bits of the information array are before the column This allows the tvvo sets of bits to two sets of uncorrelated estimates for the same set of information bits with iterative Row and column

in which the information sequence is encoded interleaver. This v,,vv·- .. · bits for the same information sequence. ~vvv,.,u,,,-, side, iterative is tvvo decoders based on these two sets of bits. Parallel concatenation is usuaHy using two convolutional encoders. schemes in form Binary concatenated and iterative ,.,,,_,,uu,u,~. commonly called turbo

iPrPI.OlBtlEMS 15J_ Prove that the concatenation of an (11 1 • k1) inner code with minimun1 distance d1 and an (112, k2) outer code with minimum distance d2 has a minimum distance of at least d1 dz. 15,2 Prove the lower bound of the minimum distance of an m-level concatenated code given by 15.12. J153 Consider the concatenation of a RS outer code over ) and the binary (m + 1. 111. 2) single parity-check inner code. Devise an error-erasure for this concatenated code. [Hint: During the inner code decoding, if is detected in 111 + 1 received bits, an erasure is declared. H no detected, the parity bit is removed to form a symbol in GF(2 111 )]. 15,Lll Form a 5-level concatenated code with a minimum distance of 16 RM codes of length 16 to form inner codes. Choose either binary or RS codes (or shortened RS codes) of length 16 as outer codes to maximize the overall code rate. ]_§,5 the 5) code into a 3-level concatenated and describe the trellis complexities of the component concatenated codes at the three levels. 15Aii the 6) code into a 3-level concatenated and the trellis complexities of the component concatenated codes at the three levels. ]_§/7 Decode the RM(2, 5) code with 3-stage decoding based on the decomposition obtained in Problem 17.5. Plot the bit- and block-error performances versus SNR. 15,~ Decode the RM(2, 6) code with 3-stage decoding based on the decomposition obtained in Problem 17.6. Plot the bit- and block-error performances versus SNR. 15,«Jl Repeat Problem 17.7 with the IMS-MLD algorithm. ]_§,]_q]) Repeat Problem 17.8 with the IMS-MLD algorithm.

164

Chapter 15

Concatenated Coding, Code Decomposition

IB!IBUOGRAPHY

]., G.D. Forney, Jr., Concatenated Codes, MIT Press, Cambridge, 1966. 2, J. Justesen, "A Class of Constructive Asymptotically Good Algebraic Codes," IEEE Trans. Inform. Theory, IT-18; 652-56, September 1972.

3, H. T. Moorthy, S. Lin, and G. Uehara, "Good Trellises for IC Implementation of Viterbi Decoders for Linear Block Codes," IEEE Trans. Commun., 45: 52-63, January 1997. 4, E. Nakamura, G. Uehara, C. Chu, and S. Lin, "A 755 Mb/s Viterbi Decoder for RM(64, 35, 8) Subcode," Proc. IEEE Intl. Solid-State Circuit Conf, San Francisco, Calif., February 15-17, 1999. 5, E. L. Blokh and V. V. Zyablov, Generalized Concatenated Codes, Moscow, USSR. Nauka. 1976.

6, V. A. Zinoviev, "Generalized Cascade Codes," Probl. Peredachi Inform., 12: 5-15, 1976. 7, S. Hirasawa, M. Kasahara, Y. Sugiyama, and T. Namekawa, "Certain Generalization of Concatenated Codes-Exponential Error Bounds and Decoding Complexity," IEEE Trans. Inform. Theory, IT-26: 527-34, September 1980. 8, T. Kasami, T. Fujiwara, and S. Lin. "A Concatenated Coding Scheme for Error Control," IEEE Trans. Commun., 34: 481-88, May 1986.

9, R. Morelos-Zaragoza, T. Fujiwara, T. Kasami, and S. Lin, "Construction of Generalized Concatenated Codes and Their Trellis-Based Decoding Complexity," IEEE Trans. Inform. Theory, 45: 725-31, March 1999. 10, A. R. Calderbank, "Multilevel Codes and Multistage Decoding," IEEE Trans.

Commun., 37: 222-29, March 1989. lJL G.D. Forney, Jr., "Coset Codes-Part II: Binary Lattices and Related Codes," IEEE Trans. Inform. Theory, 34: 1152-87, September 1988.

12, F. Hemmati, "Closest Coset Decoding of /l!ll!rui + vi Codes," IEEE I. Select. Areas Commun., 7: 982-88, August 1989. B, U. Dettmar, J. Portugheis and H. Hentsch, "New Multistage Decoding Algorithm," Electronics Letter, 28 (no. 7): 635-36, 1992. 14. J. Wu, S. Lin, T. Kasami, T. Fujiwara, and T. Takata, "An Upper Bound on the Effective Error Coefficient of Two-Stage Decoding, and Good Two-Level Decomposition of Some Reed-Muller Codes," IEEE Trans. Commun., 42: 813-18. February/March/April, 1994. 15, T. Takata, Y. Yamashita, T. Fujiwara, T. Kasami, and S. Lin,"Suboptimum Decoding of Decomposable Codes," IEEE Trans. Inform. Theory, 40: 1392-1405. September 1944.

"Soft Decision Decoding of Reed-Muller Codes as Concatenated " IEEE Trans. 41: 1l7, D.

M. P. C. Fossorier, and S. of Multilevel 11-14.1999. S. and M. P. C. Fossorier, "Iterative "'Proc 2000 IEEE Intl. 2000. vv~·~rn,,..,

Algo-

" Ph.D. dissertation, Dept. of June 2003. "List .,, IEEE Trans. Commun .. 42:

Zit N. Seshadri and C. W.

with

1994.

C H A PT E R

16

Turbo Coding As noted in Chapter 1, Shannon's noisy channel coding theorem implies that arbitrarily low decoding error probabilities can be achieved at any transmission rate R less than the channel capacity C by using sufficiently long block ( or constraint) lengths. In particular, Shannon showed that randomly chosen codes, along with maximum likelihood decoding (MLD), can provide capacity-achieving performance. He did this by proving that the average performance of a randomly chosen ensemble of codes results in an exponentially decreasing decoding error probability with increasing block (or constraint) length; however, he gave no guidance about how to actually construct good codes, that is, codes that perform at least as well as the average, or to implement MLD for such codes. fo the ensuing years after the publication of Shannon's paper in 1948 [1 ], a large amount of research was conducted into the construction of specific codes with good error-correcting capabilities and the development of efficient decoding algorithms for these codes. Much of this research has been described in the previous chapters of this book. Typically, the best code designs contain a large amount of structure, either algebraic, as is the case with most block codes, such as RM and BCH codes, or topological, as is the case with most convolutional codes, which can be represented using trellis or tree diagrams. The structure is a key component of the code design, since it can be used to guarantee good minimum distance properties for the code, as in the BCH bound, and since particular decoding methods, such as the Berlekamp-Massey algorithm for BCH codes and the Viterbi algorithm for convolutional codes, are based on this structure. fo fact, one can say generally that the more structure a code contains, the easier it is to decode; however, structure does not always result in the best distance properties for a code, and most highly structured code designs usually fall far short of achieving the performance promised by Shannon. Primarily because of the need to provide structure to develop easily implementable decoding algorithms, little attention was paid to the design of codes with randomlike properties, as originally envisioned by Shannon. Random code designs, because they lacked structure, were thought to be too difficult to decode. In this chapter we discuss a relatively new coding technique, dubbed turbo coding by its inventors [2], that succeeds in achieving a ranclomlike code design with just enough structure to allow for an efficient iterative decoding method. Because of this feature, these codes have exceptionally good performance, particularly at moderate BERs and for large block lengths. In fact, for essentially any code rate and information block lengths greater than about 104 bits, turbo codes with iterative decoding can achieve BERs as low as 10-5 at SNRs within 1 dB of the Shannon limit, that is, the value of E1,/ No for which the code rate equals channel capacity. This typically exceeds the performance of previously known codes of comparable length and decoding complexity by several decibels. although because decoding is iterative, l::irgPr rlPrnrling rlPhy~ rP<mlt.

766

Section 16. 1

Introduction to Turbo

7/(£ 7J

feature of turbo codes is that they are composed of two in a variation of the concatenation with a interleaver. Because the intedeaver is a complete maximum likelihood decoder for the entire code would be prohibitively because more than one code is to soft-in soft-out (SISO) decoders for each constituent code in an iterative in which the soft-output values of one decoder are to the other, and vice versa, until the final estimate is obtained. Extensive with turbo decoders has shown that this iterative ~,~~~-~·­ almost converges to the maximum likelihood solution. In summary, turbo consists of two fundamental ideas: a code design n,·,,.rn,,re,~ a code with randomlike and a decoder design that makes values and iterative decoding. The basic features of turbo code in Sections 16.1 to 16.4, and the principles of iterative turbo in Section 16.5. of turbo coding was introduced in a paper uu,u 1,,,u.uw [2] delivered at the IEEE International Conference on Communications held in in May 1993, and it was further elaborated in 4]. Many of the earlier ideas of BatiH and Hagenauer on randomlike codes and iterative were influential in leading to the turbo coding concept [5-8]. Much of the research was originally skeptical of the performance but the end of 1994 the basic results had been confirmed by several other research groups (9, 10]. Two papers Benedetto and Montorsi [11, 12] provided the first for the exceptional performance of the codes, and further '-'0'-'H•~,,u in [13, 14]. The research group of Hagenauer was primarily '"'"'·Pv110""""' for the benefits of iterative decoding [15, 16], and additional progress reports, beginning in .~.,,-,,.,., vvas achieved in [17, 18]. The of Divsalar, Dolinar, and colleagues at the Jet Propulsion Laboratory a detailed look at many aspects of turbo coding [19-22] as communication issues. Numerous variations of turbo coding have in the such as serial concatenation, and serial and self-concatenation [23-25]. A comprehensive overview of the first five years of turbo coding is given in the book by Heegard and Wicker [26], which also contains a thorough discussion of iterative decoding from the point of view of the of belief propagation [27]. Another book covering roughly the same material was written Vucetic [28], and a readable summary of the main of turbo coding, with an emphasis on iterative decoding, was published by [29]. CWC,vvC0

115. 1

i~ll\~OIDJQ.D(l!O~l lO 1U~l8l0 (COJIDl~JG A block diagram of the basic turbo encoding structure is illustrated in Figure 16.l(a), and a example is shown in Figure 16.l(b). The basic system consists of an information (input) sequence, two (2, 1, v) systematic feedback (recursive) convolutional and an interleaver, denoted by n. We will assume that the information sequence contains K* information bits plus v termination bits to return the first encoder to the all-zero state So = @, where v is the constraint length of the

168

Chapter 16

Turbo Coding

Ill= (u11,ll1, ..... ,UJ(-\)

Encoder 1

o~~-

J~

E"cod'.'i (a)

_lll--,--------------------------P--v(OJ

(b)

FIGURE 16.1: The basic turbo encoding structure.

first encoder. The information sequence (including termination bits) is considered to be a block of length K = K* + v and is represented by the vector u

= (uo, u1, · · · , uK-1).

(16.1)

Because encoding is systematic, the information sequence sequence; that is lll

= v
'

v(O) ) K-1

·

lill

is the first transmitted (16.2a)

The first encoder generates the parity sequence vCl) = (/ll

0 '

v(l)

l

'

...

'

v(l) ) K -1

·

(16.2b)

The interleaver reorders or permutes the K bits in the information block so that the second encoder receives a permuted information sequence llll' different from the first. (Note that the second encoder may or may not be terminated.) The parity

Section 16.1

Introduction to Turbo

by the second encoder is

sequence

, (2) _

o/

-

( (2) (2) . . . (2) ) VO , V l ' ,VK-1'

and the final transmitted sequence (codeword) is "v _ (v(O) v(l) v(2)

·- · o o o ·

(OJ

vl

(1)

Vi

(16.2c)

by the vector

(2) vl

(16.3)

code has N = 3K and rate for large K. The information sequence il!l sequence ,/ll is referred constituent and the information sequence Di to as the is not with the second sequence o/( 2l is referred to as the second constituent code. In Figure 16.l(b) both constituent codes are the same (2, 1. 4) feedback encoder whose matrix is so that the overall - v)/3K;:.::;

of turbo codes.

The in later sections of this chapter. o

To achieve close to the Shannon limit, the information block length (interleaver size) K is chosen to be very large, at least several thousand bits.

o

The best performance at moderate BERs down to about 10- 5 is achieved with short constraint length constituent v = 4 or less.

o

The constituent codes are by the same encoder, as in but this is not necessary for good In sorne code have been shown to give very good [30].

o

Recursive constituent codes, generated by much better performance than nonrecursive constituent feedforward encoders. For rate R

=

from the parity sequences to puncturing alternate bits from o/(ll and o/< 2 > 1/2 code.

systematic or

,..,_.,,,,. ""' from the information sequence to produce partially ,vu,~,,~ turbo codes [31].

rate codes. For example, rate R = 1/4 can be achieved with three constituent codes and two interleavers, as shown in Figure 16.2. This configuration is called 2, multiple turbo code [19]. o

The best interleavers reorder the bits in a pseudorandom manner. Conventional block (row-column) interleavers do not perform well in turbo codes, except at relatively short block lengths.

110

Chapter 16

Turbo Coding

-1!11--.------------------------i>

Encoder1

Encode

------.P.-1 _

Encoder3

V(O)

,,___ _ _ _ _ V(I)

-----l>-V(2J

J-----~

v(~J

FIGURE 16.2: A rate R = 1/4 multiple turbo code.

o Because only the ordering of the bits is changed by the interleaver, the sequence u' that enters the second encoder has the same weight as the sequence u that enters the first encoder. 0

®

®

The interleaver is an integral part of the overall encoder, and thus the state complexity of turbo codes is extremely large, making trellis-based ML or MAP decoding impossible. Suboptimum iterative decoding, which employs individual SISO decoders for each of the constituent codes in an iterative manner, typically achieves performance within a few tenths of a decibel of overall ML or MAP decoding. The best performance is obtained when the BCJR, or MAP, algorithm is used as the SISO decoder for each constituent code. 1 Because the MAP decoder uses a forward-backward algorithm, the information is arranged in blocks. Thus, the first constituent encoder is terminated by appending v bits to return it to the O state. Because the interleaver reorders the input sequence, the second encoder will not normally return to the Ostate, but this has little effect on performance for large block lengths. If desired, though, modifications can be made to ensure termination of both encoders.

"' Block codes also can be used as constituent codes in turbo encoders. ®

Decoding can be stopped, and a final decoding estimate declared, after some fixed number of iterations (usually on the order of 10-20) or based on a stopping criterion that is designed to detect when the estimate is reliable with very high probability.

1It is important to note th8t iterative SISO decoding, using individual lVIP..,..P decoders for eu.ch constituent code, is not the same as overall ML or MAP decoding

Section 16.1

Introduction to Turbo

scheme illustrated in Figures 16.1 and 16.2 is referred to as because of its original concatenation conc:ate,n2mcm. in concatenation rather than its enters the second by the interleaver. fo other the two encoders on different versions of the information sequence. The code shown in Figure to rate 1/2. is capable of a BERatanSNRofEb/No = 0.7 dBvvithaninformationblock = 65536 bits after 18 iterations of a S][SO MAP decoder. the NASA standard (2. 1. 6) convolutional code with ML Eb/ No of 4.2 dB to achieve the same BER The tvvo codes is shown in 16.3. the rate R l/2 turbo code achieves a 3.5-dB coding with the (2. 1, 6) convolutional code at a 10- 5 BER The of the two codes is since a 16-state MAP decoder has about the same as a 64-state Viterbi decoder. This of turbo codes over conventional methods of over the entire range of code rates; that several decibels of coding gain can be achieved at moderate BERs with turbo codes of the same rate and roughly the same decoding as conventional codes. In the np·a-t,-,r,m of turbo codes at moderate BERs is within 1 dB of For the example shown in Figure 16.3, the BER is 0.7 dB away from the and 0.5 dB away from the for channels. Turbo codes suffer from two a large decoding delay, owing to the large block required for near-capacity O~Nono.r

le+OO (2. 1. 6) NASA ~-~ Turbo code - ·>< -

le-01

le-02

'i

le-03

n

n

t

~

~

n I

a:'.' le-04

c

·c:;

le-05

0,

~

·c:;

le-06

le-07 -1

FIGURE 16.3:

"'ro

"'

u ~

0,

(/)

Cl

ca ""'

u

0

n n

n n n

u

2

3

4

5

6

Performance comparison of convolutional codes and turbo codes.

i!Tl

Chapter 16

Turbo Coding

performance, and significantly weakened performance at BERs below 10-5 owing to the fact that the codes have a relatively poor minimum distance, which manifests itself at very low BERs. The large delay appears to make turbo codes unsuitable for real-time applications such as voice transmission and packet communication in high-speed networks. It is possible, though, to trade delay for performance in such a way that turbo codes may be useful in some real-time applications involving block lengths on the order of a few thousand, or even a few hundred, bits. The fact that turbo codes typically do not have large minimum distances causes the performance curve to flatten out at BERs below 10- 5 , as shown in Figure 16.3. This phenomenon, sometimes called an error floor, is due to the unusual weight distribution of turbo codes, which will be discussed in the next section. Because of the error floor, turbo codes may not be suitable for applications requiring extremely low BERs, such as some scientific or command-and-control applications; however, measures can be taken to mitigate this problem. Interleavers can be designed to improve the minimum distance of the code, thus lowering the error floor. Also, an outer code, or a second layer of concatenation, can be used with a turbo code to correct many of the errors caused by the small minimum distance, at a cost of a small decrease in overall rate. Both techniques will be discussed later in this chapter. The fundamental property of turbo codes that underlies their excellent performance at moderate BERs is the randomlike weight spectrum of the codewords produced by the pseudorandom interleaver when systematic feedback encoders are used. To understand this feature we consider a series of examples. IEXAMPllE 11fU

Consider the conventional (2, 1, 4) convolutional code with nonsystematic feedforward generator matrix Gtf (D) = [ 1 + D + D 2 + D 3 + D 4

1 + D4

(16.5)

] .

The minimum free distance of this code is 6, obtained from the information sequence l!R = (11 0 · · · ). H the encoder is converted to systematic feedback form, the generator matrix is then given by G.11,(D)=[ 1

(l+D 4 )/(l+D+D 2 +D 3 +D 4 )

].

(16.6)

Because the code is exactly the same, the free distance is still 6, but in this case the minimum-weight codeword is obtained from the information sequence 5 llR = [100 0 0 100 · · · ]; that is, l!R(D) = 1 + D . The two different encoders result in identical codes, but with different mappings between information sequences and codewords. Now, consider that each encoder is terminated after an information block of length K* = K - 4 by appending 4 bits to return the encoders to the 0 state. (Note that for the feedback encoder, the K* termination bits depend on the information block and are in general nonzero.) In this case we obtain an (N. K*) = (2K, K - 4) block code with rate R 1 = (K - 4) /2K ~ 1/2 for large K. This block code contains exactly K -5 weight-6 codewords, because the information sequence that produces the weight-6 codeword can begin at any of the first K - 5 information positions and generate the san1e codevvord. /11).. similar analysis reveals

Section 16.1

Introduction to Turbo Coding

7/7/3

TABLE 16.1: VJ eight spectra of two (32, 12) codes.

~ID) 1ferrmmiinnID1teQ11 icl]])nnwl]])Ilununl]])nil!llil

QllD} IP'IDmilileil ICl]])IlTIIC:lllitennID1teall

\lfl/efi1i;Ilntl

Miunil1tli]Plilkii1ty

vl'ell!l;M

Miunil1til]Plilkn1ty

1 0 0 0 0 0 11 12 23 38 61 126 200 332 425 502 545 520 491 346 212 132 68 38 11 2 0 0 0 0 0 0 0

0 1 2 3 4 5 6 7 8 9

1 0 0 0 0 1 4 8 16 30 73 144 210 308 404 496 571 558 478 352 222 123 64 24 4 4 1 0 0 0 0

0 1 2 3 4 5 6 7 0

0

9 10

11 12 13 14 15 16 17 18 19 20 21 22

23 24 25 26 27 28 29 30 31 32

10

11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

0 0

that for weight-7 and other low weights, the number of codewords is also large, on the order of Kor larger. In Table 16.l(a), we give the complete weight spectrum of the (32, 12) code that results from choosing K. = 16. Observe that the number of codewords at each grows rapidly until it reaches a peak at length 16, half the block length. In other words, the weight spectrum of the code is dense at the low end, and this results in a relatively high probability of error at low SNRs, even with ML decoding.

714

Chapter 16

Turbo Coding

In general, if an unterminated convolutional code has Ad codewords of weight d caused by a set of information sequences {llR(D)} whose first one occurs at time unit l = 0, then it also has Ad codewords of weight d caused by the set of information

sequences {Dl!ll(D)}, and so on. Terminated convolutional codes have essentially the same property. In other words, convolutional encoders are time-invariant, and it is this property that accounts for the relatively large numbers of low-weight codewords in terminated convolutional codes. Next, we look at an example in which a pseudorandom interleaver is used to produce a parallel concatenation of two identical systematic feedback convolutional encoders.

IEXAMIPllE 115.2 Consider the systematic feedback convolutional encoder of (16.6), a length K input sequence, and the size-16 interleaving pattern given by the permutation

f1 = [0.8, 15,9,4, 7, 11,5, 1,3, 14,6, 13, 12,10,2].

= 16

(16.7)

16

The input sequence is first encoded by the parity generator (1 + D 4 ) / (1 + D + D 2 + D 3 + D 4 ), producing the parity sequence w< 1l (D). Then, the interleaver takes the 12 information bits plus the 4 termination bits and reorders them such that (16.8) This permuted input sequence is then reencoded using the same parity generator (1 + D 4 ) / (1 + D + D 2 + D 3 + D 4 ), thus producing another version of the parity sequence. To compare with the code of Example 16.1, we now puncture alternate bits from the two versions of the parity sequence using the period T = 2 puncturing matrix:

P=[~

~l

(16.9)

The result is a parallel concatenated code with the same dimensions as the code in Example 16.1, that is, a (32, 12) code. The weight spectrum for this parallel concatenated code is given in Table 16.l(b). We see that there is a noticeable difference between this weight spectrum and the one for the terminated convolutional code shown in Table 16.l(a), even though the code generators are the same. This altered weight spectrum is a direct result of the interleaver that permutes the bits for reencoding. Note that the free distance has decreased, from 6 to 5, but that there is only one weight-5 codeword. More importantly, the multiplicities of the weight-6 through weight-9 codewords are less for the parallel concatenated code than for the terminated convolutional code. In other words, in the parallel concatenated case, there has been a shift from lower-weight codewords to higher-weight codewords reJ::itive to the convoh1timrnl code_ This shiftinr, of low-wei3ht ~oclewonls tow::ircl higher v1eights in the parallel concatenation of feedback convolutional encoders has

Section 16. 1

Introduction to Turbo Coding

775

[13] and results when causes almost aH sequences in the first constituent code to be matched with sequences in the second constituent code. For consider parity the sequence ill! = (1000010 · · · 0) that causes the sequence v< 1l = (11001100 ... 0). Thus, without the interleaver, the terminated consequence volutional code produces a codeword of weight 6. The interleaved is given by un' = (100000010 · · · 0) and produces the v( 2) = (1100101111000110). v< 1l and v< 2l and then bits the T = 2 matrix sequence (1100100101000100). Thus, the same sequence ,·w,.. codeword of 8 in the concatenated code. This behavior is that fo the next example we see that becomes more dramatic for larger block mr'P.Q

uurnuucu;,,

§(())®lf:ii:rnil

lMrmrunlnl\9)

Consider the same ·-,.,-.. - feedback convolutional encoder as in ~,.us.up>v0 l6.l and 16.2 but with a block length of K = 32, including the v = 4 termination bits. The weight spectrum of the terminated (64. 28) convolutional code is shown in Table 16.2(a). Now, consider the same encoder in a concatenation ~v,,v,.,w, using the size-32 pattern given

n 32

= [O. 16. 7, 17. 12, 28,

2.8. 11,22,9,4.20.

1,3,14,6,13,31, 10,29,25.24.15,30,5.23,27,

(16.10)

and the same period T = 2 puncturing matrix as in 16.2. The resuH is the ( 64, 28) parallel concatenated code whose is given in Table 16.2(b ). We note that there is now a more difference between this weight spectrum and the one for the terminated convolutional code in Table 16.2(a) than in the K = 16 case. The free distance of both codes is 6; that there is no change in dtree, but the multiplicities of the codewords are greatly reduced in the parallel concatenated code. This result can be seen more clearly in the of the two weight spectra shown in Figure 16.4. In other the effect of thinning becomes more dramatic as the block length size) K increases. In fact, for even larger values of K, the codeword and bit of the lovv-weight codewords in the turbo code spectrum are reduced by a factor of K, the interleaver size, compared with the terminated convolutional code. This reduction a factor of K in the low-weight multiplicities is referred to as the interleaver gain [11] and will be verified analytically in the next section. Several remarks can be made regarding Examples 16.1-16.3: o

Different interleavers and matrices would produce different but the behavior observed is typical in most cases.

116

Chapter 16

Turbo Coding TABLE 16.2: Weight spectra of two (64, 28) codes.

(a) Terminated convo!ll!tiom:ul

(b) Parallel com:atenate«I!

Weight

Mll!Iltiplicity

Weight

Mllllitill_]J!idty

Weight

Mnltiplicity

Weight

Mnltiplidty

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

1 0 0 0 0 0 27 28 71 118 253 558 1372 3028 6573 14036 29171 60664 122093 240636 457660 838810 1476615 2484952 3991923 6098296 8845265 12167068 15844169 19504724 22702421 24967160 25927128

33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64

25431436 23509909 20436392 16674749 12757248 9168248 6179244 3888210 2271250 1226350 615942 287487 124728 50466 19092 6888 2172 642 140 35 6 2 0 0 0 0 0 0 0 0 0 0

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

1 0 0 0 0 0 6 9 15 9 80 119 484 1027 3007 6852 17408 40616 90244 193196 390392 754819 1368864 2367949 3874836 5988326 8778945 12149055 15907872 19684668 22978613 25318411 26289667

33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64

25716960 23669905 20464409 16623260 12662360 9024333 6012086 3729485 2156481 1160459 573214 262676 110369 42264 15269 4556 1416 335 103 20 5 1 0 0 0 0 0 0 0 0 0 0

®

®

®

Spectral thinning has little effect on the minimum free distance, but it greatly reduces the multiplicities of the low-weight codewords. As the block length and corresponding interleaver size K increase, the weight spectrum of parallel concatenated convolutional codes begins to approximate a randomlike distribution, that is, the distribution that would result if each bit in every codeword were selected randomly from an independent and identically distributed probability distribution. There is only a small spectral thinning effect if feedforward constituent encoders are used, as will be seen in the next section.

"' One can explain the superiority of feedback encoders in parallel concatenation as a consequence of their being UR, rather than FIR, filters, that is, their response to single input 1's is not localized to the constraint length of the code but extends over the entire block length. This property of feedback encoders is exploited by a pseudorandom interleaver to produce the spectral thinning effect.

Introduction to Turbo Coding

Section 16.1

77/7/

le+08 _ Terminated convolutional ~ Parallel concatenated - ,,, -

le+07 le+06 le+05

0

:g ~ ;5 ;;;

le+04 _ le+03 _ le+02 _ le+Ol _

40 30 Codeword weight

FIGURE 16.4:

An illustration of

50

60

70

thinning.

that parallel concatenated codes are no longer timeH is also worth invariant. This can easily be seen the effect when the input sequence in Example 16.2 is delayed by one time that is. consider the input sequence fill = [0100001000000000]. The first parity sequence o/
(D)

=

D

1

(

1+D

5

),

1=

o. 1. · · · , 10.

(16.11)

118

Chapter 16

Turbo Coding

I •• ; . . . .

I

I

I

-/-.J

~--·:::} .. !...... l.[ ....

I I I

I

I

I • • •j" • 1··· I 1•••• -,

I

I

I

.,•••• •,

:

_/

••• • I

I

I :

:

/

I I

I

I

I

/

I I I

1 I

/

~------,- · · -- -:- · -l- ""· --1-- ·/· I

I

• • ••)

./•" •• •· :

••• -~·· 1

I

1t·· :1I I 11 1 I 11 •.. i!.-.--·:_ ----}_ ---------/_ ---------~ --. - . ----------------l> 1I

••

1

Write

I I I I I

I I I

Read

+

FIGURE 16.5: A (4 x 4) block interleaver for a (32. 12) turbo code.

are bad because they generate a low-weight (4 in this case) parity sequence. As can be seen from a careful examination of Figure 16.5, if a 4 x 4 block (row-column) interleaver is employed, 9 of these 11 sequences will maintain the same bad pattern after interleaving, resulting in a large multiplicity of low-weight codewords. The weight spectrum of the code in Example 16.2 with this block interleaver, shown in Table 16.3, is clearly inferior to the weight spectrum shown in Table 16.l(b) obtained using the interleaver of (16.7). Pseudorandom interleavers, such as those in (16.7) and (16.10), generate a weight spectrum that has many of the same characteristics as the binomial distribution, which is equivalent to the weight spectrum assumed by Shannon in his random-coding proof of the noisy-channel coding theorem. fo other words, codes with random (binomial) weight distributions can achieve the performance guaranteed by Shannon's bound. Turbo coding with pseudorandom interleaving results in a way of constructing codes with weight spectra similar to a binomial distribution, and a simple, near-optimal iterative decoding method exists. Pseudorandom interleaving patterns can be generated in many ways, for example, by using a primitive polynomial to generate a maximum-length shiftregister sequence whose cycle structure determines the permutation. Another method uses a computationally simple algorithm based on the quadratic congruence Cm

=

km(m

2

+ 1)

(mod K), 0:::: m < K.

(16.12)

to generate an index mapping function c111 --+ c111 +1 (mod K), where K is the interleaver size, and k is an odd integer. For example, for K = 16 and k = 1, we obtain (co, c1, · · · , c15) == (0, 1, 3, 6, 10, 15, 5. 12, 4, 13, 7, 2, 14, 11, 9, 8),

Section 16.1

Introduction to Turbo Coding

7J7J'r!J

TABLE 16.3: The weight spectrum

of a block interleaved (32, 12) turbo code. Wen1c1;Ilntl

Ml!l!Iltrn]llJilndtry

0 1 2 3 4 5 6 7 8 9

1 0 0 0 0 0 21 6 13 40 67 154 190 308 411 486 555 532 493 350 230 140 64 28 3 4 0 0 0 0 0 0

10

11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

0

which implies that index O (input bit u 0) in the interleaved sequence 1!111 is mapped into index 1 in the original sequence 1!11 (i.e., u 0 = iq), index 1 in 1!111 is mapped into index 3 in ill! (u~ = u3), and so on, resulting in the permutation

fl = [L 3, 14, 6, 13, 12, 10. 2. 0, 8, 15. 9, 4, 7, 11, 5]. 16

(16.14)

780

Chapter 16

Turbo Coding

= 8 times, we obtain the interleaving pattern of (16.7). For K a power of 2, it can be shown that these quadratic interleavers have statistical properties similar to those of randomly chosen interleavers, and thus they give good performance when used in turbo coding [32]. Other good interleaving patterns can be generated by varying k and r, and the special case r = K /2 (used to obtain (16.7)) results in an interleaver that simply interchanges pairs of indices (see Problem 16.2). This special case is particularly interesting in terms of implementation, since the interleaving and deinterleaving functions (both used in decoding) are identical. Finally, when K is not a power of 2, the foregoing algorithm can be modified to generate similar permutations with good statistical properties. The basic structure of an iterative turbo decoder is shown in Figure 16.6. (We assume here a rate R = 1/3 parallel concatenated code without puncturing.) It employs two SISO decoders using the MAP algorithm presented earlier in Chapter 12. At each time unit l, three output values are received from the channel, one for the information bit u1 = vi 0l, denoted by r 1(0), and two for the parity bits and denoted by and r?l, and the 3K -dimensional received vector is denoted by Ir (,JO\Cl\(2) rCO\(llr(2) ... r(O) r(l) r(2) ) (16.15)

If this interleaving pattern is shifted cyclically to the right r

v?>

v?,

r?)

-

0

0

0

' 1

1

1

'

'

K-1 K-1 K-1

·

Now, let each transmitted bit be represented using the mapping O--,,. -1 and 1--,,. +l. Then, for an AWGN channel with unquantized (soft) outputs, we define the log-likelihood ratio (L-value) L ( 1 = L ( u1 I (before decoding) of

v/0l ,?l)

a transmitted information bit u1

P (u1 =

(O)

L (u1

I r1

r/0l) given the received value r/ 0l as

)

= ln

(

+11,?>)

r/)0 ) 0 P (r/ ) I u1 = +1) P (u1 = +1) =In-----------? (r I = -1) P (u1 = -1) p

U[

= -11

(0) 1

U/

f=IGURE 16.6: Basic structure of an iterative turbo decoder.

Section 16.1

= ln

Introduction to Turbo Coding

P(r/Ollu1=+l) P (r/0l

I u1 = -l) (0)

e -(E,/No) ( r1 -1

+ ln

,

P(ui=+l)

+ ln - - - - -

)"

= ln

e -(E,/No)(,)°l+lr

P(u1=

P (ui = +l) p (u1

=

,

(16.16)

-1)

where Es/ No is the channel SNR, and u1 and r/°l have both been normalized simplifies to factor of p;. This L (u1 I

r/ 01 )

= -

!~ {

2

(ri(Ol - 1)

-

(r/0l +

=

(0)

Lcr1

P (u1

=

a

P(u1 =+1) -} + lnP-(u1- = -1)

Es (Ol P = +1) =4-rI + l n - - - - No

1811

(16.17)

-1)

+ La (u1),

where Le = 4(Es / No) is the channel reliability factor (see Section 12.5), and La (uz) is the a priori L-value of the bit u1. In the case of a transmitted parity bit v?, given the received valuer?), j = 1, 2, the L-value (before decoding) is given by (16.18)

since in a linear code with equally likely information bits, the parity bits are also equally likely to be + 1 or -1, and thus the a priori L-values of the parity bits are O; that is, p = +1) I . (16.19) La v1 = In ( . ) = 0, J = 1, 2. P v;'l=-l

( (j))

(/il

(We note here that La (uz) also equals O for the first iteration of decoder 1 but that thereafter the a priori L-values of the information bits are replaced extrinsic L-values from the other decoder, as will now be explained.) The received soft channel L-values Lcr/0 l for u1 and Lcr?l for v;l) enter decoder 1, and the (properly interleaved) received soft channel L-values Lcr/0l for uz and the received soft channel L-values Lcr?> for v? enter decoder 2. The output of decoder 1 contains two terms: :Il.o L(l)

(uz) = ln

lp (uz = +l / li'l, ILi >) / 1

P

(uz = -11 ir1, ILi1))

l

the a posteri-

ori L-value (after decoding) of each information bit produced by decoder 1 given the (partial) received vector ir 1 ~ [,JO\Cl) ,.<0),.Cll . . . r(O) r(l) 0 0 ' 1 1 ' ' K-l K-1

J

and the a priori input vector ILi1l ~ [ d,1l(uo), decoder 1, and

J,

d 1\iq), · · ·, Li1\uK-1)]

for

20 Li1l (u1) = L Cl) (uz) - [ Lcr?) + Lfl (u1) the extrinsic a posteriori L-value (after decoding) associated with each information bit produced by decoder 1,

782

Chapter 16

Turbo Coding

which, after interleaving, is passed to the input of decoder 2 as the a priori val ue (2) ( u 1),. Subtracting the term in brackets, namely, Lcr/0 l + d 2l (u1), removes the effect of the current information bit u 1 from L (l) (u1 ), leaving only the effect of the parity constraints, thus providing an independent estimate of the information bit uz to decoder 2 in addition to the received soft channel L-values at time l. Similarly, the output of decoder 2 contains two terms: 1, L( 2 l (u1) = ln

[p

(uz =

+1 / ir2, Lfl) /

P (u1

=

-11 ll'2, Lfl)

l

where

r2

is the

JLi,2 ) the a priori input vector for decoder 2, and 1 [ Lcrt) + d ) (u1) J, and the extrinsic a posteriori L-

(partial) received vector and 2,

d 2) (u1) = L (2l (u1) values d 2l (uz) produced by decoder 2, after deinterleaving, are passed back to the input of decoder 1 as the a priori values

L~l) (u1 ).

Thus, the input to each decoder contains three terms, the soft channel L-values 2 1 Lcr1(0) and Leri°) (or Lcr?)) and the extrinsic a posteriori L-values d l (uz)= L~ \uz) (or d 1l (u1) = L~2 ) (uz)) passed from. the other decoder. (In the initial iteration of decoder 1, the extrinsic a posteriori L-values d 2l (uz)= d,1\uz) are just the original a priori L-values La (u1), which, as noted before, are all equal to O for equally likely information bits. Thus the extrinsic L-values passed from one decoder to the other during the iterative decoding process are treated like new sets of a priori probabilities by the MAP algorithm.) Decoding then proceeds iteratively, with each decoder passing its respective extrinsic £-values back to the other decoder. This results in a turbo or bootstrapping effect in which each estimate becomes successively more reliable. After a sufficient number of iterations, the decoded information bits are determined from the a posteriori L-values L <2l (u 1) , l = 0, 1, · · · , K - v - 1, at the output of decoder 2. Because the decoded output is taken only after the final iteration, it is more accurate to refer to the SISO constituent decoders as a posteriori probability (APP) estimators rather than MAP decoders, since their outputs are extrinsic a posteriori L-values that are passed to their companion decoder for more processing. A more complete discussion of iterative turbo decoding is given in Section 16.5. Although it is true, as stated previously, that the extrinsic a posteriori Lvalues Le (u1) passed between decoders during the first iteration of decoding are independent of u1, this is not so for subsequent iterations. Thus, the extrinsic information becomes less helpful in obtaining successively more reliable estimates of the information bits as the iterations continue. Eventually, a point is reached at which no further improvement is possible, the iterations are stopped, and the final decoding estimate is produced. Methods for determining when to stop the iterations, known as stopping rules, are discussed in Section 16.5. It is worth pointing out here that the term turbo in turbo coding has more to do with decoding than encoding. Indeed, it is the successive feedback of extrinsic information from the SISO decoders in the iterative decoding process that mimics thP. fpp,-lh~rk n.f pvh~11~t

ga~p.~ -in a tnrh.n.rh-::i.rgP..1

P.ng1nP.,

Section 16.2

Distance Properties of Turbo Codes

11313

Finally, before moving on to a more detailed discussion of turbo coding, we note that many of its features are similar to those observed for low-density parity-check (LDPC) codes, to be discussed in Chapter 17. Both encoding schemes produce codes with randomlike weight distributions and a thin weight spectrum. Both decoding methods make use of APP likelihoods in an iterative process, and they both employ the concept of extrinsic information. In fact, it was the discovery of turbo coding in 1993 that led to a rediscovery of the merits of LDPC codes, which had been largely neglected by the research community for more than 30 years. 115"2

IIJJ!:SilAN(IE i?'r.Ol?'IEr.lHE5 Of llL!IRl!l\O (OIIJJIES

As illustrated in Examples 16.1 through 16.3, the fundamental property of turbo codes that allows them to achieve such excellent performance is the randomlike weight spectrum, or spectral thinning, produced by the pseudorandom interleaver. In this section we examine the weight spectrum of turbo codes in more detail. In particular, we consider a series of examples for parallel concatenated codes (PCCs), including both parallel concatenated block codes (PCBCs) and parallel concatenated convolutional codes (PCCCs ). As noted in the remarks following Example 16.3, the exact weight spectrum of a turbo code depends on the particular interleaver chosen. Thus, in order to avoid exhaustively searching all possible interleavers for the best weight spectrum for a specific PCC, we introduce the concept of a uniform interleaver [11]. ]D)JEnNnmN ]_(iii"]_

A uniform interleaver of length K is a probabilistic device

that maps a given input block of weight w into all its distinct ( tions with equal probability

!)

permuta-

( Kw)·

Using the notion of a uniform interleaver allows us to calculate the average (over all possible interleavers) weight spectrum of a PCC. This average weight spectrum is typical of the weight spectrum obtained for a randomly chosen interleaver.

Consider the (7, 4, 3) Hamming code in systematic form. The weight enumerating function (WEF) for this code is given by A(X)

= 7X 3 + 7X 4 + X 7 ;

(16.20)

that is, in addition to the all-zero codeword, the code contains 7 codewords of weight 3, 7 codewords of weight 4, and the all-one codeword of weight 7. The complete list of 16 codewords is shown in Table 16.4. Splitting the contributions of the information and parity bits gives the input redundancy weight enumerating function (IRWEF) (16.21) In other words, there are 3 codewords with information weight 1 and parity weight

2, 1 codeword with information weight 1 and parity weight 3, 3 codewords with

784

Chapter 16

Turbo Coding TABLE 16.4: Codeword list for

the (7, 4, 3) Hamming code. Information 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1

0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1

0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1

Parity 0 1 1 0 0 1 1 0 1 0 0 1 1 0 0 1

0 0 1 1 1 1 0 0 1 1 0 0 0 0 1 1

0 1 0 1 1 0 1 0 1 0 1 0 0 1 0 1

information weight 2 and parity weight 1, and so on. Finally, the conditional weight enumerating function (CWEF) for each input weight is given by A 1 (Z) = 3Z 2 + z3, A2(Z)

= 3Z + 3Z 2,

A3(Z)

= 1 + 3Z,

A4(Z)

= Z 3.

(16.22)

Now, we examine how the uniform interleaver concept can be used to compute the average IRWEF for PCCs. First, consider the general case shown in Figure 16.7 of a PCBC with two different (n, k) systematic block constituent codes, C1 and C2, with CWEFs given by A~ 1 (Z) and A~2 (Z), w = 1, · · · , k, connected by a uniform interleaver of size K = k. The original information block and both parity blocks are transmitted, resulting in a (2n - k, k) PCBC. Assume that a particular input block of weight w enters the first encoder, thereby generating one of the parity weights in A~1 (Z). From the definition of the uniform interleaver, it follows that there is an equal probability that the second encoder will match that parity weight with any of the parity weights in A~2 (Z). Thus, the average CWEF of the PCBC is given by (16.23)

Section 16.2

Distance Properties of Turbo Codes

735

Transmitted bits

-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ~ - - - - - - - - - I

1 I

: 1, k . ~>I, n - k ~ 1, n - k L--------------1-------------\:------ ______

~

I

:

~-----I

Systematic code C 1(n,k) Interleaver length k Systematic code Cin,k)~

k

FIGURE 16.7: A parallel concatenated block code.

and the average XRWEF is given (16.24)

Also, we can express the bit CWEF and the bit XRWEF as follows: (16.25a)

and BPC (W, Z)

=L

Bw,z ww zz

=

w.z

where Bw,z

=

(16.25b) (l::ow::oK)

(w/ K)Aw,z· Finally, the average codeword and bit WEFs are given by

A Pc (X)

=

L

AdXd

=

Ape (W, Z)lw=Z=X

(dmin:'.od)

(16.26a)

and BPC (X)

=

L

BdXd

= BPC (W,

Z)IW=Z=X

(dmin:'.od)

L

(16.26b) ww B;c (Z) IW=Z=X

.

(l:c:w9()

The following remarks apply to the use of the uniform interleaver concept to calculate WEFs for PCBCs.

786

Chapter 16 ®

®

@

Turbo Coding

The codeword CWEF A~:C (Z), IRWEF A PC (W, Z), and WEF A PC (X) of the PCBC are all average quantities over the entire class of uniform interleavers, and thus their coefficients may not be integers, unlike the case in which these quantities are computed for the constituent codes individually. Equations (16.26) represents two different ways of expressing the codeword and bit WEFs of a PCBC, one as a sum over the codeword weights d that is valid for both systematic and nonsystematic codes and the other as a sum over input weights w that applies only to systematic codes. The codeword and bit WEF expressions in (16.26a) and (16.26b) are general and apply to all systematic codes. Expressions similar to the sums over input weights w that are valid for nonsystematic codes are given by A(X)

=

A(W, X)lw=l

=

B(X)

=

B(W, X)lw=l

=

I: ww Aw(X)jW=l (1:::w:::K)

(16.26c)

ww Bw(X)jW=l,

(16.26d)

and

I:

(1:::w:::K)

where A(W, X) and B(W, X) are the input-output weight enumerating functions (IOWEFs), and Aw(X) and Bw(X) are the conditional weight enumerating functions (CWEFs) of the nonsystematic code. @

A more general class of PCBCs results if the two constituent codes, C1 and C2, have different block lengths, n1 and n2.

EXAMPLE 115.4

(Co111tirn.1ed)

For the (10, 4) PCBC with two identical (7, 4, 3) Hamming codes as constituent codes, the CWEFs are A[c(Z)

=

AfC (Z) = AfC (Z) =

(3z2

+ z3)2 4

(3Z + 3Z 2)2 6 (1

+ 3Z) 2 4

=

= 2.25z 4 + 1.5Z5 + o.25z 6 ,

= 1.5Z2 + 3Z3 + 1.5Z4, (16.27) 0.25 + l.5Z + 2.25Z 2,

( z3)2

A[c(Z) = -1- = z6,

the IRWEFs are A Pc cw, Z) =

w (2.25z 4 + 1.5z5 + o.25z 6) + w2 ( 1.5Z 2 + 3z 3 + 1.5z 4 ) + w3 ( 0.25 + 1.5z + 2.25z 2) + w4 z 6

(16.28a)

Section 16.2

Distance Properties of Turbo Codes

7/87/

and

=

w (o.s6z 4 + V"V

3

( 0.19

( 0.

+ 1.sz3 + o.75z 4 ) +

+ 1.12Z +

(16.28b)

and the WEFs are A PC (X)

= 0.25X 3 + 3X 4 + 7 .5X 5 +

+ 0.25X 7 + X1o

(16.28c)

and BPC (X)

= 0.19X 3

+ 1.87X4 + 3.75X5 + 1.12x 6 + 0.06X 7 + x 10 .

(16.28d)

We now make the following observations regarding Example 16.4: o

The coefficients of the codeword WEF's for the PCBC are fractionaL owing to the effect of averaging.

o

The sum of aH the coefficients in A PC (X) equals 15, the total number of nonzero codewords in the PCBC.

o The minimum distance of the (10. 4) PCBC is almost certainly either 3 or 4, depending on the particular interleaver chosen. o fo this example, a detailed analysis (see [11]) of the 4! = 24 possible interleaving patterns reveals that exactly 6 result in a minimum distance of 3, and the other 18 result in a minimum distance of 4. o The average codeword multiplicities of the low-weight codewords are given by A3 = 0.25 and A4 = 3.0. o

The average bit multiplicities of the low-weight codewords are given by B3 = 0.19 and B4 = 1.87.

Now, we examine the more general case, illustrated in Figure 16.8, of forming the constituent codes in a PCBC by concatenating h codewords of a basic (n, k) systematic code C to form an (hn, hk) h-repeated code c" and using an interleaver size of K = hie Again, the original information block and both parity blocks are transmitted, resulting in a (2h11 - hie hk) PCBC. In this case, the IRWEF of the code c" is given by (16.29) where we have included the 1 'sin (16.29) to account for the fact that a combination of all-zero codewords and nonzero codewords in the basic code C can be used ch c" to form a nonzero codeword in the h-repeated code c". If Aw 1 (Z) and Aw2 (Z)

188

Chapter 16

Turbo Coding Transmitted bits

__________________________ _/ ________________ _

--

Cod,C('fEf;1. •.·.· · -· I

----:-f:"tN~~~--c -~~-Le! I

I • 1 . . h I I_____________________ I

1

•

-

Interleaver length K = hk

FIGURE 16.8: A PCBC with repeated block constituent codes.

represent the CWEFs of two h-repeated constituent codes average CWEF of the PCBC is given by

er and C1, then the (16.30)

the average IRWEFs are given by Ape (W, Z)

=

L

ww A~c (Z)

(16.31a)

(l'S.w'S.hk)

and BPC(W, Z) =

~ L

~WwAPC(z)

hk

w

,

(16.31b)

(l'S.w'S.hk)

and the average WEFs can be computed using (16.26). EXAMPLE 16.5

A PCIBC with Repeated 1Bk1dc Constituent Codes

Consider the (20, 8) PCBC formed by using h = 2 codewords from the (7, 4, 3) Hamming code, that is, a (14, 8) 2-repeated Hamming code, as both constituent codes, along with a uniform interleaver of size K = hk = 8. The IRWEF of the (14, 8) constituent code is A(W, Z)

= [1 + W(3Z 2 + Z 3 ) + W2 (3Z + 3Z 2 ) + W3 (1 + 3Z) + W 4 z 3 J2

-1

= W(6Z 2 + 22 3 ) + w2 (6Z + 6Z 2 + 9z 4 + 6z 5 + z 6 ) +

w3 (2 + 6Z + isz 3 + 24Z 4 + 6Z 5 ) + w4 (15Z 2 + 40z 3 + 1sz4 ) + W5 (6Z + 24Z 2 + isz 3 + 6Z 5 + 2z 6)

+

w6 c1 + 6z + 9z 2 + 6z 4 + 6Z 5 ) + w7 c2z 3 + 6Z 4 ) + w8 z 6 ,

(16.32)

Section 16.2

Distance Properties o"f Turbo Codes

1~'.'lJ

and its CWEFs for each input weight are given by

+ A3(Z)

As(Z) =

= 1sz2 +

+6z 5 ,

= 2 + 6Z + 18Z3 +

6Z + 24Z 2 + 18Z3 + 6Z 5 + 2z 6, A6 (Z) = 1 + 6Z + 9Z 2 + 6Z 4 + 6Z 5 , (16.33)

Note that this code still has a minimum distance of 3; that is, it is a (14, 8, 3) code, since each of the seven weight-3 codewords in one code can be paired with the all-zero codeword in the other code, resulting in a total of 14 weight-3 codewords in the 2-repeated code. Thus, by itself, this (14, 8, 3) code would not be interesting, since it is longer (more complex) than the (7, 4, 3) code but does not have better distance properties; however, when it is used as a constituent code in a (20, 8) PCBC, the resulting code has better distance properties than the (10, 4) PCBC formed from a single Hamming code.

Using (16.30) we can compute the CWEFs of the (20, 8) PCBC with two identical (14, 8, 3) 2-repeated Hamming codes as constituent codes as follows: 2

(6z 2 + 2z 3 ) ---8

= 4.5z 4 + 3z5 + o.5z 6,

(6z + 6Z 2 + 9Z 4 + 6zs + z6) 28

2

APC(Z) = - - - - - - - - - 2

= L29Z 2 + 2.57z 3 + L29Z 4 + 3.86z5 +

6A3Z

6

+ 3Z 7 +

3.32z 8 + 3.86Z 9 + L93z 10 + OA3Z 11 + o.o4z 12 ,

~

(2 + 6Z + 18Z3 + 24Z 4 + 62 5 ) A3 (Z) = 56 PC

2

= 0.07 + OA32 + 0.642 2 + L292 3 + 5.572 4 + 5.572 5 + 7.07z 6 + 15A3Z7 + 14.1428 + 5.142 9 + o.642 10 , (15Z2 + 402 3 + 152 4 ) 70

2

APC(Z)= ~ - - - - - - ~ 4

= 3212 4 + 17.142 5 + 29.29Z 6 + 17.14Z7 + 3212 8 , Pc ( 6z As (Z) =

+ 24Z 2 + 18Z3 + 6z 5 + 226) 56

2

= 0.64Z 2 + 5.14Z 3 + 14.14Z4 + 15A3Z5 + 7.07Z 6 + 5.57Z 7 + 5.572 8 + L292 9 + o.64Z 10 + OA32 11 + o.072 12 ,

790

Chapter 16 PC

A6 (Z) =

Turbo Coding

(1

+ 6Z + 9Z 2 + 6Z 4 + 6Z 5 )

2

28

= o.o4 + 0.43Z + l.93Z 2 + 3.86Z 3 + 3.32Z 4 + 3z5 + 6.43Z 6 + 3.86Z 7

Afc (Z) = A{c(Z)

=

(2Z 3 + 6Z 4 ) 8

(z6)2

-1-

+ 1.29z8 + 2.57z 9 + 1.29z10 ,

2

= z12_

= o.5z 6 + 3z7 + 4.5z 8 , (16.34)

Then, we can compute the IRWEFs A PC (W, Z) and sPC (W, Z) and the WEFs A PC (X) and sPC (X) using (16.31) and (16.26), respectively (see Problem 16.4). We now conclude our discussion of PCBCs with a few observations regarding Example 16.5. ®

0

®

The minimum distance of the (20, 8) PCBC is almost certainly either 3 or 4, depending on the particular interleaver chosen, the same as for the (10, 4) PCBC. However, the average codeword multiplicities of the low-weight codewords have decreased from A3 = 0.25 to A3 = 0.07 and from A4 = 3.0 to A4 = 1.72, respectively, despite the fact that the (20, 8) PCBC contains 16 times as many codewords as the (10, 4) PCBC. Also, the average bit multiplicities of the low-weight codewords have decreased from B3 = 0.19 to B 3 = 0.03 and from B4 = 1.87 to B4 = 0.48, respectively.

"' This is an example of spectral thinning; that is, the multiplicities of the lowweight codewords in a PCBC are decreased by increasing the length of the constituent code and the interleaver size. e

®

Increasing the code length and interleaver size by further increasing the repeat factor h leads to additional spectral thinning, which results in improved performance at low SNRs, but for block constituent codes the beneficial effect of increasing h diminishes for large h. A better approach would be to increase the interleaver size by using longer block constituent codes, but efficient SISO decoding of the constituent codes is more difficult for large block sizes.

Now, we consider the case of PCCCs, in which the constituent codes are generated by convolutional encoders, as illustrated in Figure 16.1. An exact analysis, similar to the foregoing examples for PCBCs, is possible but is complicated by issues involving termination of the constituent encoders. Hence, we make the simplifying assumption that both constituent encoders are terminated to the all-zero state. (As noted previously, this is normally the case for the first encoder but not for the second encoder, because the required termination bits are generated by the interleaver only

Section 16.2

Distance Properties of Turbo Codes

191

with probability 1;2v, where vis the constraint length of the encoder.) The resulting analysis is exact whenever the interleaver size K is at least an order of magnitude larger than the constraint length v of the constituent encoders. Because turbo codes are most effective for short constraint lengths and large interleaver sizes, this condition always holds in practice. We begin by illustrating the procedure for computing the CWEFs Aw(Z) of the equivalent block code produced by a convolutional encoder that starts in the all-zero state So = I[]) and returns to the all-zero state after an sequence of length K, including termination bits. For an (n, 1, v) convolutional encoder, the equivalent block code has dimensions (nK, K - v). The situation here is somewhat different from that presented in Chapter 11, where we were interested in computing the WEF of an codewords that diverged from the aH-zero state at a particular time and remerged only once. This WEF was the appropriate one to consider for evaluating the event- and bit-error probabilities per unit time of an encoder driven by semi-infinite (unterminated) input sequences. To evaluate the block- and bit-error probabilities of PCCCs, however, the WEF must include the effect of multiple-error events, that is, error events that diverge from and remerge with the all-zero state more than once. This is because the encoders are driven finite-length (terminated) input sequences, resulting in an equivalent block code, and the error probability analysis must consider the entire block, rather than just a particular time unit. Thus, we will modify the single-error event WEF from Chapter 11 to obtain a multiple-error event WEF appropriate for PCCCs. From Figure 16.9 we can see that any multiple-error event in a codeword belonging to a terminated convolutional code can be viewed as a succession of single-error events separated by sequences of O's. We begin by considering all codewords that can be constructed from a single-error event of length Jc ::: K; the error event is surrounded by (K - Jc) O's. Because the (K - Jc) O's are divided into two groups, one preceding and one following the error event, the number of single-error events is the number of ways of summing two nonnegative integers that add to K - Jc. Thus, the multiplicity of block codewords for single-error events is given by c[Jc,1]=( K-:+l

)=K-A+l.

(16.35)

Next, consider a pair of error events with total length Jc ::: K; that is, the two error events have a total of (K - Jc) O's appearing before, after, or between them. In this case, the (K - Jc) O's are divided into three groups, and the number of double-error events is the number of ways of summing three nonnegative integers that add to All-zero (correct) codeword

~

Error events FIGURE 16.9:

Multiple-error events in a terminated convolutional code.

792

Chapter 16

Turbo Coding

K - A. Thus, the multiplicity of block codewords for double-error events is given by _ ( K - A + 2 ) _ (K - A+ 2) (K - A + 1) c [ A, 2] . 2 2

(16.36)

In general, the number of h-error events with total length A ::: K is the number of ways of summing h + 1 nonnegative integers that add to K - A, and the multiplicity of block codewords for h-error events is given by (see Problem 16.7) c [A, h]

=(

K - :

+h

) .

(16.37)

To proceed with the computation of the CWEFs Aw (Z) of the equivalent block code produced by a convolutional encoder that terminates after an input sequence of length K, we must know the number A~:Z.>c of single-error events of length A ::: K with input weight w and parity weight z, the number A~:z.>c of double-error events of total length A ::: K with input weight w and parity weight z, and so on. We can obtain the single-error event enumerators A~\ (Z) = L(z) A~:Z.>c zz, A ::: K, directly from the IRWEF A(W, Z, L) of the unterminated convolutional code by simply dropping all terms of order larger than L K and then collecting together all terms with input weight w and length A. To determine the double-error event enumerators A~\ (Z) = L(z) A~'.z.>c zz, A ::: K, we must examine all pairs of terms in the IRWEF A(W, Z, L) for which the total length of the two error events is A ::: K. We can do this by defining the double-error event IRWEF A( 2l(W, Z, L) of the unterminated convolutional code as follows: A (2) (W, Z, L) -_ [ A (W, Z, L) ]2 .

(16.38)

We can now obtain the double-error event enumerators A~\ (Z), A ::: K, from the double-error event IRWEF A <2l(W, Z, L) by dropping all terms of order larger than LK and then collecting together all terms with input weight wand length A. We can find higher-order error event enumerators in a similar way. Then, we can compute the CWEFs as (16.39) Aw(Z) = where hmax, the largest possible number of error events associated with a weight w input sequence, depends on w and the code parameters. EXAMPLE 16.6

a

Computing WHs for a Terminated (onvoh.11tio111a! Code and

PC((

For the (2, 1, 2) systematic feedback convolutional encoder whose generator matrix is given by (16.40) and whose encoder block diagram and state diagram are shown in Figure 16.10, consider the (18, 7) block code obtained from a length K = 9 input sequence.

Section 16.2

FIGURE 16.10:

Distance Properties of Turbo Codes

7~3

Encoder diagram and state diagram for Example 16.6.

including v = 2 termination bits. The IRWEF of this encoder is given by (see Example 11.14 in Chapter 11)

+ wsz2 + wsz4) + + L1 cw2z6 + 5 4 L (2W Z + L 9 (3w 3 z 6 + 3w 4 z 6 + + 2w z + + .. ,. (16.41) 3W 5 z 4 + w6 z 2 + 2w 6 z 4 + L 6 (2W 3 Z 4 8

3

6

+ +

After dropping all terms of order larger than enumerators as follows:

A~11 (Z)

= z4,

1 Ai ) (Z) = 3

"·

z2,

1 A~ ~(Z) =

z6

=

z4.

A~11(Z)

, we obtain the single-error event

1 1 A~ , ~(Z) = 22 4. A~ . ~(Z) = 2Z 6, An(Z) = 3Z 6 ,

A (l) (Z) = 2Z 4 A (l) (Z) = 3Z 4 A (l) (Z) = 3z6 4. 7

1 A~ , j(Z)

= z2 + z4,

·

4. 8

·

4, 9

1 A~ ,

icz) = 22 4.

A~li(Z) = z4, An(Z) =

z4 .

(16.42)

Note that there are no weight-1 input sequences for this code, since a second 1 is needed to terminate the encoder. To determine the double-error event enumerators A;:\. (Z), we first form the double-error event IRWEF A (2) (W, Z. L) as follows: A< 2l(W, Z, L)

=

[A(W. Z. L)] 2

= L6w 6

z 4 + 2L 7 w 5 z 6 + L 8 (w 4 z 8 + 2w 6 z6 + 2w 7 z 4) + (16.43)

794

Chapter 16

Turbo Coding

Dropping all terms of order greater than L 9 gives us the double-error event enumerators Ai~1(Z) = z8 A~~~(Z)

= 2z 6 ,

A~~l(Z)

= 2z 8 (16.44)

2

A~~~(Z)

= z4,

A~~1cz)

= 2z4, Aflcz) = 2z 6 .

A~ 1cz)

= 2z

6

,

Aflcz)

= 6Z

6

Following the same procedure, we obtain the triple-error event IRWEF, A< 3\W, Z, L)

=

[A(W, Z, L)] 3

=

L9

w9 z 6 + · · · ,

(16.45)

and we see that there is only one triple-error event of total length A :'S 9. The triple-error event enumerator is given by (16.46) and it is clear that there are no higher-order error events of total length A :": 9. Before using (16.39) to compute the CWEFs, we must determine hmax for each input weight w. From the double-error event IRWEF in (16.43) we see that there are no double-error events with input weight w < 4; that is, hmax = 1 for input weights 2 and 3. Similarly, from (16.45) we see that there are no triple-error events with input weight w < 9; that is, hmax = 2 for input weights 4 through 7. Finally, it is clear that hmax = 3 for input weight 9. Now, using (16.37) and (16.39), we can compute the CWEFs for the (18, 7) block code as follows: A2(Z) = c[4, 1JAt1(Z) + c[7, l]Ai~~(Z) A3(Z) = c[3, l]A1~1(Z) (1) c [8 , 1] A3_3(Z)

=

A4(Z)

=

As(Z)

+ c[S, l]A~~1(Z) + c[6, l]A?~(Z) + (1) + c [9 , 1] A3_9(Z)

7Z 2 + 13Z 4

+ 7Z 6 c[S, l]Ai~1(Z) + c[6, l]An(Z) + c[7, l]Ai~~(Z) + c[8, l]Ai~1(Z) + c[9, l]An(Z)

=

= 6Z 4 + 3z 6

+ c[8, 2]Ai~1(Z)

sz 2 + 16Z4 + 3Z 6 + 3Z 8

= c[7. l]A1~~(Z) + c[8, l]An(Z) + c[9, l]An(Z) + c[7, 2]A1~~(Z)

+ c[9, 2]A12l(Z)

= 3Z 2 + 10z4 + 12z 6 + 2z 8

Section 16.2

A6(Z)

Distance Properties of Turbo Codes

1915

= c[8, l]A~:~(Z) + c[9, l]A~11(Z) + c[6, 2]A~~1(Z) + c[8, 2]A~~~(Z)

+ c[9, 2]A~~1(Z)

= z 2 + 14Z 4 + 12z 6 1 1]A 7.9 < \z)

A7 (Z) = A9(Z)

2 2 + c[8, 2]A 7.8 < ) (Z) + c[9, 2]A 7.9 < \z)

= c[9, 3JA31(Z) = z 6 .

= 7z 4

+ 2z 6 (16.47)

As a check, we note that the CWEFs include a total of 127 nonzero the correct number for an (18, 7) code. Finally, the IRWEF and WEF are given by A(W,

=

w2 ( 6Z 4 + 3z 6) + w3 (1z 2 + 13Z4 + 7z 6) + w4 (5z 2 + 16z4 + 3Z 6 + 3z8) + w5 (3z 2 + 1024 + 12z6 + 22 8) + w6

(16.48a)

and

(16.48b) Note that this terminated convolutional code has free distance 5, the same as the free distance of the unterminated code; that is, it is an (18, 7, 5) code. We can also see that it has codeword multiplicities of As = 7, A6 = 11, A7 = 16, · · ·; that is, it has a dense weight spectrum. We next calculate its average-weight spectrum when it is used, along with a size K = 9 interleaver, as a constituent code in a (3K, K - v) = (27, 7) PCCC. Before proceeding with this calculation, however, we must modify the uniform interleaver concept to take into account that the termination bits of a convolutional encoder are not information bits. fo other words, for an input sequence of length the number of information bits is only N - v, where v is the encoder constraint length, and we must modify the denominator in the uniform interleaver averaging expression given in (16.23) so that we divide only by the number of valid input sequences of each weight and not by all length-K sequences of a given weight; that is, because the v termination bits are fixed for a given information sequence of length K - v, only a fraction l/2v of all length-K input sequences are valid. This technical detail complicates our example somewhat, but, as we will see shortly, for large K and small v, that is, normal turbo code parameter values, we can use an approximate analysis that makes this modification unnecessary.

We begin by noting from (16.47) that there is 1 valid input sequence for weights 0 and 9, there are no valid input sequences for weights 1 and 8, there are 9 valid input

796

Chapter 16

Turbo Coding

sequences for weights 2 and 7, and there are 27 valid input sequences for weights 3, 4, 5, and 6. Then, using the modified version of (16.23), we can compute the average CWEFs of the (27, 7) PCCC as follows: Afc(Z)=

2 (6z 4 + 3z6) 9 =4zs+4z10+z12, 2

APC(Z)

(7z 2 + 13z4 + 7Z 6) =------

27 4 = 1.81z + 6.74Z 6 + 9.89Z 8 + 6.74z 10 + 1.81Z 12 , 6 + 3zs) 2 +_ (sz 2 + 16z42_7 3z_ Af c (Z) = _____ _ __ 3

= 0.93Z 4 + 5.93Z 6 + 10.59z8 + 4.67z 10 + 3.89Z 12 + o.67z 14 + o.33Z 16 ,

AfC (Z) =

(3z 2 + 10z4 + 12z 6 + 2z 8 ) 27

2

= o.33Z 4 + 2.22z 6 + 6.37Z 8 + 9.33Z 10 + 6.81z 12 + 1.78Z 14 + o.1sz 16 , 2 (z 2 + 14Z4 + 12z6) A[c(Z) = 27 = o.o4z 4 + 1.04z6 + 8.1sz 8 + 12.44z 10 + 5.33z 12 , 2 (7z 4 + 2z 6) Afc (Z) = = 5.44z 8 + 3.11z 10 + 0.44z 12 , 9 A9PC(Z)

( z6)2 = ~1- = z12 .

(16.49)

Finally, we can compute the average IRWEF A PC (W, Z) and the average WEF A PC (X) using (16.24) and (16.26a), respectively (see Problem 16.8). We now make a few observations regarding Example 16.6. 0

o

111

The free distance of the (27, 7) PCCC is 7, an increase of 2 compared with the (18, 7) constituent code. An increase in free distance is expected, though, since the (27, 7) code has a lower rate than the (18, 7) code. Note that in this case the minimum-weight codeword is produced by an input sequence of weight 3. The average multiplicities of the low-weight codewords, namely, A7 = 1.81, As = 0.93, and A9 = 7.07, are small compared with the low-weight multiplicities of the (18, 7) code. This is another example of spectral thinning. Increasing the code length and interleaver size K leads to additional spectral thinning, which results in improved performance at low SNRs. In fact, we will see shortly that for large K, the multiplicities of the low-weight codewords in a PCCC are reduced by a factor of K compared with the constituent codes. This multiplicity reduction factor is called the interleaver gain.

Section 16.2 o

Distance Properties of Turbo Codes

71971

The calculation of the average bit KRWEFs B(W, Z) and BPC (W, Z) for this example is also complicated by the fact that the termination bits are not information bits, and the factor ( w / K) in (16.25) must be adjusted to consider only the information bits (see Problem 16.9).

o The observations that turbo coding produces (1) a normal increase in free distance owing to the reduction in code rate and (2) a large decrease in codeword and bit multiplicities owing to the interleaver gain illustrate that, unlike conventional code design, turbo codes are designed to reduce the low-weight multiplicities rather than to increase the free distance. This results in much better performance than conventional codes achieve at low and moderate SNRs but somewhat weaker performance at high SNRs. o

By including the termination bits for the first encoder in the interleaver, we cannot guarantee that the second encoder terminates (in fact, as noted earlier, it will do so only with probability 1/2v), but this has little effect on code performance for large K.

Before extending the analysis to larger block lengths and interleaver sizes, we note that the uniform interleaver averaging represented by (16.23) results in dividing the product of the constituent code CWEFs by the factor (

! ).

Because

there are no weight-1 input sequences to a terminated systematic feedback encoder (because of the termination bits), the nonzero input weight with the smallest division factor in (16.23) is the w w

= 2 term, for which the division factor is ( ~ ) . For

= 3, the division factor (

1)

is larger by roughly a factor of K. fo other words,

compared with their relative influence in the individual constituent codes, weight3 input sequences in PCCCs are less important than weight-2 input sequences, since they are associated with lower average codeword and bit multiplicities. Thus, particularly for large block lengths K, the codewords associated with weight-2 input sequences become the most important contributors to the low-order terms in the weight spectrum of PCCCs. To develop an approximate weight spectrum analysis of PCCCs for large block lengths, we begin by simplifying the expression given in (16.37) for the number of codewords containing h error events of total length Jc in a terminated convolutional code with block length K. Because, for the low-weight codewords of primary interest, both the number of error events h and the total length Jc of error events cannot be large (or else the weight would also be large), for large K it is reasonable to approximate the number of codewords containing h error events in a block as (16.50) independent of the total length Jc of the error events. Then, the CWEFs of the equivalent (n K - v) block code are given by Aw(Z)

=

L (l:5:ch:5:chmax)

C

[h] A~\Z),

(16.51)

798

Turbo Coding

Chapter 16

where A;i\Z) is the h-error event enumerator for input weight w. (We note here that the h-error event enumerators of (16.39) and (16.51), A~'\_(Z) and A~'l(Z),

A~\

respectively, are related by the expression A~'\Z) = L(),:::~) (Z); that is, A~') (Z) counts h-error events of all lengths.) From (16.23), (16.50), and (16.51), the average CWEFs of the PCCC with the same (nK, K - v) terminated convolutional code as constituent codes are given by C

[h1] C [h2] A (hi) (Z)A (h2) (Z)

(!)

(l:::h1 :Shmax) (l:::h2:Shmax)

For K

w

w

(16.52) .

» h, we can use the approximation K ) ;::::; K" ( h h!

(16.53)

to obtain (16.54)

We can further approximate by saving only the (most significant) term in the double summation of (16.54) with the highest power of K, that is, the term corresponding to h1 = h2 = hmax, which gives us the approximate codeword CWEF A PC (Z)

w!

;::::;

K(2h 111 ax-W)

[A

(hmax!)2

w

(h11,ax)

(Z)

]2

(16.55a)

w

and the approximate bit CWEF BPC (Z) w

w! ;::::; _w . K(2hmax-W) [ A (hmax) (Z) K (hmaxl)2 w

w · w!

=

(hmaxl)

K(2hmax-w-1)

[ A~max\z)

2

]2

]2 (16.55b) .

(It is worth noting that taking only the term corresponding to h1 = h2 = hmax in (16.54) is equivalent to saying that for a fixed input weight w, the lowest-weight codeword is likely to contain the maximum number of error events, so that it is merged with the all-zero codeword as much as possible.) Finally, the average IR WEFs of the PCCC are given by Ape (W, Z)

=

L

ww A~c (Z)

(16.56a)

(1:::w:::K)

and BPC(W,

Z) =

) "-d

(1:::w:::K)

~ ww A~c(Z) =

K

L (l:::w:::K)

ww s:,c (Z),

(16.56b)

Section 16.2

Distance Properties of Turbo Codes

1!11'9

and we can compute the average WEFs using (16.26). (In the calculation of the approximate IRWEFs from (16.56), the sums should be taken over those w for which the approximations of (16.50), (16.53), and (16.55) are reasonably accurate, that is, w's of relatively low weight.) The extension of this analysis to PCCCs with different constituent codes is straightforward.

Consider a (3K. K - 2) PCCC in which both constituent codes are generated by the (2. 1, 2) systematic feedback encoder of Example 16.6, but now for a block length of K » v = 2. For input weights w = 2 and 3, h 111 ax = 1 and (16.55) simplifies to A~c (Z)

and B,~c (Z)

and for input weights w

~

w! · K( 2 -wl [ A~~\Z) ]

~ w. w!. K(l-w)

[ A2l (Z)

2

(16.57a)

J2.

(16.57b)

= 4 and 5, hmax = 2, and (16.55) is given by

iwl ·Kc

A~c (Z) ~

4 -wl

[A~J (Z) ]2

(16.58a)

and (16.58b) (Note that in Example 16.6 when K = 9, h 111 ax = 2 also for w = 6 and 7, but this is not the case for large Including only these terms in the approximate average IR WEFs for the PCCC gives APC(W, Z) ~

L

ww A~c(Z)

(2"":W"":5)

-30 w5 [ A 5<2\z) ]2 + ... [(

(16.59a)

and BPc (W. Z) ~

L

ww B~c (Z)

(2"":W"":5)

4 2 [ A< 1\Z) ]2 = -W K 2 2 150 K 2 w5 [ A~ l (Z)

18W 3 [ A(l)(Z) ]2 + -W 24 4 [ A( 2)(Z) ]2 + + -K2 3 K 4

]2 + ....

(16.59b)

The single- and double-error event enumerators A2l (Z) and A~) (Z) needed to evaluate (16.59) include all single- and double-error events up to length K; however, the terms of most interest, that is, the lowest-weight terms, can be obtained from

800!

Chapter 16

Turbo Coding

Example 16.6, which counts all error events up to length K and (16.44), we obtain

= 9. Thus, from (16.42)

A;1\z) ~ z 4 + z 6

A;ll(z) ~ z 2 + 3Z 4 + sz 6

Ai2) (Z) ~ z 8

A~2) (Z) ~ 2Z 6 + 2Z 8 ,

(16.60)

and (16.59) becomes APc(W,Z)~2w 2 [z 4 +z 6 6W =

4

[z

8

J2 + !w

J2 + ; w

5

3

[z 2 +3z 4 +sz 6

6

[2z + 2z

w2 ( 2z 8 + 4Z 10 + 2z 12 )

+

( 12oz

12

J2 + ...

w4 ( 6z 16 )

4 w3 ]:_(6Z + 36z 6 + 114Z 8 + K

w5 ~

8

J2 +

+

isoz 10 + 1soz 12 ) +

+ 24oz 14 + 12oz 16 ) + ...

(16.61a)

and sPc

cw, Z) ~ w2

!(

4Z 8 + 8Z 10 + 4z 12 ) +

w3 ;

2 ( 18Z

w5 ;

2

4

( 6002

w4 ~

( 24Z

16

)

+

+ 108z6 + 342z 8 + 54oz 10 + 45oz 12 ) + 12

+ 12ooz 14 + 6ooz 16 ) + ....

(16.61b)

Note that the approximate expressions of (16.61) contain no terms of order (1/ K) 3 or higher. (These terms clearly have little influence on the IRWEFs for large values of K.) Finally, we obtain the approximate average WEFs from (16.26) as follows:

and 18 108 4 342 8 540 4 BPC(X) ~ -X7 + -x9 + -XlO + -Xll + -x12 + - 2 xl3 + -x14 + .... K2 K2 K K2 K K K (16.61d) The following remarks apply to Example 16.7: o

The minimum free distance of the (3K, K - 2) PCCC still appears to be 7, but the average codeword multiplicity is only A7 = 6/K. For large K, this means that the probability that a particular interleaver will result in a weight-7 codeword is very small; that is, the free distance in almost all cases will be greater than 7.

Section 16.2 o

Distance

of Turbo Codes

~IOli

Approximations (16.61c) and (16.61d) do not indicate the presence of any codewords of weight 8, because we only the most significant term in the double summation of (16.54). H the term leading to weight-8 codewords had not been deleted from (16.54), however, it would have order (1/ K) 2 in A PC (X) and order (1/ K) 3 in BPC (X); that it would have a negligible effect on the WEFs for large K.

o The average multiplicity of weight-9 codewords is A9 = 36/ K; that is, for large K the probability of a weight-9 codeword is very small. o

The average multiplicity of weight-10 codewords is A10 = 2; that is, the interleaver does not purge the code of weight-10 and in almost all cases the free distance of the PCCC will be 10. We see that the weight-10 codewords are generated weight-2 input sequences, thus illustrating the importance of wei.ght-2 sequences when K is large that was noted earlier.

o

The (average of) two weight-10 codewords are produced the K - 2 input sequences un(D) = D 1(1 + D 3 ), 0 ::: l ::: K - 3, where we note that 1 + D 3 is the shortest weight-2 input sequence that terminates the encoder. This sequence generates the weight-4 parity sequence w(D) = 1 + D + D 2 + D 3 . A pseudorandom interleaver will typically leave two of these K - 2 input sequences unchanged (except for time shifts), and in these cases the second encoder will output the same weight-4 parity sequence, resulting in codewords of weight 10. (To see this, assume that the first 1 in one of the foregoing weight2 input sequences is permuted to an arbitrary position by the interleaver. There are then two positions out of the remaining K - 1 positions, that the second 1 can appear and still give the same input pattern, namely, either three positions before or three positions after the first 1. Thus each of the K - 2 "bad" input sequences has probability 2/(K - 1) of retaining its bad property after interleaving, which leads to the average multiplicity of A10 = 2 for weight-10 codewords.)

o

For each constituent terminated convolutional code by itself, the low-weight codeword multiplicities increase linearly with K (because of time invariance), but for the PCCC the multiplicities of the low-weight codewords are smaller by factors of (1/ K)l\ p ::: 1. Note that the largest factor, that is, (1/ K)1, is associated with terms resulting from input sequences of weight 2.

o

From (16.61d), we see that the bit multiplicities of the low-weight codewords are B7 = 18/K 2 , B9 = 108/K2 , B 10 = 4/K, and so on. Because the bit multiplicities of the low-weight codewords in the PCCC decrease with we say that the PCCC possesses interleaver gain.

We continue the example by computing the approximate low-weight codeword and = 100, 1000, and 10000. The results are presented in Table 16.5, where the entries for the bit multiplicities Bd clearly portray the interleaver gain effect. For each weight d, the bit multiplicities, using (16.61c) and (16.61d), for three specific cases: K

802

Chapter 16

Turbo Coding

TABLE 16.5: Approximate codeword and bit multiplicities for the (3K, K - 2) PCCC of Example 16.7.

K

100

1000

10000

d

Aa

Ba

Aa

Ba

Aa

Ba

7 8 9 10 11 12 13 14 15 16 17 18 19

0.06 0.00 0.36 2.00 5.40 4.00 1.80 2.00 1.50 0.00 1.20 0.00 2.40

0.0018 0.0000 0.0108 0.0400 0.1620 0.0800 0.0540 0.0400 0.0450 0.0000 0.0600 0.0000 0.1200

0.006 0.000 0.036 2.000 0.540 4.000 0.180 2.000 0.150 0.000 0.120 0.000 0.240

0.000018 0.000000 0.000108 0.004000 0.001620 0.008000 0.000540 0.004000 0.000450 0.000000 0.000600 0.000000 0.001200

0.0006 0.0000 0.0036 2.0000 0.0540 4.0000 0.0180 2.0000 0.0150 0.0000 0.0120 0.0000 0.0240

0.00000018 0.00000000 0.00000108 0.00040000 0.00001620 0.00080000 0.00000540 0.00040000 0.00000450 0.00000000 0.00000600 0.00000000 0.00001200

value of Bd decreases by one or two orders of magnitude for each order of magnitude increase in K. Note in particular that the interleaver gain for codeword weights 7 and 9 is larger by roughly a factor of (1/ K) than for codeword weight 10. This is because weight-10 codewords are generated by weight-2 input sequences, whereas weight-7 and weight-9 codewords are generated by weight-3 input sequences. This is another illustration of the importance of weight-2 input sequences when the block length K becomes very large. In Problems 16.10 and 16.11 we repeat Examples 16.6 and 16.7 for the systematic feedback convolutional encoder whose generator polynomials are inverted, that is, the encoder given by (16.62) The (2, 1, 2) convolutional code generated by this encoder also has dtree = 5; however, in this case, the minimum-weight codeword is produced by an input sequence of weight-2, that is, lll(D) = 1 + D 2 . Thus, when this code is converted to a (3K, K - 2) PCCC, its minimum free distance for both small and large values of K will be 8. (The parity weight corresponding to the weight-2 input sequence is 3, which is repeated twice in the PCCC.) In other words, when the minimum-weight codeword is produced by an input sequence of weight-2, increasing the block length K results in interleaver gain, which reduces the bit multiplicities, but no improvement in free distance is achieved. This contrasts with the code in Examples 16.6 and 16.7, where the free distance improved from 7 to 10 by increasing the block length K. From the results of Examples 16.6 and 16.7 and Problems 16.10 and 16.11 we conclude that to maximize the free distance of a PCCC, a constituent code should

Distance Properties of Turbo Codes

fBIIJ)3l

be chosen whose minimum-weight codeword is produced by an input sequence of weight greater than 2. This is usually the case when the feedback polynomial is chosen to be primitive, as in Examples 16.6 and 16.7. This is because a primitive polynomial has maximum cycle length; that is, it maximizes the length of the weight2 input sequence needed to terminate the encoder. Depending on the numerator polynomial, this normally results in higher parity weight than when one or more shorter, higher-weight sequences terminate the encoder. Because the free distance determines the asymptotic, that is, large-SNR behavior of a code, primitive feedback polynomials are good choices to optimize the large-SNR performance of PCCCs. One should not conclude, however, that the choice of primitive feedback polynomials also optimizes the small-SNR performance, where the low-weight codeword multiplicities and the dynamics of iterative decoding are more important than the free distance. fo fact, experience with different code selections suggests that, in general, primitive feedback polynomials optimize large-SNR behavior, but nonprimitive polynomials with shorter cycle lengths are better for small SNRs. A more extensive analysis of constituent code design for PCCCs is included in Section 16.4 The preceding discussion, along with a dose examination of the power of namely, 2hmax - w - 1, in (16.55b) illustrate that the input weights that produce lowweight codewords in the constituent code must be at least 2 to achieve interleaver gain in PCCCs. Interleaver gain results only when the power of K in (16.55b) is less than or equal to -1, so that the bit multiplicities decrease with increasing K. Because hmax is always 1 for the lowest-weight input sequences, this means that the lowest input weight w that produces low-weight codewords must be at least 2. When feedback encoders, that recursive convolutional codes, are used as constituent codes, almost an weight-1 input sequences produce high-weight parity sequences, and thus high-weight codewords. (The only exception is when the nonzero input does not appear until the end of the information block, but the number of these sequences is small and does not grow linearly with K. Also, when the encoder is terminated, then at least one more 1 must enter the encoder, and hence the total input weight is still at least 2 in this case.) Thus, for recursive constituent codes, all low-weight codewords are produced by input sequences of at least weight 2, and interleaver gain is achieved. This is not the case when short block codes are used as constituent codes. When a short block code is repeated to achieve a large block length and interleaver size, there exist low-weight codewords produced by a single 1, thus negating the possibility of interleaver gain. If a single long block code is used as a constituent code, the desired property can be achieved by choosing the weight of each row of the generator matrix (G to be large; however, efficient SISO decoding methods for such long codes are difficult to realize. Interleaver gain is also not possible when short constraint length feedforward convolutional encoders, that is, nonrecursive convolutional codes, are used as constituent codes. This is because single input 1's in short constraint length feedforward encoders produce short, low-weight output sequences, and thus large low-weight codeword multiplicities result from parallel concatenation. This point is illustrated in the next example.

804

Chapter 16

EXAMPLE 16.8

Turbo Coding

A PCCC with Feedforward Constituent Em:oders

Consider the (3K, K -2) turbo code (PCCC) generated by the parallel concatenation of a (2, 1, 2) nonsystematic feedforward encoder, as shown in Figure 16.11. (Recall that for nonsystematic encoders the weight spectrum is represented by the IOWEF rather than the IRWEF.) Assume that the generator matrix of the encoder is given by (16.63) G}1 (D) = [ 1 + D + D 2 1+ D2 ] . The code produced by this encoder is identical to the code produced by the systematic feedback encoder of Examples 16.6 and 16.7, but the mapping between input sequences and codewords is different. That is, the two codes have identical WEFs but different IOWEFs. To analyze the weight spectrum of this PCCC, we must determine both the conditional IOWEF A:;; 1 (X) of the nonsystematic nonrecursive constituent code C1 and the conditional IRWEF A:;;2 (Z) of the (2, 1, 2) systematic nonrecursive constituent code C2 whose parity generator is used to reencode the input sequence after interleaving. In our example, this systematic nonrecursive code is generated by the feedforward encoder G}1 (D) = [ 1

1 + D2 ] .

(16.64)

Then, the effect of uniform interleaving is represented by

A;,c (X. Z) ~

!j

A C1 (X)A C2 (Z) "' (

.

(16.65)

and the approximate expression for large K is (16.66) (1::::h J ::::hmax) (l::::h2 ::::hmax)

illl

,,10)

FiGURE ·j 6. ·1 ·1: A PCCC using feedforward constituent encoders.

Section 16.2

Distance Properties of Turbo Codes

iilllCJS

where A~ 1\X) and AZ12 l are the conditional IO and IR h-error event WEFs of the nonsystematic and systematic constituent codes, C1 and following the approximations of (16.53)-(16.55) results in A wPC (X,

(16.67a)

and :::::; w.

w! !1lmax ·l · h2max ·l

Because the total codeword and the average CWEFs are

l((h1,1111,+h2111",-w-l) A(h1,,111,l(Y)A(h2"'"'l W L -w

in the

PCCC are sums of powers of X

A~c (X) = A~c (X,

I

B:,c (X) = B:,c (X,

I

(16.68a)

Z=X

and Z=X

,

the average IOWEFs are given

L

A PC (W, X) =

ww A~c (X)

(1-sw-c:K)

and BPC(W,

X)

=

w Ww A PC (X)

'\""'

~[(

w

(1-c:w-c:K)

=

'\""' L,,

Ww

(16.69b)

(1-c:w-c:K)

and the average WEFs are given by Ape (X)

=

Ape (W, X)lw=l

(16.69c)

and (16.69d) Beginning with the nonsystematic constituent code generated see that its IOWEF is given by (see Problem 11.23) A(W, X, L)

=

we

+ W2 L 4 (1 + L)X 6 + W 3 L 5 (l + L)2X7 + ... + ww L w+2(l + L)w-1 xw+4 +... . (16.70)

WX 5 L 3

Going directly to the large-I< case, we note that for any 1, h 111 ax = w, since the weight one single-error event can times, and therefore (16.71) Now, for the systematic constituent code generated by (16.64), we find that its IRWEF is given by (see Problem 11.24) A(W,

z,

L)

= wz 2 L 3 + w 2 z 4 L 4 + w 2 z 2 L 5 (1 + wz 2 ) + (16.72)

806

Chapter 16

Turbo Coding

Again, for any input weight w, w 2:: 1,

hmax

=

ALw)(Z)=z2w,

w, and hence,

w::::l.

(16.73)

Now, substituting into (16.67) we obtain A~c (X, Z)

w'

R;

•

(w!) · (w!)

Kw K(w+w--w) xsw z2w = -xsw z2w w!

and

(16.74a)

K(w-1)

BPC (X Z) w

R;

(w-1)!

'

x5w z2w

(16.74b)

Finally, applying (16.68) and (16.69) gives us the average IOWEFs, APC(W, X)

Kw ww-x7w w!

R;

=

KWX7

K2

K3

2

6

+ -W2x14 + -W3X21 + ... (16.75a)

and sPccw, X)

R;

wx1

K2

+ Kw2x14 + Tw3x21 + ... ,

(16.75b)

and the average WEFs, APC(X)

R;

KX7

K2

K3

2

6

+ -x14 + -x21 + ...

(16.75c)

and (16.75d) We now conclude this section with a few observations regarding Example 16.8. @

@

The free distance of the (3K, K - 2) PCCC is 7, and the average codeword multiplicity A7 R; K grows linearly with K. Also, the average bit multiplicity of the weight-7 codewords is B7 = 1, because the weight-7 codewords are generated by weight-1 input sequences, and the interleaver has no effect (other than time shifts) on weight-1 input sequences. This contrasts with the feedback encoder case, in which the bit multiplicities all decrease with increasing K. Thus interleaver gain does not apply to feedforward encoders. The approximate expressions of (16.75) do not indicate the presence of any codewords with weights between 7 and 14, because we kept only the most significant term in the double summation of (16.66). If the terms leading to codeword weights between 7 and 14 are also retained, we find that the corresponding codeword and bit multiplicities do not grow linearly with K, and thus they will have a negligible effect for large K compared with codeword weights 7 and 14.

e From this example it is clear that when feedforward encoders are employed, hmm: is always equal tow. This is because, in the feedforward case, the weight-1

Section 16.3

Performance Analysis of Turbo Codes

~11]1

input sequence produces the minimum length (equal to the constraint length v) terminated output sequence, and hence the maximum number of error events that can be produced by a weight-w input sequence is achieved by simply repeating this shortest error event w times. This is not true for feedback encoders, since in this case the weight-1 input sequence produces an unterminated output sequence. o As in the case of block constituent codes, long constraint length feedforward encoders could be used as constituent codes, but efficient SISO decoding methods for such long codes are difficult to realize.

In the next section we use the results of the weight spectrum analysis presented in this section, along with standard union bounding techniques, to evaluate the performance of PCCCs.

We can now use the union bounding techniques developed in Chapter 12 to estimate the performance of turbo codes. We will consider both the word-error probability

L

Pw(E) :S

Ac1Pc1,

(16.76)

Bc1Pc1,

(16.77)

(d[,ee:"'cd)

and the bit-error probability

L

Pb(E) ::=:

(dfiee:"'cd)

where Ac1 is the codeword multiplicity, Bc1 is the bit multiplicity, Pc1 is the pairwise error probability for a codeword of weight d, and dfree is the minimum free distance of the code. In this chapter we concentrate on code performance for a binary-input, unquantized-output A WGN channel. fa this case, as shown in Chapter 12, Pc1 is given by (16.78) which can be bounded as Pc1 :Sf ( dfre~:Eb). (e-RE1,/Nol,

(16.79)

where R is the code rate, Eb/ No is the energy per information bit to one-sided noise power spectral density ratio, that is, the SNR, and the function f (·) is defined as

f

(x) = ex · Q

(Ex).

(16.80)

Now, substituting (16.79) into (16.76) and (16.77) gives us the expressions Pw(E) :Sf ( dfre~Eb)

L (d[,ee:"'cd)

Ac1 (e-RE1,/No

t

(16.81)

808

Chapter 16

Turbo Coding

and

(16.82)

Thus, to bound the performance of a turbo code, all one needs is the free distance dtree, the codeword WEF A(X), and the bit WEF B(X). For values of Eb/ No above the SNR at which the code rate R equals the channel cutoff rate Ro, that is, for Eb > _!_ ln No R

(21-R -1)-1

'

(16.83)

the first few terms of each WEF are all that are needed for an accurate estimate of performance. For values of Eb/ No below this limit, however, more terms are needed and the bound diverges, a characteristic that is typical of all union bounds. We now consider several examples illustrating the application of the foregoing bounds to estimate the performance of turbo codes.

Consider the rate R = 1/3 PCCC generated by the systematic feedback encoder of Examples 16.6 and 16.7. Approximations to the codeword and bit WEFs are given in (16.61c) and (16.61d), and the free distance of this code is dtree = 7. (Note that for large K, most interleavers will result in a free distance of dtree = 10, but in evaluating the bound we must use the smallest possible free distance, namely, dtree = 7.) Substituting these expressions into (16.81) and (16.82) allows us to calculate the bounds on word- and bit-error probability as functions of Eb/ No. Results are shown in Figure 16.12 for three values of the interleaver size (information block length): K = 100, 1000, and 10000. (The program used to plot the bounds of (16.81) and (16.82) was based on the approximations of the WEFs given in (16.61c) and (16.61d) but included more terms for greater accuracy.) Pw (E) is plotted in Figure 16.12(a), and Pb(£) in Figure 16.12(b). For comparison we also show the three corresponding Pw (E) simulation curves for the optimum free distance (3, 1, 4) terminated convolutional code in Figure 16.12(a), and the corresponding Pb (E) simulation curve for the unterminated version of this code in Figure 16.12(b). A 16-state convolutional code is chosen for comparison with the 4-state PCCC because their decoding complexities are roughly equal. Finally, a simulation curve for the PCCC with K = 10000 is shown in each figure. The PCCC simulations were generated using a typical pseudorandom interleaver and 18 iterations of decoding. Several observations regarding Example 16.9 follow: ®

The PCCC curves in Figure 16.12(b) clearly indicate the effect of interleaver gain. For each factor-of-10 increase in K, the Pb (E) curves improve by roughly the same factor. (Note, however, that the SNR difference in the Pb (E) curves, that is, the coding gain, achieved by increasing K becomes smaller for large Eb/ No.)

Section 16.3 <,

·,,

Performance Analysis of Turbo Codes

=.,~

\'

.,',:~

1\

\\

le-01

\\

'\

le-02

~l[l!9l

PCCC simulation 10000 ~~ PCCC bound 100 - - - · PCCC bound 1000 • • • • • PCCC bound 10000 .......... (3, 1, 4) simulation 100 ~ ·, \, (3, 1, 4) simulation 1000 -,, ·, '\ (3, 1, 4) simulation 10000 ~ • · -

\

'

'

\

'

\

\

1~

\

le-03

2

3

4

5

6

7

E1,!N0 (dB)

(a)

j

le+OO _ le-01

~

+1

PCCC simulation 10000 ~PCCC bound 100- - - · PCCC bound 1000 • • • • • -= PCCC bound 10000 .......... (3, 1, 4) simulation~-·,

-.., -

le-02 le-03 ~

kl

~

le-04 le-05 le-06 le-07 le-08

0

2

3

4

5

6

7

E,,IN0 (dB)

(b)

FIGURE 16.12: (a) Word- and (b) bit-error probability curves for a PCCC and a convolutional code with rate R = 1/3.

810

Turbo Coding

Chapter 16 ®

®

®

The Pw (E) curves for the PCCC in Figure 16.12(a) merge for large K because the number of weight-10 codewords remains fixed at two and does not decrease with increasing K. For the convolutional code, on the other hand, Pw (E) becomes worse as K increases. This is to be expected, since with increasing block length the occurrence of at least one error event becomes more likely. This effect is masked by the interleaver for the PCCC, because there are no low-weight codewords whose multiplicities increase linearly with K. Compared with the convolutional code, the PCCC offers substantial coding gain, both in terms of Pw (E) and Pb (E). For example, for Pb (E) = 10-5 , the PCCC with K = 10000 achieves a 3.8-dB coding gain compared with the convolutional code, despite the fact that the convolutional code has free distance 12, larger than the (most likely) free distance of 10 for the PCCC. The bounds are not extended to SNRs below 2 dB, the cutoff rate limit for rate R = 1/3 given by (16.83), since they diverge below this point, as noted previously. (In the case of K = 10000, the PCCC simulation is used to extend the bound below 2 dB.) At lower SNRs, the simulation curves show much better performance than predicted by the bounds, but at higher SNRs the bounds predict the performance very accurately. Tighter bounding arguments can be employed to improve the bounds at low SNRs [33], however. the bounds are being used to estimate performance for a particular interleaver and block length for which the low-order terms in the WEFs are known, the factor f (df,eeREb/ No) should be adjusted to the actual value of dtree·

e If

Turbo (ode Performance as a Functkm of Constraint

EXAMPLE 16.10 length

Now, consider the series of rate R = 1/3 PCCCs with constraint lengths v = 1, 2, 3, and 4 whose constituent encoders are shown in Table 16.6. With the exception of code E, each of the generators was chosen to have a primitive feedback polynomial so as to achieve the best possible minimum free distance. The generator for code E has a nonprimitive feedback polynomial. The results of the Pb (E) bound of TABLE 16.6:

Code parameters for Figures 16.13 and 16.14.

Code

Generator matrix G(D)

A

[1

B

1/(l+D)]

(1 + D 2 )/(1 + D + D 2 )]

[1

C

[1

(1 + D + D 2 + D 3 )/(1 + D + D 3 )]

D

[1

(1 + D + D 2 + D 4 ) / (1 + D 3 + D 4 )]

D L

r1 L,.

\JI V

11

,

n4,) / \1. 11

+

n

V

+

r.2

V

+

r.3

V

+

,.4,JJl

V

Performance Analysis of Turbo Codes

Section 16.3 le-01

AlOO~ A 1000--x~B 100 •. ,,. · B 1000 .... " .... c100~-c1000~-D 100 ... ,0 .. , D 1000 .... " .. .. E 100 .... c')... .. ElOOO-~

le-02 le-03 le-04 ~

~11

le-05

kl

~

le-06 le-07 le-08 le-09 le-10

2

FIGURE 16.13:

and 4 and K

3

4

5

7

8

9

10

Bit-error probability bounds for rate R = 1/3 PCCCs with 1J = 1. 2. 3.

= 100 and 1000.

(16.82) for each code are shown in Figure 16.13 for two values of the interleaver size (information block length): K = 100 and 1000. fo addition, the simulated performance of the two 16-state (1J = 4) codes, codes D and E, is shown in Figure 16.14 for a pseudorandom interleaver of size K = 210 = 1024 and 18 iterations of decoding. The following remarks apply to Example 16.10 o

The bounds in Figure 16.13 indicate that substantial coding gains can be achieved at P1, (E)'s below 10- 5 by increasing the constraint length 1J of the constituent code. For example, for P1, (£) = 10- 8 and a block length of K = 1000, the 16-state (1J = 4) primitive constituent code (code D) gains 6.5 dB compared with the 2-state (1J = 1) constituent code (code A).

o

Selecting a larger constraint length can be used as a substitute for a larger block length if decoding delay is an issue. For example, the 16-state primitive code (code D) with K = 100 gains 0.5 dB compared with the 4-state code (code B) with K = 1000 for P1, (E) = 10- 7 .

o

The bounds in Figure 16.13 indicate that the 16-state primitive constituent code (code D) performs better than the 16-state nonprimitive code (code E), but the simulations in Figure 16.14 show that this is true only at high SNRs. This result illustrates that the bounds accurately predict performance only at SNRs above the cutoff rate limit, 2 dB in this case. (Note that the simulation

812

Chapter 16

Turbo Coding

DE-"'-

le-02

le-03

@:

~

le-04

---~--- -~----%-

le-05

---

le-06 _

le-07~~~~~~~~~~~~~~~~~~~~~~~~~

0.2

0.4

0.6

0.8

1.2

1.4

1.6

1.8

2

E1,IN0 (dB)

FIGURE 16.14: Bit-error probability simulations for two rate R v = 4 and K = 1024.

= 1/3 PCCCs with

results in Figure 16.14 show the performance of these two codes at a lower SNR range than the bounds in Figure 16.13). 0

The superior performance of the 16-state primitive code (code D) at higher SNRs is due to its larger free distance (see Problem 16.15).

IEX.AMPllE 16.11

Perrformance of Recmsive and Nomewrsive PCCCs

Finally, consider the two rate R = 1/3 PCCCs generated by the nonsystematic feedforward encoder of Example 16.8 and by the systematic feedback encoder of Examples 16.6 and 16.7. As we have noted previously, the constituent encoders by themselves generate identical rate R = 1/2 codes. The approximate bit IOWEF of the nonrecursive PCCC is given in (16.75b ). For large K, the free distance of this code is djree = 7. An approximation to the bit WEF is given in (16.75d). In Figure 16.15 we show the P1, (£) bound obtained by substituting this expression into (16.82), along with the corresponding bound for the recursive PCCC from Examples 16.6 and 16.7, shown previously in Figure 16.12(b). The interleaver size (information block length) is K = 1000. Also plotted in the figure are the P1, (E) bounds for the unterminated rate R = 1/3 convolutional codes obtained by simply repeating the second output of each R = 1/2 code without permuting the input sequence. These latter curves represent the equivalent R = 1/3 performance of the two constituent codes (nonrecursive and recursive) without parallel concatenation.

Section 16.3

Performance Analysis of Turbo Codes

8B

A: Convolutional nonrecursive ~~ B: PCCC nonrecursive ~ ~ ~ C: Convolutional recursive D: PCCC recursive .........

OJ

O

O

O

O

0

0.01 0.001 ~

~

~

0.0001 le-05 le-06 le-07 le-08 le-09 2

3

4

5 E1JN0 (dB)

6

7

8

FIGURE 16.15: Bit-error probability bounds for nonrecursive and recursive PCCCs and convolutional codes with rate R = 1/3.

We now conclude this section with some comments regarding Example 16.11. 0

o

0

0

The Pb (E) curves for the unterminated convolutional codes without parallel concatenation (curves A and C) indicate that the performance of the two codes is almost the same. This is to be expected, since the two encoders generate identical codes, and the only difference in their BERs is a consequence of the different mappings between information sequences and codewords. For the nonrecursive codes parallel concatenation (curves A and B) produces little improvement, because there is no interleaver gain for feedforward encoders. For the recursive codes, on the other hand, parallel concatenation results in substantial coding gain (curves C and D). For example, for Pb (E) = 10-6 , the recursive PCCC with K = 1000 achieves a 4.3-dB coding gain compared with the rate R = 1/3 equivalent code without parallel concatenation. The superiority of employing systematic feedback encoders rather than nonsystematic feedforward encoders as constituent codes in a PCCC can clearly be seen by comparing curves B and D. For example, for P1, (E) = 10-6 , the recursive PCCC gains 3.8 dB compared with the nonrecursive PCCC.

In the next section we continue our analysis of turbo codes by examining the parameters that provide the best constituent code designs.

314

16.4

Chapter 16

Turbo Coding

DESIGN OF TURBO CODES

For Sl'-TRs (values of Eb/ No) above the cutoff rate limit of (16.83), our discussion of the design of PCCCs is based on the upper bound on bit-error probability derived in the previous section. (For smaller SNRs, considered later in this section, the dynamics of iterative decoding have a larger influence on code design than error probability bounds.) We begin by using (16.25a) and (16.26b) to express the bit WEF B(X) of a systematic code as a sum over input weights was follows:

B(X)

=

(16.84)

Then, using the looser form of the bound of (16.82) (i.e., without the function f( ·), a scaling factor that depends only on dtree) and (16.84), we can approximate the bound on the bit-error probability of a \CCC as follows: (16.85) where Wmin is the minimum-weight nonzero valid input sequence, that is, the minimum-weight nonzero input sequence that terminates the encoder. Now, using the approximation for large K of (16.55a), we obtain

(16.86) We now note that for (n, 1, v) systematic constituent encoders, the resulting PCCC has rate K -v 1 R = ~ (for large K). (16.87) (112 1)K 211- 1 We next consider nonrecursive PCCCs with (11, 1, v) systematic feedforward constituent encoders. In this case, Wmin = 1, since only zeros are required to terminate an information sequence for a feedforward encoder. Also, as noted in the previous section, h max = w for feedforward encoders. Finally, since for any (11, L v) systematic feedforward encoder ASw\Z) = [Ai1\Z)]w (see Problem 16.16), we obtain from (16.86) for nonrecursive PCCCs with (11, 1, v) systematic feedforward constituent encoders and large K (16.88) From (16.88) we see that, even in the best case (w = 1), Pb(£) does not decrease with increasing K; that is, no interleaver gain is possible with feedforward encoders.

Section 16.4

Design of Turbo Codes

3'15

A similar applies to PCBCs with (hn, hk) block constituent codes and an interleaver size of K = hk. We now consider recursive PCCCs with (n, 1, v) systematic feedback constituent encoders. In this case, Wmin = 2, since a weight-1 input sequence cannot terminate the but there always exists a weight-2 input sequence, of degree no than 2v - 1, that does terminate the encoder (see Problem 16.17). Also, since each single-error event in a multiple-error event requires at least two input 1's to terminate, hmax = Lw/2J for feedback encoders. We now note that for the term involving the block length K in (16.86) [((2hnw,-w-l)

= J K-2

l

~f W ~S odd K- 1 1f w 1s even.

(16.89)

Thus, for large K. the terms with odd w are negligible compared with the terms with even w, and in (16.86) we need only consider terms of the

Ai:>

Ai~; (Z) = [Ai1\Z) (see Problem 16.16), we must examine the properties of the term Ail) (Z), that

Further. since for any

(11,

1, v) systematic feedback encoder

r

the weight enumerator for single-error events with input weight 2. '\Ne begin by considering an example. IEJ{AM~llE 11BiXl IErrilW©l®rr

Consider the state diagram of the (2, 1, 2) systematic feedback encoder of Examples 16.6 and 16.7 shown in Figure 16.10. In this case, it is clear that the shortest single-error event generated by a weight-2 input follows the state sequence So -> S1 -> S3 -;. S2 -;. So and has parity weight 4. All other single-error events with 2 are generated by adding one or more cycles around the S2 -;. S1 -;. S3 -;. S2 to the foregoing shortest state sequence. Because each of these cycles adds weight 2, =Z

4

4

2 +z 6 +z s +···=--). 1- z-

(16.90)

In general. for any (n, 1, v) systematic feedback encoder, the input sequence 1lll

= 1000. . . will produce a cycle in the state diagram with input weight O that

starts in state S1 = (10 · · · 0), arrives in state S2 ,,-1 = (0 · · · 01) after at most 2" - 2 steps, and then returns to state S1 in one step (see Problem 16.18). Let Zc be the parity weight generated by this cycle. Also, it is easy to show that a 1 input from state S2 ,-1 = (0 · · · 01) terminates the encoder (see Problem 16.19). Further, as in Example 16.12, if each of the 11 - 1 numerator polynomials has the form 1 + .. · + Dv, that is, they are monic polynomials of degree v, then the zero-input branch connecting state S2 ,,-1 = (0 · · · 01) to state S1 = (10 · · · 0) has zero parity weight. fo this case, if z1 is the total parity weight of the two input weight-1 branches that connect the initial and final states to the zero-input weight cycle, the total parity weight associated with any input weight-2 single-error event is jzc + z 1 , where j is the number of zero-input weight cycles included in the error event, and Zmin

= Zc + Zt

(16.91)

816

Chapter 16

Turbo Coding

is the minimum parity weight associated with a weight-2 input. (If the preceding condition on the numerator polynomials is not satisfied, the branch connecting state S2 v-1 = (0 · · · 01) to state S1 = (10 · · · 0) may have nonzero parity weight. In this case, since this branch is not included in the path that achieves minimum parity weight, its parity weight must be subtracted from the right side of (16.91).) Thus the parity weight enumerator for single-error events with input weight 2 is given by A(l\z)

=

zznun

+ z(2Zmin-Zt) + z(3Zmin-2z,) + ... =

2

ZZmin

_. _ .

1 - z<-111111-",l

(16.92)

(Continued)

EXAMPLE 16.12

For the encoder shown in Figure 16.10, Zc

= 2, z 1 = 2, and Zmin = 4.

Using (16.86), we can now state the approximate bit-error probability bound for large K for recursive PCCCs with (n, 1, v) systematic feedback constituent encoders as follows:

~

Pb(E):::::;

2w ( 2w ) K-lw2w [

.L..,

(1:c:w:c:LK/2J)

~

L (l:c:w:c:LK/2J)

W

1-

]2wl

ZZmin z(Z111i11-Z1)

W=Z=e-RE1,/No

2w ( 2w ) K-1

[y<2+2z,,,;11Jr

[l -

W

(16.93)

y(z111i11-z1)]2w Y=e-RE1,/No

We see that, unlike (16.88) for feedforward encoders, (16.93) contains the term. K- 1 , indicating the presence of interleaver gain for feedback encoders. Also, the exponential behavior of the most significant term in the bound is determined by Zmin, the minimum parity weight associated with weight-2 input sequences. Thus, to guarantee good performance for large K, Zmin should be as large as possible. Finally, since Zmin depends primarily on Zc, the parity weight in the zero-input weight cycle should be as large as possible. This implies that the cycle length should be as large as possible, which means that a primitive denominator polynomial should be chosen to achieve a large Zmin (see Problem 16.20). Because the weight 2 + 2zmin is the coefficient of REb/ No in the most significant term of the bound, we define the effective minimum free distance of a rate R = 1/ (2n - 1) PCCC with an (n, 1, v) constituent encoder as deft =

EXAMPLE 16.12

For the rate

R

2 + 2Zmin.

(16.94)

(Continued)

= 1/3 PCCC based on the encoder shown in Figure 16.10, deft= 10.

EXAMPLE 16.13

Calculating deff for PCC(s

Consider the rate R = 1/3 recursive PCCCs generated by the following (2, 1, 3) systematic feedback encoders: (16.95a)

Section 16.4

Design o-f Turbo Codes

gn

and where
[12] For an (n, 1, v)

feedback encoder with gene-

rator matrix
=[

1

n1 (D)/d(D)

n 11 _1(D)/d(D) ] ,

(16.96)

such that the denominator polynomial d(D) is a polynomial of v, and the numerator polynomials n i (D) i- d(D), 1 _:s j _:s n - l, the minimum weight associated with weight-2 sequences satisfies Zmin

= s(2v-l + 2) + (n

- s -1) (2v-l)

:S (n -1)(2v-l

+ 2),

(16.97)

where s, 0 _:s s _:s n - 1, is the number of numerator polynomials that are monic and of degree v.

!Proof, We begin by noting that since d(D) is primitive, it is the generator polynomial of a cyclic (2v -1, 2v - v -1) Hamming code. Also, the minimumdegree binomial that has d(D) as a factor is D 2 ' -l + 1, and the polynomial q(D) = (D 2"-l + 1)/d(D) is the generator polynomial of a (2v - 1, v) maximum-length block code. Next, we consider numerator polynomials n(D) such that deg[n (D)] < v. In this case, the weight-2 input sequence D 2 ' -l + 1 generates the sequence n(D)q(D), which has a weight of exactly 2v-1, since it is a nonzero codeword in a (2v 1, v) maximum-length code. Also, if n (D) has degree v but is not monic, it can be expressed as n(D) = Drv(D), 1 _:s r _:s v. where

318

Chapter 16

Turbo Coding

v(D) is monic and has degree less than v. In this case, the parity sequence = Drv(D)q(D) again has weight 2v-l.

n(D)q(D)

Now, consider numerator polynomials n(D) that are monic and of degree v; that is, n(D) = 1 + · · · + Dv. In this case, we can write

(16.98) where nCll(D)

= nv-l

and nC 2l(D)

= n(D)

- Dv both have degree less than

v. Thus, c(ll(D) ~ and ~ n< 2l(D)q(D) are both nonzero codewords in a (2v -1, v) maximum-length code and have weight 2v-l. We note that c< 1l(D) has the maximum degree of 2v - 2 for any code polynomial and v - lleading zeros; that is, c(ll (D) = nv-l +c~1l Dv + ... +ci~~ 3 n 2"- 3 + n2"-2, and c<2l (D) has a nonzero constant term. We can now write the cyclic shift ,,y(l) (D) of c< 1l (D) as nC 1l(D)q(D)

cC 2l(D)

,,y(l)(D) _ 1 + DV + c(l) nv+l + ... + c(l) n2"-2 -

=

2"-3

V

DcCll(D) + 1 + D 2 "-l.

(16.99)

Because ,,y(ll(D) and c< 2l (D) are both codewords, x< 1) (D) + c< 2l(D) is also a codeword, has a zero constant term, and has weight 2v-l. Now, we can write the parity sequence as n(D)q(D)

= DcCl\D) + = x< 1l(D) +

cC 2\D) c< 2 \D) + 1 + D 2 "-l.

(16.100)

Because ,,yCll (D) + c<2l(D) has weight 2v- 1 , a zero constant term, and degree of at most 2v - 2, the parity sequence n(D)q(D) has a weight of exactly 2v-l + 2. The theorem now follows by defining s as the number of numerator polynomials that are monic and of degree v. Q.E.D. In Problem 16.21 it is shown that the upper bound in (16.97) also holds for nonprimitive feedback polynomials. Thus, since for primitive feedback polynomials Theorem 16.1 guarantees that the upper bound can be achieved by properly selecting the numerator polynomials, we can achieve the best performance for large K, that is, the largest effective free distance deft, by choosing constituent codes with primitive feedback polynomials and monic numerator polynomials of degree v. Selecting constituent codes that result in large values of deft guarantees good high-SNR performance. For lower SNRs, however, codeword multiplicities, additional terms in the weight spectrum, and the dynamics of iterative decoding must be considered. In searching for PCCCs with systematic feedback constituent encoders that offer good performance over the entire SNR range, it is important to maximize Zmin, and thus the effective free distance deft of the PCCC, and to minimize the multiplicities of the minimum-weight codewords produced by each of the lowest-weight input sequences, that is, input weights w = 2, 3, 4, · · ·. We denote the minimum weight of codewords produced by weight-w input sequences in an encoder by dw, and the number of such codewords by Aw,dw, the coefficient

Section 16A TABLE 16.7:

Design of Turbo Codes

~19

Systematic feedback convolutional encoders with optimum-weight

spectrum.

(a) LOW RATE ENCODERS k/Pll 1/4 1/4 1/4 1/4 1/4 1/3 1/3 1/3 1/3 1/3 1/3 1/2 1/2 1/2 1/2 1/2 1/2 1/2

II

1 2 3 4 5 1 2 3 4 5 6 1 2 3 4 5 6 6*

~(3)

ii;(2)

ii;(l)

ii;(qJ)

d/,z, A2,d2

d3, A3,d3

d"', A4,d4

2 6 11 27 71

3 7 17 37 45 2 5 17 33 45 101 -

1 5 15 35 51 3 7 15 37 51 131 2 5 17 37 45 101 115

3 7 13 23 67 3 7 13 23 67 163 3 7 13 23 67 147 147

i!ii,1 ]_l[D,1 20,1 32,1 56,1 5,1 ~,1 14,1 22,1 38,1 70,1 3,1 6,1 8,1 12,1 20,1 36,1 36,1

00

00

00

]_l[D,1 U,l 16,1 23,1

14,2 14,1 li4!,1 20,3

16,1 18,2 18,3 li5,1

-

-

-

-

-

-

-

-

00

00

~,1 ]_l[D,2 12,1 17,2 23,1

10,1 ]_l[D,1 ]_l[D,1 16,6 14,1

00

00

§,1 7,3 8,3 10,2 13,1 13,3

6,1 IOi,l IOi,l 8,1 8,2 10,2

ds, As,d5

00

12,1 14,6 12,1 n,1 n,1 oc 7,1 9,9 10,14 8,2 7,1 9,2

(b) HIGH RATE ENCODERS k/n

II

2/3 2/3 2/3 2/3 2/3 2/3

1 2 3 4 5 6

Jbi(3)

-

-

3/4 1 2 3/4 2 6 3/4 3 17 3/4 4 33 3/4 5 63 Adapted from [34].

Jbi(2)

Jbi(l)

lhl(ll)

dz, A2,d2

d3, A3,d3

2 6 15 27 73 135

2 5 17 35 51 133

3 7 13 23 75 147

2,1 4,3 5,1 8,2 12,2 20,2

3,1 4!,l 5,3 i!ii,3 8,4

2 2 13 31 51

2 7 15 27

3 5 11 25 45

Z,3 3,3 4,1 6,2 9,2

3,3 4!,3 4!,1 §,2

71

00

00

d"', A4,d4 6,4 4,1 5,1 6,9 i!ii,3 IOi,3 5,6 4,7 4!,l 5,8 5,5

ds, As,d5 00

5,1 6,6 7,29 i!ii,5 6,2 00

5,15 5,5 5,4 §,2

of the lowest-order term in the conditional IOWEF Aw(X). (Note that in this case deft= 2d2 - 2.) Table 16.7 lists optimum-weight spectrum (n, k, v) systematic feedback convolutional encoders for rates R = 1/4, 1/3, 1/2, 2/3, and 3/4. The optimization was performed by first maximizing dw and then minimizing Aw,dw, successively, for increasing values of w. For each encoder, we list the rate R = k/n, the constraint length v, the (right-justified) octal generator sequences gCO), gCl), gC2l,

820

Chapter 16

Turbo Coding

and g< 3l in Table 16.7(a) for rates R = 1/4, 1/3, and 1/2 and the (right-justified) octal parity-check sequences 11n<0l, nn< 1l, nnC 2l, and nnC 3l in Table 16.7(b) for rates R = 2/3 and 3/4, and the pair (dw, Aw,dw) for w = 2, 3, 4, and 5. In the table, g
=[

1

g< 1l (D) /g
g<2l (D) /gCO) (D) ]

(l+D 2 +D 3 )/(l+D+D 3 )

(l+D+D 2 +D 3 )/(l+D+D 3 )

].

(16.101) Similarly, in Table 16.7(b), lb.CO) represents the denominator (feedback) polynomial of a systematic feedback encoder parity-check matrix H(D) and .ii 1l, ii2l, and lln< 3l represent the numerator polynomials. For example, for the rate R = 2/3, 8-state (3, 2, 3) encoder, 11n
= [h(Zl (D) /h< 0\D) h(ll (D) /lb.CO\D) 1] = [(D 3 +

D 2 + 1)/(D3 + D + 1)

(D 3 + D 2 + D + l)/(D 3 + D + 1)

1].

(16.102) The bold entries in the table denote the minimum free distance dtree of the code, the symbol oo indicates that there are no codewords corresponding to that input weight (only single-error events are considered), and the * entry indicates the existence of an encoder with the same d2 and a larger dtree· The following remarks pertain to these optimum-weight spectrum encoders. ®

®

Many of the encoders in Table 16.7 do not give optimum dfree codes, as can be seen by comparison with the code lists in Chapter 12; however, the encoders in Table 16.7 are optimum in the sense of minimizing the number of low-weight codewords corresponding to low-weight input sequences. The encoders in Table 16.7 can be used to produce a variety of PCCC code rates, as seen in Table 16.8. Other code rates can be achieved by puncturing, TABLE 16.8: PCCC code rates

for different constituent encoder rates. encoder nnt,e

JP'CCC code rate

1/4 1/3 1/2 2/3

1/7 1/5 1/3 2/4

3/4

3/5

Con§tiitmelll!t

Section 16.4

Design of Turbo Codes

i?b21

using constituent encoders of different rates, or by employing multiple interleavers (see Figure 16.2). o

h is to note that in examining the weight spectrum of PCCCs, the values of interest are given by the lowest-order coefficients in the conditional IOWEFs Aw(X) for small values of w. This contrasts with the case for normal convolutional codes and block codes, in which the low-order coefficients in the standard WEF A(X) are the values of interest. This is because the interleaver in PCCCs has a different effect on input sequences of different weights, thus the importance of the codewords produced low-weight input sequences.

We now summarize the major points to remember in designing good PCCCs for high SNRs: o

The bit multiplicities in a PCCC include a factor of , where K is the interleaver w is the weight of an input sequence that produces a terminated and w ::: w 111 ;11 , the minimum-weight input sequence that produces a terminated codeword.

o

All feedback encoders have w 111 ;11 = 2, resulting in an interleaver gain factor of for the associated PCCCs. In contrast, Wmin = 1 for feedforward encoders and for repeated block codes, so there is no interleaver gain in these cases, although some spectral thinning does occur.

o

For large values of Kand SNRs, the performance of a PCCC is optimized by maximizing its effective free distance deft = 2 + 22 111 ;11 , where Zmin is the minimum parity weight corresponding to a weight-2 input sequence in the constituent encoder. feedback convolutional encoders, it is possible to achieve Zmill = (11 - 1)(2"- 1 + 2) by choosing a primitive denominator polynomial and monic numerator polynomials of degree v.

o For (11, 1, v)

o

For SNRs closer to the cutoff rate limit, good performance is achieved by optimizing the low-input weight terms in the IOWEF of the constituent encoder.

o

These PCCC design rules are based on an average (uniform interleaver) analysis, and performance can vary in a particular case depending on the choice of an interleaver. Thus, it is important to support a code design with appropriate computer simulations.

fo addition, although pseudorandom interleavers such as the ones discussed earlier in this chapter give very good performance on average, it is possible to design the interleaver to improve the value of deft and thus the high-SNR performance of the code. A codeword of weight deft is produced when a weight-2 input sequence that generates the weight-Zmin parity sequence in the first constituent code is interleaved in such a way that the interleaved input sequence maintains the same spacing between l's, thus also generating the weight-2 111; 11 parity sequence in the

822

Chapter 16

Turbo Coding

second constituent code. If we refer to a weight-2 input sequence that generates the weight-Zmin parity sequence in the first constituent code as a "bad" input sequence, interleaver designs that "break up" all bad input sequences, that is, they change the spacing between l's, result in a larger value of def!; however, putting too much structure in the interleaver can result in low-weight codewords owing to higher-weight input sequences; that is, the ability of a pseudorandom interleaver to reduce the influence of higher-weight input sequences is mitigated when structure is added. Interleaver designs that attempt to improve deff by breaking up the bad input sequences while also maintaining most of the characteristics of pseudorandom interleavers are referred to as semirandom ( or S-random) interleavers [35]. For example, in an S-random interleaver, integers i in the range 1 :s i :S K are chosen randomly (without replacement) to denote the interleaved positions of successive bits in the input sequence. If a selected integer is within a distance ±S of any of the S previous selections, it is rejected and another integer is selected. Choosing S equal to the cycle length of the denominator polynomial guarantees that all bad weight-2 input sequences are broken up; however, the algorithm is not guaranteed to successfully assign all K integers, so that the final permutation may have to be altered further to satisfy the spreading condition. The preceding rules for designing good PCCCs implicitly assume the use of (optimum) maximum likelihood decoding. Iterative decoding, in contrast, is suboptimum, and different code design rules apply in this case. At moderate-to-high SNRs, iterative decoding is essentially maximum likelihood, and the preceding design rules will give the best codes. At low SNRs, below the channel cutoff rate, however, the dynamics of iterative decoding are more important in determining the performance of PCCCs than the weight spectrum of the code. To explain the dynamics of iterative decoding, it is helpful to introduce the extrinsic information transfer chart, or EXIT chart [36], based on the techniques of density evolution [37, 38] and threshold analysis [39]. These techniques relate a parameter of the input to a constituent decoder, either the signal-to-noise ratio of the a priori L-value La(uz) or the mutual information between an information bit uz and its a priori L-value La (uz), to a parameter of the decoder output, the signal-to-noise ratio of the a posteriori extrinsic L-value Le(uz) or the mutual information between uz and its a posteriori extrinsic L-value Le(uz), respectively. Following the mutualinformation approach, we first model the a priori L-value inputs to a constituent decoder as independent Gaussian random variables with variance and mean f.la = /2, where the sign of f.la depends on the transmitted value of uz. This Gaussian approximation is based on two facts: (1) the input channel L-values to a constituent decoder (the terms Lcr1(0) in (16.17)) are independent Gaussian random variables with variance 2Lc and mean ±Le (see Problem 16.23), and (2) extensive simulations of the a posteriori extrinsic L-values Le(uz) for a constituent decoder with very large block lengths support this assumption [40]. Thus, we can express the mutual information Ia [uz; La (uz)] between an information bit uz and its a priori L-value La(uz) as (see [41])

a;

±a;

I u .L u a[ /,

- -1

a( 1)]- 2

l+oopLJ~I u

L -

ui=-1,+1

00

1)

Io

g2 p

La

(~lu

2PLa ("', lu l )

l

= -1) +

P

La

(~lu

d

l

= +1) ~' (16.103a)

Section 16.4

where the conditional probability

Design of Turbo Codes

function PL,,(~ lu1) of La (u1) is

~235

by (16.103b)

Next, histograms of the conditional pdf's PL,(~ lu1 = -1) and (~ lu1 = ciated with the a posteriori extrinsic L-values (u1) are determined by the BCJR with independent Gaussian-distributed a priori L-value for a particular constituent code and a large block length I(. "We then determine the mutual information Ie[in; Le(u1)] between the information bit u1 and its a extrinsic L-value Le(u1) using (16.103a). We can now generate a decoder transfer curve by a series of simulations of the BCJR algorithm for one constituent code and different values of la[u1; La(i!f)] (completely determined by a}), for a fixed channel signal-tonoise ratio Eb/No and plotting the resulting values of Ie[u1; Le(u1)]. Note that each point on the curve is obtained by simulating the BCJR once for a constituent code and does not require iterative decoding. A set of transfer curves is obtained by repeating this procedure for different values of the channel SNR. E1, / No. An example is shown in Figure 16.16 for the original turbo code of Figure 16.l(b ), with alternate parity bits punctured to produce a code rate of R = 1/2, and various values of the channel SNR. Note that a value of le, = 0 corresponds to the first iteration of decoding when all a priori L-values are to Oand that more values of la correspond to more reliable a priori estimates. A observation is that if a transfer curve cuts below the straight line representing l 0 = this corresponds

0.9 h Q)

u

0

u

-

0.8

Q)

u

0

E1,IN0 E 1,IN0 E 1,IN0 E1,IN0

= = = =

0.25 dB~~ 0.50 dB-~ 0.75 d B ~ 1.00 d B ~ y=x-~

0.7

::,

P.,

;

0

0.6

~

--i 0.5 i::

.g

0.4

8"'h £ 0.3 ,5

;"'

::,

~

Mutual Information I" at input of decoder

FIGURE 16.16:

Decoder input-output transfer curves.

824

Chapter 16

Turbo Coding

to a situation in which an iteration results in an le at the decoder output below the la at the decoder input; that is, the extrinsic estimates are getting less reliable. In this case iterative decoding fails to converge to a reliable solution, resulting in a high probability of decoding errors. In contrast, if the transfer curve stays above the Ia = le line, the extrinsic estimates are getting more reliable, and iterative decoding converges to a reliable solution. We now define the SNR threshold of iterative decoding for a particular constituent code as the smallest value of the channel SNR, Eb/ No, for which the transfer curve stays above the Ia = le line. For example, in Figure 16.16 we see that the iterative decoding threshold for the original rate R = 1/2 turbo code is approximately 0.5 dB. This threshold corresponds to the value of the channel SNR, Eb/ No, at which the bit-error probability curve of a PCCC begins its characteristic sharp drop, referred to as the waterfall region of the BER curve (see Figure 16.3). An EXIT chart for a particular channel SNR Eb/ No can be formed by plotting a transfer curve and its mirror image (about the line Ia = le) on the same set of axes, as shown in Figure 16.17(a) for Eb/ No = 0.75 dB. The two curves can be interpreted as representing the input-output transfer characteristics of the two constituent decoders in an iterative turbo decoder. The chart can then be used to trace the trajectory of iterative decoding as follows. For a given Eb/ No, assume initially that la = 0, corresponding to the first iteration of decoder 1, and determine the resulting le (vertically) using the transfer curve for decoder 1. Because the a posteriori extrinsic L-value of decoder 1 becomes the a priori L-value of decoder 2, the value of le from decoder 1 becomes Ia for the first iteration of decoder 2, and the resulting le for decoder 2 is determined (horizontally) using the transfer curve for decoder 2. This procedure is repeated and traces the "staircase" trajectory of iterative decoding, as shown in Figure 16.17(a). If a tunnel exists between the two transfer curves, as shown in Figure 16.l 7(a) for Eb/ No = 0.75 dB, iterative decoding converges as le approaches a value of 1. As the channel SNR is lowered, the two transfer curves come closer together, and the smallest SNR for which the two curves do not touch, that is, for which a tunnel exists, is the SNR threshold. If no tunnel exists, that is, the two transfer curves cross each other, as shown in Figure 16.17(b) for Eb/ No = 0.25 dB, a decoder fixed point exists, le becomes stuck at a value less than 1, and iterative decoding does not converge. The EXIT chart is particularly useful for visualizing the dynamics of iterative decoding. For example, we see from Figures 16.16 and 16.17 that as the channel SNR is lowered the tunnel narrows, and more iterations are needed before the mutual information becomes large enough for the decoder to converge to a reliable solution. We also note that the EXIT chart technique can be used to determine the threshold of a turbo code with asymmetric constituent codes simply by plotting the transfer curves of the two (different) constituent codes, which will no longer be the mirror image of each other. The foregoing discussion indicates that determining the transfer curves and EXIT charts of various constituent codes is a useful tool in designing PCCCs with good low-SNR performance, that is, good performance in the waterfall region of the BER curve. The design rules to follow in a particular case, that is, the high-SNR uniform interleaver analysis or the low-SNR threshold analysis, depend on the application. In communication systems designed to achieve very low bit

Section 16.4

E rl H

Design of Turbo Codes

1325

0.9


"Cl 0 (.)


"Cl

E 1,IN0 = 0.75 dB

0.8

"""0

0.7 E 0.

~

g

E

Decoder 1

0.6

fr

-" 0

0.5

Cl

~ -..."

0.4

C:

.g s Cl

0.3

H

c8 0.2 -;;; " 0.1 E

,5

::;

0

0

0.1

0.2 0.3 0.5 0.6 0.9 0.4 0.7 0.8 Mutual Information/,,(/,) at input (output) of decoder 1(2)

(a)

E rl

H

0.9


"Cl 0 (.)


"Cl

0.8

4-,

0 ~

E 0.

E,

E 0. E 0

0.7 0.6 0.5

""

~ 0.4 ~ C:

.g §"'

0.3

2 0.2 ,5 -;;;

"

E

0.1

~

0

0

0.1

0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Mutual Information/,,(/,) at input ( output) of decoder 1(2)

(b)

FIGURE 16.17: Extrinsic information transfer (EXIT) charts for (a) Eb/ No = 0.75 dB and (b) Eb/No= 0.25 dB.

826

Chapter 16

Turbo Coding

error probabilities at moderate SNRs, the uniform interleaver analysis is preferred, whereas in systems designed to achieve moderate BERs at very low SNRs close to capacity, the threshold analysis is best. The usefulness of the SNR threshold analysis discussed here depends on two factors: the accuracies of the Gaussian approximation for the a priori L-values, and the simulated estimates of the conditional pdf's for Le. Both these factors depend on the length of the input block K used in the simulations. Experience with threshold analysis has shown that, in general, the constituent codes with the lowest SNR thresholds have denominator polynomials with short cycle lengths, in contrast with the long cycle lengths needed to achieve large values of deff- (See, for example, the list of SNR thresholds given in [39] for 8- and 16-state constituent codes.) Asymmetric code designs for PCCCs attempt to exploit these dual requirements by choosing different constituent codes, one with a short cycle length and the other with a long cycle length [30]. 16.5

!TIERATIVIE DECODING Of TURBO CODES

A brief summary of turbo decoding was given in Section 16.1. As noted there, the best performance is achieved when the individual SISO decoders make use of the MAP, or APP, decoding algorithm. The details of this algorithm were presented earlier, both as an optimum bit-by-bit decoding algorithm for convolutional codes in Chapter 12 and as an optimum method for trellis-based decoding of block codes in Chapter 14. Thus, in this section, we emphasize the use of this algorithm as an iterative decoder of PCCs. Other SISO decoders that are simpler to implement, such as the SOVA or Max-log-MAP algorithms, also can be used to iteratively decode PCCs, but their performance is not as good. The basic idea behind iterative decoding is to divide the decoding of a PCC, for which optimum decoding would be prohibitively complex, into a suboptimum iterative process involving only the relatively simple APP decoders for the constituent codes. After each iteration, the extrinsic L-values, representing reliability information about the bits to be decoded, are passed from one decoder to the other, ensuring that very little information is lost relative to optimal decoding. The basic structure of an iterative turbo decoder was shown in Figure 16.6. In the first iteration of decoder 1, the inputs are the channel L-values Lcr1(0) and Lcr?) for each received symbol (information and parity) of constituent code 1 and the a priori L-values La (u1) of each information bit, which all equal O for equally likely inputs. In the first iteration of decoder 2, the inputs are the channel L-values Lcr/0l and Lcr?) for each received symbol (information and parity) of constituent code 2 and the extrinsic a posteriori L-values Li1l (u1) of each information bit received from the output of decoder 1. These extrinsic L-values, along with the L-values of the received information symbols, must be interleaved according to the same pattern used at the encoder before entering decoder 2. In all subsequent iterations of decoder 1, the a priori L-values Li1l (u1) are replaced by the extrinsic a posteriori L-values d 2l (u 1) received, after proper deinterleaving, from the output of decoder 2, and all subsequent iterations of decoder 2 are identical to the first iteration, except that the a priori inputs will be different. Finally, after a sufficient number of iterations, the decoded output can be derived from the a posteriori L-values of the information

Section 16.5

8521

Iterative Decoding of Turbo Codes

bits L C2\u1 ), properly deinterleaved, at the output of decoder 2. Positive L-values are decoded as +1 and negative L-values as -1. We now present a simple example, using the log-MAP algorithm first presented in 12, to illustrate the principle of iterative decoding.

Consider the PCCC formed by using the 2-state (2, l. 1) SRCC with matrix
(16.104)

as the constituent code. A block diagram of the encoder is shown in Figure 16.18(a). Also consider an input sequence of length K = 4, including one termination along with a 2 x 2 block (row-column) interleaver, resulting in a (12. 3) PCCC with overall rate R = (Recall that the termination bit is not an information bit.) The length K = 4 decoding trellis for the constituent code is shown in Figure 16.18(b ), where the branches are labeled using the mapping 0 ---+ -1 and 1 ---+ +1. The input block is given by the vector ill = [uo, iq, u2, u 3], the interleaved block is ill1 = [uo, u~, u;, u~] = [uo, u2, u1, u3], the parity vector for the first constituent code is given by JPl(ll = [P~l), p?). 1). Pill and

J,

pi

the parity vector for the second constituent code is JPl( 2l

[p6

2 ), p?),

=

Pi

2

),

1

2

p~

We can the 12 transmitted bits in a rectangular array, as shown in Figure 16.19(a), where the input vector un determines the parity vector JPl(l) in the first two rows, and the interleaved input vector ill' determines the parity vector JPl( 2l

____ __ _ ,, ~ - - - - - - - - - - , ! > - 'l'(l)

~Gt~_________ :

"9!>-

.,,121

(a)

(b)

(a) A 2-state turbo encoder and (b) the decoding trellis for the (2, 1, 1) constituent code with K = 4.

FIGURE 16.18:

828

Chapter 16

Turbo Coding

Uo

U1

Po(l)

Pi 11

-1

+1

-1

+1

+0.8 +1.0 +0.1 -0.5

Uz

U3

PP) PP)

-1

+l

+1

-1

-1.8 +1.6 +1.1 -1.6

p?> PP 1

-1

+1

-1.2 +0.2

PP) pp>

-1

-1

+1.2 -1.1

(12,3) PCCC (a)

Coded values

Received L-values

(b)

(c)

-0.32 -0.38

-0.88 -0.69

-0.40 -0.07

+0.77 +0.47

+0.23 -0.04

-0.80 +2.03

Extrinsic L-values after first row decoding

Extrinsic L-values after first column decoding

(d)

(e)

Soft-output L-values after the first row and column decoding (f)

-0.01 -0.01

-0.98 -0.81

-0.19 +0.18

+0.43 +0.77

+0.07 -0.21

-1.30 +2.16

Extrinsic L-values after second row decoding

Extrinsic L-values after second column decoding

Soft-output L-values after the second row and column decoding

(g)

(h)

(i)

-0.9 -0.9

-0.8 -0.8

-0.9 -0.7

+1.4 -0.3

+1.1 +0.1

+0.7 +1.4

Approximate extrinsic L-values after first row decoding

U)

Approximate extrinsic L-values after first column decoding (k)

Approximate soft-output L-values after the first row and column decoding (l)

-0.1 -0.1

-0.9 -0.7

-0.2 +0.2

+0.6 +0.5

+0.3 -0.5

-0.9 + 1.6

Approximate extrinsic L-values after second row decoding (m)

Approximate extrinsic Approximate soft-output L-values L-values after after the second row and second column decoding column decoding (n) (o)

FIGURE 16.19: Iterative decoding example for a (12, 3) PCCC.

Section 16.5

0

L c rCil 1

0

0 '1

1

=it(_) \No , ,

1 ° 2

===---======-=~Rtheof

Iterative Decc -- ·

2

e

z&S-_..::-5

in the first two columns. For purposes of values shown in Figure 16.19(b ). We also assume a (-6.02 dB), so that the received channel L-values vector rr· = [r(O\(ll,.<2) i-<0Ull,J2) IJO),.(ll,J2) ,JO) 1 -

iTurb

---- '; ' =---;;;;. 5

=====ling t<

-

-=:_ven b

2 "3

-c

/il 1

= /il 1 .

l = 0 , 1.2. 3 ,

2.

::::1 chai

-=

for purposes of a set of ~ log-' in 16.19(c). in Fi: In the first iteration of decoder 1 ~ fou1 is applied to the trellis of the 2-state (2. L 1) code = to pa the L--values LC 1\u 1) for each ~ ~ /.::: -;. ==~;:::======-·coder L-values ~ ::::-~ :/ ,,..-. __ ~ivecl i - - ~-=-' ~1aluei ==eriori V ely i -

---=-=--

.. - ..

Before returning to 16.14, YNe a posteriori L-values L(u1) and the extrinsic a (2.1. 1) constituent code in 16.18(b) notation, we denote the transmitted vector z_ q.;. 1"1 = (u1, p1), l = 0, 1. 2, 3. u1 is an and p1 i ~ --= received vector is denoted as rr = (rro, rr 1 , rr 2 . vvhere ~ to the trnnsrr::> --corncsr1m1d1mr, to the transmitted (see (12.106) and 11::::,

e L-va

---

= (r,u ~d inI

---~

bi

. P (u1 = +1 I ff) L( u 1) = I n - - - - - P (u1 =

J :J

= ln

L('.I

- Le.,

-11

ff)

1

l 1:+ p(s .s,

.S E

I

c

-=

·J "- p(s', s. , ....

.I ,S

E "-/

s rep state where s' represents a state at time l ( denoted s' E CF/ ~ l +1 s E 0-,+1), and the sums are over a l ~ - · - - - - ~ - - - = : 1 . -. in tl :::if the u1 = +1 or -1, respectively. that every b r a n ~ ~

----------

state at time l to a state at time l + 1 is included in one -~ . can write the joint probabilities p(s'. s. ff) in (16.106) ac'- - ~

- - - - ~ - ~ - - l(s)

p

(s', s, n·)

= e0!/(s')+yf'(s'.s)+f3i~

,1tec\ fc ~ e deco 2For iterative decoding. extrinsic a posteriori L-values are comp~ a prior bits as well as information bits. (Note that for feedback encoders, th _;.-of the terminating bits.) Before the first iteration of decoding. the e bits are assumed to be zero.

~r 16

Turbo Coding £"

, Y/' (s', s), and f3t'+i (s) are the familiar log-domain a's, y's, and ,B's ~ _:__ orithm (see (12.129)). (Recall that the three tenus in (16.107), ex;' (sr ~ " ' > .., ~1 (s) represent the influence of the received sequence before tirr"'L- ··· ~ · ,B1 ,ct after time l on the estimate of the (nput bit at time!, respectively.) mtinuous-output A WGN channel with an SNR of Es/ No, f o l l o w i n ~ .128c), and (12.128e), we can write the MAP decoding equations as

(16.108a)

metric:

a;~ 1 (s) =

1

max;,E 0 Jy/(s , s)

+ cxz'(s')].

l = 0, 1, 2, 3,

(16.108b)

rd metric: f3;'(s1)

= max~·Eai+i (y/'(s', s) + f3;~ 1 (s)],

(16.108c)

' function is defined in (12.127), and the initial conditions are = 0, and a 0(S1) = {34'(51) = -oo. (Note that for iterative decoding, i.107a)) is used to compute the branch metrics corresponding to 1s well as information bits.) Further simplifying the branch metric,

1

·, s) =

=

u1La(u1) 2 U/

2

Le

+2

(itiru1

[La(u1) +Lerud+

+ p1rp1) Pl

2

(16.109) Lcrp/, l = 0, 1, 2, 3.

serve from Figure 16.18(b) that to determine the a postenon 1ere is only one term in each of the sums in (16.106), because at tly one +1 transition and one -1 transition in the trellis diagram. ss the a posteriori L-value of uo as = So,

s=

+ y0(s'

=

l- y0(s' =

S1, ir)

- ln p (s' = So, s = So, ir)

So, s = S1) + .81(S1)] So, s =So)+ /31(So)]

uo) + Lcruo] + !Lcrpo + f3i'(S1)} to)+ Lcruo] - !Lcrpo + /3i'(S0)} o) + L,ruo]} - {-HLa(uo) + Lcruo]} +

!

+ f3l' (S1) + L,rpo - /31(So)} = Lcr110 + L" (uo) + Le (uo). (16.110a)

Section 16.5

Iterative

of Turbo Codes

81:?tu

where (110)

=

(16.110b)

the

L-value of 110. The final form of of the a L-value of uo

fJ~HvOCH0

o

the received channel L-value corresn,on,oJm to bit 110. which was the decoder

o

L-value of uo. which vvas also part of the decocleT for the first iteration of decoder 1. this term equals the extrinsic a L-value of uo received from the of the other decoder. the first iteration of decoder 1, (uo) = 0 for

o

Le(uo): the extrinsic of the a L-value of uo. which does not ,,o or (uo). This term is then sent to the other decoder as its a

of

We now proceed in a similar manner to the a posteriori L-value of bit u 1 . We see from that in this case there are two terms in each of the sums in (16.106)., because at this time there are tvvo +1 and two -1 transitions in the trellis Thus. L(zq) = ln

[P (s'

= So.

s = S1. rr·) +

p

In [P (s' = So. s =So.re)+ p = max* {[ai'(So) + y{"

=

[a;'(S1) + y{"

= So. s = S1) + ,82(S1)].

t{[at"(00) , (' + y1.,.

= max* {(+!

= S1.s = S1.ir)]

S1. s =So)+ ,B;{(So)]}

- ma:;C

[a;'(S1)+

= S1.s = So.u·)]-

I

·(s =

so. s =

1

S) o

+

(So)].

2

(s =S1.s=S1)+,8 (St)]} (1q) +

Lcr,,i] +

i

pl+

(So)+ ,82(S1)).

(+i(L"(zq) + LJ,,t] - iLcrp1 + a 1(Si) + ,82'(So))} -max* { (-i[L"(11J) + L,r,,1] - !L,rp1 + a 1(So) + ,82'(So)), (-i(La(i1t) + L,r,i1] + iLJpl + af (S1) + ,8:z'(St))}

= {+HL"(1q)+L,r111l}-{-HL"(1q)+Lcr,,1]} + max* { [+! Lcrp1 + a 1'(So) + ,82'(S1) ]. [-!

pl

+

(Si)+ ,82(So)]}

-max* { [-!L,rpl + a 1 '(So) + ,82'(So)]. [+!L,rpl + a 1(S1) + ,82(S1)]} 11 1 +

(zq)

+

(iq).

(16.llla)

832

Chapter 16

Turbo Coding

where Le(u1) = max* { [+iLcrpl + a 1'(So) + /3:z,'(S1)], [-!Lcrp1 + a 1'(S1) + ,82(So)]} -max* { [-!Lcrpl + a!(So) + ,82(So)], [+iLcrp1 + a!(S1) + ,82(S1)]} (16.lllb) and we have made use of the identity (see Problem 16.24) max*(w + x, w + y) = w + max*(x, y). Continuing, we can use the same procedure to compute the a posteriori £-values of bits u2 and u3 as (16.112a) where Le (u2) = max* {[+i Lcrp2 + a 2(So) + ,83'(S1) ], [-! Lcrp2 + a 2(S1) + ,83(So)]}

- max* {[-! Lcrp2 + a 2(So) + ,83(So) ], [+! Lcr p2 + a 2(S1) + ,83(S1)]} , (16.112b) and (16.113a) where L (u3) =

[-i LJp3 + a3'(S1) + ,84(So)] -

[-i Lcr p3 + a3(So) + ,84(So)]

= a 3(S1) - a 3 (So).

(16.113b)

(Note that in this example, the value of the received parity symbol r p3 does not affect L (u3), since, for the trellis of Figure 16.18(b), p3 = 0 for both branches at time l = 3, and thus r p3 conveys no information useful for decoding.) We now need expressions for the terms a 1(So), a!(S1), a 2(So), a 2(S1), a 3 (So), a 3 (S1), ,B~'(So), ,B{(S1), ,82(So), ,82(S1). ,83(So), and ,83'(S1) that are used to calculate the extrinsic a posteriori £-values Le(u1), l = 0, 1, 2, 3. Using (16.108) and (16.109) and Figure 16.18(b) and adopting the shorthand notation Lut = Lcr,t1 + La (u1) and L pl Lcrpt, l = 0, 1, 2, 3, for intrinsic information bit L-values and parity bit £-values, respectively, we obtain (see Problem 16.25)

=

a 1(So)=-! (Luo+ Lpo)

(16.114a)

(S1) =-!(Luo+ Lpo)

(16.114b)

a~'

a 2(So) = max* {[-! (L,t1 + L pl)+ a 1(So) ]. [+!(Lui - L pl)+ a~'(S1)]}

i

(16.114c)

a 2(S1) = max* {[+ (Lu1 + L pl) + a~' (So)], [-! (Lui - L pl) + a! (S1)]}

(16.114d)

a 3(So)= max* { [-!(L,,2 + Lp2) + a 2(So)], [+!(Lu2 - Lp2) + a 2(S1)]}

(16.114e)

a 3(S1) = max* {[+!(L,,2 + Lr2) + a 2(So)], [-!(L"2 - Lp2) + a 2(S1)]}

(16.114f)

Section 16.5

(So) = -21 (Si) (So)

(3 2*

_

I

Iterative

of Turbo Codes

ia3l3l

+

1

-T:z == max;:~

- Lp3)

fIi

1

{ [+!

= max

1

l+-2

+ /3I(S1)]}

_L

I

+ ,83' (s1)] lj

- ) + /3*3 (,;;:uO )] • [-21

-

,6 i_,, (S·1) = ma;:··., {-ll + 2i

-

+

-1-

2

Lp2

+

+

- Lp1)

+ (32'(So)]. [-i(L,li

(So)].

2

-:-

+ ,82

- Lp1)

+ (32(Si)]}.

(p1)

= 0 for an l, since for a likely.

shorthand notation for the intrinsic L-values of information bits L" 1 and bits is consistent. we can write the extrinsic a L-values in terms of and as Le (uo) = Lpo (u 1) =

+ N(S1) -

f3i'(So).

(16.115a)

1! [+!- L pl + oti'(So) + ,82'(Si) ]. [-!- Lp1 + oti'(S1) + (32(So)] l§ max' { [-i L pl + oti'(So) + (3 2'(So) ], [+i L pl + (S1) + (32'(S1)]} max*

=ma2c{[+i

+

3

(So)+(3 '(S1)],[-!

+

3

+,6 '(So)J}-

+ ot2(So) + ,83'(So)], [+i and (1.13) =

(S1) -

(So).

(16.115d)

We note that in the for the extrinsic L-values (u1), the intrinsic L-value does not appear in any of the terms; that is, the extrinsic L-value of bit u1 does not directly on either the received or a priori L-values of u1.

VVe can now evaluate the extrinsic a L-values using (16.114) and (16.115) and the received channel L-values in Figure 16.19(c). Beginning with the first iteration of decoder 1 for the bits uo, u 1, u2, and u3, and recalling that the initial a L-values in the for L"o, Lu1, L 11 2, and L 11 3 are equal to 0, ·we obtain from (16.114) (So)= -!(0.8 oti'(S1)

+ 0.1) =

= +~(0.8 +

-0.45

(16.116a)

0.1) = +0.45

(16.116b)

!B4

Chapter 16 GYz(So)

Turbo Coding

= max* {[-!(1.0 - 0.5) - 0.45], [+! (1.0 + 0.5) + 0.45]} = max* (-0.70, +1.20} = 1.34

a 2(S1)

= max* {[+ i (1.0 - 0.5)

(16.116c)

0.45], [-! (1.0 + 0.5) + 0.45]}

= max* {-0.20, -0.30} = 0.44

(16.116d)

a 3(So) = max* {[-!(-L8 + Ll) + 1.34], [+!(-1.8 - Ll) + 0.44]}

= max* (1.69, -LOl} = L76

(16.116e)

a 3'(S1) = max* {[+!(-L8 + Ll) + L34], [-! (- L8 - Ll) + 0.44]}

= max* (0.99, L89} = 2.23

(16.116£)

}'33(So)

= -!(1.6 - L6) = 0

(16.116g)

}'33'(Si)

= +i(L6 + L6) = 1.60

(16.116h)

t3z(So)

= max* {[-!(-1.8 + 1.1) + O], [+!(-L8 + Ll) + L60]} = max* (0.35, 1.25} = L59

(16.116i)

2

t3 (S1) = max* { [+!(-L8 - Ll) + O], [-!(-L8 - Ll) + L60]}

= max* (-1.45, 3.05} = 3.06 t3t(So)

(16.116.i)

= max* { [-!(LO - 0.5) + L59], [+i(LO - 0.5) + 3.06]} = max* {L34, 3.31} = 3.44

(16.116k)

t3t(S1) = max* {[+!(LO+ 0.5) + L59], [-!(1.0 + 0.5) + 3.06]}

= max* {2.34, 2.31}

= 3.02.

(16.1161)

Then, from (16.115) we obtain

Li1\uo) = 0.1 + 3.02 L~1l (u 1)

3.44

= -0.32,

(16.117a)

= max* {[-0.25 - 0.45 + 3.06], [0.25 + 0.45 + L59]} - max* {[0.25 - 0.45 + L59]. [-0.25 + 0.45 + 3.06]}

= max* {2.36, 2.29} - max* {1.39, 3.26} = 3.02 - 3.40 = -0.38, (16.117b) and, using similar calculations, we have Li1l (u2) = +0.77 and d l (u3) = +0.47. These extrinsic values after the first iteration of row decoding are listed in Figure 16.19(d). Now, using these extrinsic a posteriori L-values as the a priori 1 inputs for decoder 2 (column decoding), so that Lui = L,r,i1 + L;, ) (u1), we obtain from (16.114) 1

a 1*(,.,, 00J

=-

11t\()

",.,:"'\

2 ~u.o - u ..JL -

.... I"\, 1.LJ

= O"" ..Ju

(1/; 11Qn\

\J..V,..L_U.JUJ

Section 16.5 ot

1'(S1) = (So)

+~(0.8 - 0.32 -

Iterative Decoding of Turbo Codes

(16.118b)

= -0.36

= max* {[- i(-1.8 + 0.77 -I- 1.2) + 0.36]. [+!(-1.8 + 0.77 - 1.2)

(So)

= max* {[-

= 1.31

(16.118d)

i(1.0 - 0.38 -I- 0.2) +

= max* {0.03, (S1)

i

+ 0.36]. [- (-1.8 + 0.77 - 1.2) - 0.36]}

=max* {[+i(-1.8+0.77-1= max* {0.445,

[-1-i(l.0 - 0.38 - 0.2) -I- 1.31]} (16.118e)

= 1.72

= max* {[+i(l.0 - 0.38 +

+

[-i(l.O - 0.38 - 0.2) + 1.31]} (16.118f)

= max* {0.85. 1.1} = 1.68 f3~'(So) = --1-1 .)

-I- 0.47 - 1.1) = -0.485

(16.118g)

f3:J'(S1) = +~(1.6 -I- 0.47 -I- 1.1) = 1.585

f32'(So) = max* {[-!

0.36]} (16.118c)

= max* {0.275, -1.475) = 0.4i! (S1)

!BS

(16.118h)

- 0.38 + 0.2) - 0.485]. [+i(l.0 - 0.38 + 0.2) + 1.585]}

= max* (-0.895. 1.995) = 2.05

(16.118i)

f3 2(S1) = max* {[+i(l.0 - 0.38 - 0.2) - 0.485], [-i(l.0- 0.38 - 0.2) -I- 1.585]} = max* {-0.275. 1.375} = 1.55

(16.118j) [+i(-1.8 -I- 0.77 + 1.2) + 1.55]}

{[-!(-1.8 + 0.77 + 1.2) +

(16.118k)

{1.965, 1.635) = 2.51

f3 1'(S1) =

i

{[+ (-1.8 + 0.77 - 1.2) -I- 2.05],

= max* {0.935,

[-,!: (-1.8 -I- 0.77 -

1.2) -I- 1.55]}

= 2.83,

(16.1181)

vvhere we note that the roles of u 1 and u2 in are reversed compared with (16.116), because, after interleaving, v~ = u2 and u; = v1. We now use (16.115) to for the bits uo, u2, u1, and u3, (uo)

Lf) (u2)

= -1.2 + 2.83 - 2.51 = -0.88

(16.119a)

= max* {[0.6 + 0.36 + 1.55], [-0.6 - 0.36 + 2.05]} - max* {[-0.6 + 0.36 + 2.05], {2.SL 1.09} -

- 0.36 -I- 1.55]}

{1.81, 1.79) = 2.72 - 2.49 = +0.23,

(16.119b)

and, using similar calculations, we have Lfl (u 1 ) = -0.69 and d 2l (u3) = -0.04. (Again note that in using (16.115) to evaluate the expressions for d 2) (uz), the roles of iq and u2 are reversed.) These extrinsic values after the first iteration of column

836

Chapter 16

Turbo Coding

decoding are listed in Figure 16.19( e ). Finally, the a posteriori L-values of the four information bits after the first complete iteration of decoding are given by L <2\uo) = Lcruo

0.8 - 0.32 - 0.88 = -0.40

L<2\u2) =

-1.8 + 0.77

2

L< \u1) =

L( 2l(u3) =

+ Lfl (uo) + Lfl (uo) = Lcru2 + Li,2\u2) + L~2\u2) = Lcru1 + Lfl(u1) + Li2\u1) = Lcru3 + Li,2\u3) + Lf\u3) =

+ 0.23 =

(16.120a)

-0.80 (16.120b)

1.0 - 0.38 - 0.69 = -0.07

(16.120c)

1.6 + 0.47 - 0.04 = 2.03

(16.120d)

and shown in Figure 16.19(f), where the a priori L-values for decoder 2, L~2 ) (u 1), are equal to the extrinsic a posteriori L-values, L~1\u1), l = 0, 1, 2, 3, after the first iteration of decoder 1. Thus, if a final decoding decision was made after just one iteration, the three information bits would be decoded as

uo=-1.

u2=-l,

(16.121)

ii1=-l,

and bit u 1 would be incorrectly decoded in this case. A second iteration of decoding would use the extrinsic a posteriori L-values 1 Lf\u1) from the first iteration of decoder 2 as a priori L-values \u1 ). l = 0, 1, 2, 3, for decoder 1, thus producing a new set of extrinsic a posteriori Lvalues d 1l (u1) at the output of decoder 1. These would then be used as a priori inputs to decoder 2, and so on. In Figure 16.19(g), (h), and (i), we give the Lvalues dl)(u1), Lf\u1), and LC 2\u1), respectively, l = 0, 1, 2, 3, for the second iteration of decoding (see Problem 16.26). Note that in this case, if a final decoding decision was made after two iterations of decoding, all information bits would be correctly decoded.

d

Note that for iterative decoding, although u 3 is not an information bit and does not need to be decoded, its extrinsic L-values must be updated throughout decoding. Also, for this example, all input sequences that terminate encoder 1, when interleaved, also terminate encoder 2. Thus, the operation of the log-MAP algorithm is exactly the same for both decoders. In general, however, the interleaved input sequence will not terminate encoder 2. In this case, the log-MAP algorithm must specify initial values for the /3*'s at each state; that is, /3~ (Si), i = 0, 1. · · · , 2v - 1, must be specified. These can be set either to all zeros or to the final values of the a*'s at time K. The most difficult calculations in the preceding example are the evaluations of the exponential terms needed in the computation of the max* operation. These calculations can be simplified by using the Max-log-MAP algorithm discussed previously in Chapter 12. We now use this simplified algorithm to revisit the above example. EXAM PlE 16.15

When the approximation max*(x, y) ~ max(x, y) is applied to the forward and backward recursions in (16.114) and the extrinsic a poste1iori L-value calculations

Sectior1 1

of Turbo Codes

Iterative

we obtain for the first iteration of decoder 1 (So);::,; lTIBX {-0.70. 1.20} = 1.20 (S1) ;::,; max

{-0.20, -0.30} = -0.20

== max

l

+

+

-1.65}

= 1.55

(Si);::,; max [+!(-1.8 + 1.1) + 1 ll

-

= max {0.85, (3 2 '(So) ;::,; ma;c

[+!

l

2

- 1.1) -

- 1.1) - 0.20]

= 1.25 1.25} = 1.25

(3 2 '(Si);::,; max{-1.45, 3.05)

f '

= 3.05

,B;'(So);::,; max {[-!(LO - 0.5) + 1.25],

i(LO -

+ 3.05]

f J

= max (1.00, 3.30) = 3.30 (S1);::,; max { [+i(LO

+ 0.5) + 1.25], [-i(LO + 0.5) +

= max {2.00. 2.30} = 2.30.

Lil)(uo);::,; 0.1 + 2.30 - 3.30

= -0.90

- 0.45 +

(16.123a) + 0.45 + L25]}

- max {[0.25 - 0.45 + 1.25]. [--0.25 + 0.45 + 3.05]}

== max

1.95} - max (1.05, 3.25) = 2.35 - 3.25 = -0.90,

1

~~"'HH ... ~

L11

we have L1 \u2) ;::,; +1.4 and \u3) ;::,; -0.3. L-values as a L-values for decoder 2, that the roles of u l and u2 are reversed for decoder 2, we obtain

at(So) = -!(0.8 - 0.9 - L2) = 0.65 at(S1) =

+i(0.8 - 0.9 - 1.2) = -0.65

(16.124b)

(So) ;::,; max {[-!(-1.8 + 1.4 + L2) + 0.65], [+!(-1.8 + 1.4 - 1.2) - 0.65]}

= max {0.25, -1.45} = 0.25

(16.124c)

(S1);::,; max {[+!(-L8 + 1.4 + 1.2) + 0.65], [-!(-1.8 + L4 - L2) - 0.65]}

= max {1.05, 0.15) = 1.05 (So);::,; max {[-!(1.0 - 0.9 + 0.2) + 0.25], [+i(l.0- 0.9 -

= max {0.10. LOO) = 1.00

(16.124d) +LOS]} (16.124e)

838

Chapter 16

Turbo Coding

a 3(S1) ~ max {[+i(l.O - 0.9 + 0.2) + 0.25], [-!(1.0 - 0.9 - 0.2) + 1.05]}

/3 (So)

= max {0.40, 1.10} = 1.10 = -!(1.6 - 0.3 -1.1) = -0.10

(16.124g)

/33(S1)

= +i(l.6 - 0.3 + 1.1) = 1.20

(16.124h)

/3l(So)

~ max {[-!(1.0 - 0.9 + 0.2) - 0.10], [+!(1.0 - 0.9 + 0.2) + 1.20]}

3

(16.124f)

= max {-0.25. 1.35} = 1.35

/3 2* (S1)

~

(16.124i)

t

1 max { [+ 21 (1.0 - 0.9 - 0.2) - 0.10], [- 21 (1.0 - 0.9 - 0.2) + 1.20J-

= max {-0.15. 1.25} = 1.25

1

/3 '(So)

(16.124j)

~ max {[-!(-1.8 + 1.4 + 1.2) + 1.35], [+!(-1.8 + 1.4 + 1.2) + 1.25]} = max {0.95, 1.65} = 1.65

1

/3 (S1)

(16.124k)

~ max {[+!(-1.8 + 1.4 - 1.2) + 1.35], [-!(-1.8 + 1.4 - 1.2) + 1.25]}

= max {0.55, 2.05} = 2.05. Lf\uo) ~ -1.2 + 2.05 - 1.65

(16.1241)

= -0.80

(16.125a)

Lfl (u2) ~ max {[0.6 + 0.65 + 1.25], [-0.6 - 0.65 + 1.35]} - max {[-0.6 + 0.65 + 1.35]. [0.6 - 0.65 + 1.25]}

= max {2.5, 0.1} - max {1.4, 1.2} = 2.5 - 1.4 = 1.10,

(16.125b)

and, using similar calculations, we have d 2) (u 1) ~ -0.8 and Lfl (u3) ~ +0.1. These approximate extrinsic a posteriori L-values for row and column decoding are listed in Figure 16.19(j) and 16.19(k), respectively. Finally, we calculate the approximate a posteriori L-value of information bit uo after the first complete iteration of decoding as L< 2\uo)

= Lcruo + Lf\uo) + Le(uo)

~ 0.8 - 0.9 - 0.8

= -0.9,

(16.126)

and we similarly obtain the remaining approximate a posteriori L-values as L< 2l(u2) ~ +0.7, L< 2l(u1) ~ -0.7, and L<2\u3) ~ +1.4. These approximate a posteriori L-values are shown in Figure 16.19(1), and we see that in this case the Max-log-MAP algorithm would incorrectly decode both bits u 1 and u2 if decod;_g '"nn n·~-pe,.l nJ'•=~ ~"= 1·•=•·n•;o" T" I;';~,,~= 1 C.. 1 Olm\ In\ nn,.l (~ \ u,o ~ ; H O tho 111 Vva,:, .:')tVp u ct1.l\..,l VU\,., Ll,,..,j_QU H, .JLl.l ..ll lt)U.l\.., .LV,.l..Y\1 lJ~ \ .LJ, a.1 'U. VJ, V\IV 5.tvv UJ.V

approximate L-values L~1\u1), Li2\u1), and L<2\u1), respectively, l = 0, 1, 2. 3, for a second iteration of Max-log-MAP decoding (see Problem 16.26). We see from Figure 16.19(o) that if a final decoding decision was made after two iterations of decoding, the Max-log-MAP algorithm would decode all bits correctly, the same as the log-MAP algorithm.

Section ·16.5

iterative

of 1·urbo Codes

239

The

is sin1ilar to in the sense that the extrinsic infor1nation from the the SJ\TR at the ~,~vc•,,.ii,h

to

0

a factor of 2 the two decoders In this case, the a L-values for the first iteration of decoder 2 7vill be the same as for decoder to 0). and the L-values will then be

o

After i:1 sufficient taken frorn the a these

decision can be

" the L-values of the bits rern.ain constant concatenated iterative however, outer decoder enter the inner and thus the L-values of the iterations [25]. o

The

is ineffective .for since channel L-valnes for the information bits are not available as to decoder 2; hmvevec the iterative decoder of Figure 16.6 can be modified to decode PCCCs with constituent codes [42].

trative purposes achieved with and the iterative o

r,r,rr,rh~~

is normally for large block lengths, remains the same.

Further iterations of in these would most likely improve the a L-values of the input bits; that L C2l (uo) and L C2 l(u 2 ) 2 2 would become more and LC l(iq) and LC )(u 3 ) would become more (see Problem 16.27). His however, on very noisy channels. for the decoder to converge to the correct decision and then diverge again, or even to "oscillate" between correct and incorrect decisions.

840

Chapter 16 @

@

Turbo Coding

Iterations can be stopped after some fixed number, typically in the range 10-20 for most turbo codes, or stopping rules based on reliability statistics can be used to halt decoding. The Max-log-MAP algorithm is simpler to implement than the log-MAP algorithm; however, it typically suffers a performance degradation of about 0.5 dB.

0

It can be shown that the Max-log-MAP algorithm is equivalent to the SOVA

algorithm presented in Chapter 12 [43]. The effect of the number of decoding iterations on BER performance is illustrated in Figure 16.20 for the rate R = 1/2 PCCC with information block length K = 216 = 65536 obtained by using a pseudorandom interleaver and puncturing alternate parity bits from the encoder of Figure 16.l(b). The performance improvement with increasing iterations, as well as the diminishing effect as the number of iterations gets large, is clearly evident from the figure. This is the original code proposed in the paper by Berrou, Glavieux, and Thitimajshima [2] that launched the field of turbo coding. After 18 iterations, a BER of 10-5 is achieved at an SNR of Eb/ No = 0.7 dB, which is only 0.5 dB away from the Shannon limit for channels with binary inputs. We conclude this chapter with a brief discussion of stopping rules for iterative decoding, that is, methods of detecting when decoding has converged. One such method is based on the cross-entropy (CE) of the APP distributions at the outputs of 1

-

Uncoded BPSK - - 1 iteration - - · 2 iterations - - 3 iterations ········· 6 iterations - • 18 iterations - ·

0.1

·..·.. ' '

~

\. .....·. ',' ... \ ...·.... ' \ . ·.•·. . \ ·..

0.01

\

0.001

~

o:'.'

\

0.0001

\

\ le-05

\

le-07

\ \ \

\

...

\

•.·..

... ·.·.• ...

\

\\

'\ \

\

"·,,", .

le-06

\

''

\ \

'

0

FIGURE 16.20:

1

2

4

5

The effect of iterations on decoding performance.

6

Iterative Decoding of Turbo Codes

Section 1G. 5

:S:41

the two decoders. The rr,,,es .. Pn 11 Q) of two joint probability distributions of the bits in the vector Tlil = P(Tlll) and Q(Tlll), [uo. iq, · · ·. u1. · · ·. uK-1]. is defined as (see [41]) F'(M)

II where E p 11

{.}

l=

K-1

(16.127)

Q(uu) J

denotes

vvith

Q) is a measure of the closeness of tvvo 11

Q) = 0 iff P (111) = Q (u1). u1 = ±L l = 0. 1. · · · . K - 1.

(16.128)

when two distributions are the D(P 11 Q) is close to 0, and the value of 11 Q) can be used as an indication of when iterative Uv'-'V'a'rn".:_ has and the iterations can be The CE rule is based on the difference between the a posteriori Lvalues after successive iterations at the of the two decoders. For example, let (l)

- (1) ( ) - (1) . ) + La(i) l!/ + Le(i) (U/

L (i)

(16.129a)

of decoder 1 after iteration i, and let

the a posteriori L--value at the

(16.129b)

represent the a L-value at the of decoder 2 after iteration i. Now, 2) 1 (ll ( · "h " '· th ' ) '( d r (2) ( U/ '1 = L e(i) (l) ( U/ ) , th a t 1s, · th e ( ) usmg l e iacts at LtrU) 111 = ,~e(i-lJ 111 an r ~o(i) a priori L-value at the input of one decoder is the extrinsic a posteriori L-value at the output of the other and Q (zq) and P (u1) represent the a posteriori distributions at the outputs of decoders 1 and 2, respectively, we can write (16.130a)

and (P)

LUl (u1)

=

Lcr111

- (0) , (P) + Le(i) (u1) + Le(il (u1).

We can now write the difference in the two soft

(16.130b)

as (16.131)

that is, ~L~[; (u1) represents the difference in the extrinsic a posteriori L-values of decoder 2 in two successive iterations. We now compute the CE of the a probability distributions P (uz) and Q (u1) as follows: P(u1)lI P(u1=+l) P(u1=-l) Ep { log-- =P(u 1 =+1)log +P(u1=-l)log Q (u1) Q (u1 = +1) Q (u1 = - 1)

+ (Pl

!Pl

e-Lu 1 (ut) ---P-1 1

1 + e -Lui

(111)

e-Lu 1 (11 1J

log

(P}

1 + e-L1,1

101

1 + e-L1ii /0)

(u1)

e-Lu,

(u1)

(16.132) (u1)

842

Chapter 16

Turbo Coding

where we have used expressions for the a posteriori distributions P (u1 = ±1) and Q (u1 = ±1) analogous to those given in (12.123). It can be shown (see Problem 16.30) that (16.132) simplifies to P(u1)}

Ep {l o g - Q (uz)

=

(16.133)

This expression can be simplified further under the assumption that the hard decisions based on the signs of the a posteriori L-values of both decoders do not change once decoding has converged; that is, the hard decisions after iteration i, u satisfy

ii),

u,(i) 1

(P) (Q) (u1) J. = sgn [ L(il (u1) J= sgn [L(il

(16.134)

Using (16.134) and noting that (16.135a) and

(Q) I [ (Q) J (Q) ,(i) (Q) IL(i) (u1) = sgn L(i) (u1) L(il (u1) = u1 LU) (111),

(16.135b)

we can show (see Problem 16.30) that (16.133) simplifies further to

(16.136) We now use the facts that once decoding has converged, the magnitudes of the a posteriori L-values are large; that is, (16.137) and that when x is large, e-x is small, and (16.138) Applying these approximations to (16.136), we can show that (see Problem 16.30) P (u1)} ILl(} Ep { logQ(ui) ~e- Iii

1 ( 1)I [ "

11'1 111 l-e-"1,1,1 i:lLc1;1< l

(

l+uj'·1~L~f\(u1) ) ] .

(16.139)

Noting that the magnitude of ~L~f(Cu1) will be smaller than 1 when decoding ,Iii i:lLll'I( ) 1 eui "

converges, we can approximate the term e -u 1 its series expansion as follows:

using the first two terms of

(16.140)

Section ·16.5

of Turbo Codes

ltei-ative

,cHstribu tions

we can write the CE of the

l

P (nn) _ _ ,;=c-;

K-1

(()) ("1) I e ILli,

Q (un) J

\vhere \Ne note that the statistical as the iterations ,..,_,.,v,c,;,n We next define

iiMt~l

does not hold

,.~.T<=•~D-"

K-1

value of the CE at after each iteration. (Note that in 7 (P) 7 (P) (l ) .Le(l) \ 1/ ·- Le(O)

can T(l) we need to find 0

1-e··e 1 (P) ( ·) ] S 'alre··1 (11) I , .,, V,u:.l" ~e(O) l-1/ l ",1

111: that

'·o l

bD c "l·e l 1 1°"'1··1·n1 LI la, dr

L:f0\(u1) = 0, I= 0, L · · ·, K -

1, for simulations has shown that once a factor of to 10--l vvith its co1werr,ence is and thus it is reasonable to use initial -

c?

-

T(i) <

as a

T(l)

rule for iterative decoding. The observation that led to the of (16.134) can also be used to define a rule based only on the hard-decision of each if (16.134) 1s satisfied for some number of consecutive iterations is a reasonable for all l = 0. 1. · · · . K - 1. we can declare that convergence has occurred and not quite as effective convergence 2cs the CE hard-decision-aided (HDA) Tule is to The CE rule was discussed in [1 and two rules were introduced in ["!A). Another approach to stopping the iterations in turbo is to concatenate a outer code with an inner turbo as illustrated in Figure 16.21. After each the hard-decision of the turbo decoder is used to check of the code. H no errors are detected, decoding is assumed correct and the iterations are H is important to choose an outer code with a low undetected error probability, so that iterative decoding is not

844

Chapter 16

Turbo Coding Outer cyclic

1lll

c.:ode

V(I)

E11coder1

FIGURE 16.21: A concatenation of an outer cyclic code with an inner turbo code.

prematurely. For this reason it is usually advisable not to check the syndrome of the outer code during the first few iterations, when the probability of undetected error may be larger than the probability that the turbo decoder is error free. This method of stopping the iterations is particularly effective for large block lengths, since in this case the rate of the outer code can be made very high, thus resulting in a negligible overall rate loss. For large block lengths, the foregoing idea can be extended to include outer codes, such as BCH codes, that can correct a small number of errors and still maintain a low undetected error probability. In this case, the iterations are stopped once the number of hard-decision errors at the output of the turbo decoder is within the error-correcting capability of the outer code. This method also provides a low word-error probability for the complete system; that is, the probability that the entire information block contains one or more decoding errors can be made very small. The idea of combining a turbo code with a high-rate outer BCH code was introduced in [45] and further analyzed in [46]. P'IROBLIEMS l6ol Prove that the general rate R = 1/3 turbo encoder shown in Figure 16.l(a), where encoders 1 and 2 are linear convolutional encoders (not necessary identical)

separated by an arbitrary interleaver, is a linear system. l6o2 For the length K = 16 quadratic interleaver of (16.7), determine all pairs of

indices that are interchanged by the permutation. 16o3 Consider a PCBC with two different constituent codes: the (7. 4, 3) Hamming code and the (8. 4. 4) extended Hamming code. Find the CWEFs, IRWEFs, and

WEFs of this code assuming a uniform interleaver. 16A Find the IRWEFs and WEFs for Example 16.5. 16o5 Repeat Example 16.5 for the case h = 4. What is the minimum distance of the

(40, 16) PCBC if a 4 x 4 row-column (block) interleaver is used? 16.6 Consider a PCBC with the (24, 12, 8) extended Golay code in systematic form as the constituent code. ao Find the CWEFs Aw(Z), w = 1, 2. · · · . 12, of this code by generating the codewords of the (23, 12, 7) Golay code in systematic form and then adding an overall parity check. Assuming a uniform interleaver, b. find the CWEFs A~c (Z). w =l, 2, ... , 12; Co find the IRWEFs AFC (W, Z) and sPC (W. Z); and id. find the WEFs APC(X) and sPC(X).

Problems

!ll45

(2.:c:.. 12. 8) extended Golay code fcrm as the comtituent code. Assume that the information bits in a square array and that a row-column interleaver is that is, encoder l encodes across rows of the array, 2nd encoder 2 encodes down columns of the a,..-ay. Find the (n, k, d) of the PCBC. Mi, Prove Jl1iiJl Find the codeword IRVVEF and VVEF for the PCCC in~-·-···~··- 16.6. Tl:iii}\v Find the bit ][RWEFs and VVEFs for the PCCC in Example 16.6. J°11ii,1L@ 5.6 for the encoder ',Nith the reversed the encoder with the reversed Jl(QJ_2

cocte~~;,;ords in the feeclforward en cod er

])fioJl]

Gu (D) = [1 l@Jl/!J. Cor1sider

8_

rat~

multiple turbo code G(D) = [1

Jl_;~"Jl:§

}l({\iJLfii

1+ D

(1

+D +

given

16.8.

+ ·with constituent encoder )/(1 -1- D)]

random interleavers Assuming a uniform block size , 21, CINEFs A~c (Z) and for w = 2, 3, 4, 5; lb. IRVVEFs AFC (W. and (VI/. Z); (Co find the V\/EFs AFC (X) and BPC and mL sketch the union bounds on P,,. ( E) and ( E) for K = 1000 and K = 10000, unquantized-output A-WGl\.f channel. Find the codewords to input ·weights 2 and 3 for the PCCC'.s whose generators are given in Table 16.6. In each case determine the free distance dtiw large K.. 3hovdhat for any (11' L v)

feedforward encoder

(w\Z) = [Ai1\Z) 1

Jl6i,1L7

]1!/iio1l2\

li6o1Li!JI lil6iJ;l(D Jl!1iJ~]l 1LliiiJ12

r

r

and that for any (n, l, v) systematic feedback encoder (Z) = [A; \Z) Show that a input sequence cannot terminate an (11. 1, v) systematic feedback encoder, but there always exists a weight-2 input sequence, of degree no greater than 2'' - 1, that does terminate the encoder. For an (11. 1. v) feedback show that the sequence llil = (1000 · · ·) with input weight zero starting in state S1 = (10 · · · 0), arriving in state = (0 · · · 01) after at most 2" - 2 and returning to state S1 in one For an (n, 1, v) systematic feedback encoder, show that a 1 input from state = (0 · · · 01) terminates the encoder. Zmin and derrfor the turbo codes with and nonprimitive 16-state constituent codes D and E of Table 16.6. Shmv that the bound of (16.97) is also valid for nonprimitive denominator polynomials. Use the S-random interleaver algorithm to construct a length K = 32 permutation that breaks up all weight-2 input sequences with a spacing of S = 4 or less between 1 's.

846

Chapter 16

Turbo Coding

16.23 Prove that the Gaussian random variable Lcr?) in (16.17) has variance 2Lc and mean ±Le, 16.241 Prove that for any real numbers w, x, and y, max* (w + x, w + y) = w + max* (x, y). 16o2!5 Verify the a* and fJ* expressions in (16.114). 16.26 Verify all entries in Figure 16.19 that were not computed in the text. 16.27 Complete two more iterations of decoding in Examples 16.14 and 16.15. Is there any change in the decoded output? 16.28 Calculate the cross-entropy at the end of each complete iteration in Examples 16.14 and 16.15 and Problem 16.27. 16.29 [15] Consider the (8, 4, 3) PCBC formed by using h = 2 codewords from the (3, 2, 2) systematic single parity check (SPC) code, that is, a (6, 4, 2) 2-repeated SPC code, as the constituent code, along with a 2 x 2 block (row-column) interleaver of overall size K = 4. The information block is given by the vector l!!l = [u11, u12, u21, u22J, where Uij represents the )th information bit in the ith row of the interleaver, i, j = 1, 2; the ith parity bit in the row code is given by 1 2 ), j = ), i = 1, 2; and the )th parity bit in the column code is given by 1, 2. The arrangement is shown in Figure P-16.29(a). Assume the particular bit values given in Figure P-16.29(b) and the set of received channel L-values given in Figure P-16.29(c). m. Use the trellis shown in Figure P-16.29(d) and the log-MAP algorithm to compute the extrinsic L-values for the first iteration of row and column decoding, and the soft-output L-values after the first complete iteration for each of the K = 4 information bits. lb). Repeat (a) using the Max-log-MAP algorithm. 16.30 Starting from (16.132), derive the cross-entropy expressions given in (16.133), (16.136), and (16.139).

p;

p;

ll11

Ll12

Pi'I

-1

-1

-1

-0.5 -1.5 -1.0

U21

un

111 P2

-1

+1

+1

-4.0 -1.0 + 1.5

-1

+1

p('-1

P2 12 i

-2.0 +2.5

(8. 4. 3) PCBC

Coded values

Received L-values

( a)

(b)

(c)

Decoding trellis (d)

FiGURE P-16.29

Bibliography

TI_,

C. E. "A Mathematical I., 27: 379-423, July 1948.

of

z:!U

" Bell Syst. Tech.

······-,····..···-, "Near Shannon Limit Error" Proc. IEEE Intl. Conf May 1993.

J, C. Berrou and A. "NeaT Error Correcting and Decoding: Turbo-Codes," IEEE Trans. Commun., COM-44: 1261-71, October 1996. ~, C. Berrou and A. "'Reflections on the Prize Paper: Near Optimum Error Correcting Coding and " IEEE Inform. Theory Soc. Newsletter, 48(2): 24-31, June 1998.

''Replication Decoding," IEEE May 1979.

Trans. (Q)o

G.

Combination of Simple Ones, Thanks Signals, Systems, and to 'Weighted-Output vv,,~'""'," Proc. URS! Intl. Electronics (ISSSE 1989), pp. September 1989.

7, J. Hagenauer and P. "A Viterbi Algorithm with Soft-Decision Outputs and Its " Proc. IEEE Global Commun. Conf (GLOBECOM 1989), pp. 1680-86, Tex., November 1989. §,

J. H. Lodge, P. Hoeher, and J. , "The Decoding of Multidimensional " Proc. 16th Biennial Symp. Commun., Codes Us.ing Separable MAP pp. 343-46, Kingston, fvfay 1992. ~~~'"''"' ~'"""·'""v,"

presented at IEEE Intl. Symp. June 1994.

liilllo P. Robertson, "Hluminating the Structure of Code and Decoder of Parallel Con"Proc. IEEE Global Commun. catenated Recursive Conf (GLOBECOM San Francisco, Calif., November 1994. U, S. Benedetto and G.

Parallel Concatenated March 1996.

~~-~ .. ,~."

"Unveiling Turbo Codes: Some Results on IEEE Trans. Inform. Theory, ff-42: 409-28,

U, S. Benedetto and G. Montorsi, "Design of Parallel Concatenated Convolutional

Codes," IEEE Trans. Commun., COM-4-4: 591-600, May 1996. 12t L. C. Perez, J. Seghers, and D. J. Jr., "A Distance Spectrum Interpretation of Turbo Codes," IEEE Trans. Inform. Theory, ff-42: 1698-1709, November 1996. ]..IJ.o

G. Battail, "A Conceptual Framework for Understanding Turbo Codes," IEEE J. Select. Areas Commun., SAC-16: 245-54, February 1998.

848

Chapter 16

Turbo Coding

15. J. Hagenauer, E. Offer, and L. Papke, "Iterative Decoding of Binary Block and Convolutional Codes," IEEE Trans. Inform. Theory, IT-42: 429-45, March 1996. 16. J. Hagenauer, "The Turbo Principle: Tutorial Introduction and State of the Art," Proc. 1st Intl. Symp. Turbo Codes, pp. 1-11, Brest, France, September 1997. 17. R. J. McEliece, D. J.C. MacKay, and J.-F. Cheng, "Turbo Decoding As an Instance of Pearl's Belief Propagation Algorithm," IEEE J. Select. Areas Commun., SAC-16: 140-52, February 1998. 13. A. J. Viterbi, "An Intuitive Justification and a Simplified Implementation of the MAP Decoder for Convolutional Codes," IEEE J. Select. Areas Commun., SAC-16: 260-64, February 1998. 19. D. Divsalar and F. Pollara, "Multiple Turbo Codes for Deep-Space Communications," TDA Progress Report 42-121, Jet Propulsion Laboratory, Pasadena, Calif., May 1995. 20. D. Divsalar and F. Pollara, "On the Design of Turbo Codes," TDA Progress Report 42-123, Jet Propulsion Laboratory, Pasadena, Calif., November 1995. 21. S. Benedetto, D. Divsalar, G. Montorsi, and F. Pollara, "A Soft-Input SoftOutput Maximum a Posteriori (MAP) Module to Decode Parallel and Serial Concatenated Codes," TDA Progress Report 42-127, Jet Propulsion Laboratory, Pasadena, Calif., November 1996. 22. S. Dolinar, D. Divsalar, and F. Pollara, "Code Performance As a Function of Block Size," TMO Progress Report 42-133, Jet Propulsion Laboratory, Pasadena, Calif., May 1998. 23. D. Divsalar and F. Pollara, "Serial and Hybrid Concatenated Codes with Applications," Proc. I st Intl. Symp. Turbo Codes, pp. 80-87, Brest, France, September 1997. 24. S. Benedetto, D. Divsalar, G. Montorsi, and F. Pollara, "Analysis, Design and Iterative Decoding of Double Serially Concatenated Codes with Interleavers," IEEE I. Select. Areas Commun., SAC-42: 231-44, February 1998. 2§. S. Benedetto, D. Divsalar, G. Montorsi, and F. Pollara, "Serial Concatenation of Interleaved Codes: Performance Analysis, Design and Iterative Decoding," IEEE Trans. Inform. Theory, IT-44: 909-26, May 1998. 26. C. Heegard and S. B. Wicker, Turbo Coding, Kluwer Academic, Boston, Mass., 1999. 27. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann, San Mateo, Calif., 1988. 28. B. Vucetic and J. Yuan, Turbo Codes. Kluwer Academic, Boston, Mass., 2000.

edited

J. G.

3!1t P. C. Massey and D. J. Costello Jr.. "New Codes," Proc. 2nd Intl. Turbo ber 2000.

Jr.. "Turbo Codes with Recursive "Proc. IEEE Intl. UuSHHoc,",u, DC. June 2001.

31,

32, 0. Y. Takeshita and D. J. ·~''-''"'"u,J. Jr., '·'Nevv Deterministic Interleaver "IEEE Ti·ans. IT-46: for Turbo 2000.

3:t t Sason ,md S. Shamai

Bounds on the ML Enor of Parallel and Serial Concatenated Turbo Codes via Their Ensemble Distance " IEEE Trans. ff-46: 24-47, 2000.

2,t'(!.,

R. and G. "A Search for Good Convolutional Codes to Be Used in the Construction of Turbo "IEEE Trans. Commun., COM-46: 1101-05, 1998.

S.

3r3. S. Dolinar and D. Divsalar, 1995. 806-8. May 1999.

" IEEE Trans.

31:t D. Divsalac S. Dolinar, and F. Based on 891-907.

"'Iterative Turbo Decoder

:J,9, H. El Gamal and A. R. "·······"···· .. ~ Jr.. "Analyzing the Turbo Decoder Using

the Gaussian

ff-47: 671-86.

2001. ii)!t N. Wiberg, H. A Loeliger, and R. Koetter, "Codes and Iterative Decoding on

General

"Eur. Trans. Telecommun., 513-26, September 1995.

4:-JL T. Cover and J. New York. 1991. Turbo 2000). p. 172. Sorrento,

Elements of Information and D. J. "Iterative " in Proc. IEEE Intl. Symp. Inform. June 2000.

foterscience.

850

Chapter 16

Turbo Coding

43. M. P. C. Fossorier, F. Burkert, S. Lin, and J. Hagenauer, "On the Equivalence between SOVA and Max-Log-MAP Decoding," IEEE Commun. Lett., 2: 137-39, May 1998. 44. R. Y. Shao, S. Lin, and M. P. Fossorier, "Two Simple Stopping Criteria for Turbo Decoding," IEEE Trans. Commun., COM-47: 1117-20, August 1999. 45. J. D. Andersen, "Turbo Codes Extended with Outer BCH Code," in JEE Electron. Lett., 32: 2059-60, October 1996. 46. 0. Y. Takeshita, 0. M. Collins, P. C. Massey, and D. J. Costello, Jr. , "On the Frame Error Rate of Concatenated Turbo-Codes," IEEE Trans. Commun., COM-49: 602-8, April 2001.

lJ

~l.11'. i,

" --

Besides turbo

remaikable

Dr

J

fr'i\ 17 IJQ) l~ \ / l c~

npo. u

codes forni another class codes. LDPC codes were researchers for almost 20 years, of LDPC

codes from a of view. Tanner's theorists for another 14 years, until the late 1990s when some and iterative Their research resulted in the of LDPC codes and further Long based on belief have been shown a fraction of a decibel away from the to achieve an error S, 9, 10, 12-14, 18-20]. This discovery makes the LDPC codes in many communication LDPC codes have a long interleaver to block error (3) their error floor occurs at a much lower and their trellis based. ~"•V'~'~HHF,

did propose a codes. Good LDPC codes that have been found are ancl their is very structure. and Fossorier [15-20] introduced the first algebraic and systematic construction of LDPC codes based on finite geometries. The large classes of LDPC codes have good minimum distances, and do not contain short cycles. These codes can be decoded from low to error to very or quasi-cyclic. Consequently, their encoding is simDH::rm:orn:eu with linear shift registers. Some long un1nt:-geo,rnen and can be LDPC codes have error a few tenths of a decibel away from the Shannon limit. LDPC codes and their construction based on varThis these ious methods. Also included are various methods for ranging from lm;v to and reasonable to large codgains.

852

17.1

Chapter 17

Low-Density Parity-Check Codes

INTRODUCTION TO l\DPC CODIES

As described in Chapter 3, a linear block code C of length n is uniquely ~peci.fied by either a generator matrix G or a parity-check matrix JH!. If it is specified by a parity-check matrix H, the code C is simply the null space of lH!. An 11-tuple v = (vo, v1 , · ·. , v11 ~ 1) over GF(2) is a codeword if and only ihlHIT = 0. This simply says that the code bits of a codeword in C must satisfy a set of parity-check equations specified by the rows of H. LDPC codes are specified in terms of their parity-check matrices. DJEnNnmN 17"1

An LDPC code is defined as the null space of a parity-check matrix lHI that has the following structural properties: (1) each row consists of p l's; (2) each column consists of y l's; (3) the number of l's in common between any two columns, denoted by 'A, is no greater than 1; that is, Jc = 0 or 1; and (4) both p and y are small compared with the length of the code and the number of rows in lH! [1, 2].

Properties (1) and (2) simply say that the parity-check matrix IHI has constant row and column weights p and y, respectively. Property (3) implies that no two rows of H have more than one 1 in common. Because both p and y are small compared with the code length and the number of rows in the matrix, lH! therefore has a small density of 1's. For this reason, lHl. is said to be a low-density parity-check matrix, and the code specified by lHI is hence called an LDPC code. We define the density r of the parity-check matrix H as the ratio of the total number of l's in IHI to the total number of entries in H. Then, we readily see that r

= p/n = Y / 1,

where J is the number of rows in IHI. The low density of IHl. simply implies that IHI is a sparse matrix. The LDPC code given by Definition 17.1 is called a (y, p )-reg,tlar LDPC code. If all the columns or all the rows of the parity check matrix IHI do not have the same weight, an LDPC code is then said to be irregular. In this chapter we are mainly concerned with regular LDPC codes; irregular codes are considered in Section 17.15. Note that the rows of H are not necessarily linearly independent over GF(2). In this case, to determine the dimension of the code, it is necessary to determine the rank of H. IEXAMPllE 17.1

Consider the matrix lHI given in Figure 17.1. Each column and each row of this matrix consist of four 1's, respectively. It can be checked easily that no two columns (or two rows) have more than one 1 in common. The density of this matrix is 0.267. Therefore, it is a low-density matrix. The null space of this matrix gives a (15, 7) LDPC code with a minimum distance of 5. It will be shown in a later section that this code is cyclic and is a BCH code. Let k be a positive integer greater than 1. For a given choice of p and y, Gallager gave the following construction of a class of linear codes specified by their parity-check matrices. Form a ky x kp matrix lHI that consists of y k x kp submatrices,

lntroduccion ·to LDPC Codes

Section 17. 1 0 0 0 0 0 0 0

0 0

0

0 0 0 0 0 0 0

0

0 0

0 0 0 0 0 0 0

0 0 0 0 0 0

0 0 0

0

0 0 0

0

0 0 0 0 0 0 0

0 0 0

0 0

0 0

0 0 0 0 0 0 0

0 0 0

0

0

0 0 0 0 0 0 0

0 0 0

0

0 0

0 0 0 0 0 0 0

0 0 0

=

0 0 0

0

0 0 0 0 0 0 0

0 0 0

IBI

0 0 0

0

0 0 0 0 0 0 0 0

2;53

0 0 0

0 0 0

0

0 0 0 0 0

0 0 0 0

0 0 0

0

0 0 0 0

0 0 0 0 0

0 0 0 0 0 0

0 0 0

0 0 0

0

0 0 0

0

matrix of a

0 0 0

7) LDPC code.

1911. 1%, ... , lf-l1y. Each row of a submatrix hasp and each column of a submatri:: contains a single 1. each submatrix has a total of k,o 1 's. For 1 :S i :S k, the ith row of lHI1 contains all its p l's in columns (i - l)p + 1 to The other submatrices are column of lHI1. Then, lHI2 lHI1 [ lrJy

l

(17.1)

.

From this construction of lf-ll, it is clear that no two rows in a subrnatrix of lf-ll have any 1-component in common; and (2) no two columns of a submatrix of lHI have more that one 1 in common. Because the total number of ones in Jm is kpy and the total number of entries in lHI is k 2 py, the density of lHI is 1 / k. U k is chosen much greater than L Yril has a very small density and is a sparse matrix. Now, whether lHI possesses the third of the parity-check matrix of an LDPC code defined no two columns of lHI have more than one 1 in in Definition 17.1 (that L'"''""'H-'0 on the choice of the y - 1 column of the submatrix lHI1. Therefore, the null space of IHI gives a linear block code of length n = kp, but the code is not necessarily an LDPC code with A = 0 or 1. Random permutations of columns of lHI 1 to form the other submatrices result in a class of linear block codes that contains a subclass of LDPC codes defined by Definition 17.1. Gallager did not provide a method for choosing column of the submatrix lHI1 to form the other !Hr2, ]H[3, · · · , lh'Iy, such that the overall matrix lH[ gives an LDPC code with good minimum distance and the structural properties required in Definition 17.1. Computer searches are needed to find good ,.,~HHH~H

854

Chapter 17

Low-Density Parity-Check Codes

LDPC codes, especially for long codes. Recently, such long LDPC codes have been constructed that achieve an error performance only a few tenths (or a hundredth) of a decibel away from the Shannon limit [4, 9-14, 21, 22]. EXAMPLIE U.2

Let k = 5, p = 4, and y = 3. Using Gallager's construction, we can form a 15 x 20 matrix lHI as shown in Figure 17.2, which consists of three 5 x 20 submatrices, lHI1, H2, and lHI3. Each row of lHI1 consists of four consecutive 1's, and no two rows have any 1-component in common. Submatrices lHI2 and lHI3 are obtained by two different permutations of columns of lHI1 such that no two columns (or two rows) of H have more than one 1 in common. The density of lHl is r = 0.20. Consequently, the null space of lHl gives an LDPC code. It is a (20, 7) linear code with a minimum distance of 6 (see Problem 17.11). Consider an LDPC code C of length n specified by a J x n parity-check matrix JHI. Let Iln1, Iln2, · · · , lhi1 denote the rows of lHI where lhij

=

(hj.O, hj,l, · · ·, hj, 11 -l),

for 1 :::= j < J. Let sv = (vo, v1, · · · , Vn-1) be a codeword in C. Then, the inner product 11-l

(17.2)

§J =W·Ilnj = LV/hj.1 =0 1=0

gives a parity-check sum (or parity-check equation). There are a total of J such parity-check sums specified by the J rows of lHI. A code bit vz is said to be checked by

0 0 0 0

0 0 0 0 0

0

H=

0

1 0 0 0 0

1 0 0 0 0

0

0 0

0 0

0

0 0 0

0

0 0

0

0 1

0 0

0

0

0

0

0

0

0

0 0 0

0 0 0 0

0

0 0

0

0

0 0

0

0 0

0

0

0

0

0 0

0 0

0 0

0 0

0 0

1

0 0

0 0

1 0

0

0 0

0

0

0

0

0

1

0

0

0 0 l

0 0 0

0 0 0

0

0

0 0 0 0

0

0 0 0

0 0 0

0 0

0

0

0

1 0

0

0

0 0

0 0

0

0 0 0 0

0 1 0 0 0

0 0

0 0

1 0

0

0

0

0

0

0 0 0 0

1

0 0 0

0

0 0 0 0

0 0 0 0

0 0 0 1 0

0 0 0 1 0

0 0

0 0 0

0 0

0 0 0 1 0

0 0 0

0 0 0

0

1

0

0

0

1

1

0 0 0 0

0

0 0

0

0 0 0 0

0

0 0 0

0 0

0 0

0 0

0

0

0

0

0

1

0

0

0

0

FIGURE 17.2: The parity-check matrix of a Gallager LDPC code.

0 0 0 0

0 0 0 0 1

0 0 0 1

0 0 0 0

0

0 0

0 0 0

0 0

0 1

Section 7.2

for Linear Bleck Codes

= 1. For O :S l < n, let

the sum sv -lhl i by the row lhl i) if h denote the set of rows in lHl that (/)

Tanner

= {lhlil). lhl~J. · · · , lhl~l} j y, let

on the code bit Vt. For 1 (/)

- (/)

:s :s

(/)

(17.3)

Iln J =(h,J. 0 .h,]. 1 .···.h J.11- 1).

hn

1il\

1255

(/)

= = 1. It follmvs from the third structural"'"'''"'''""' of the parity-check matrix an LDPC code that any code bit other than Vt is checked at most one row in At. the y rows in Ai are on the code bit vi (see Section 8.1 for the definition of Let S1 denote the set of sums formed by the rows in At. Then, the code bit vi is sum in S1, and each of the other n - 1 code bits is contained in at most one sum in S1. The sums in S1 are said to be on the code bit Vt. For every code bit of a regular LDPC code C. there are y sums that are on it. C can be decoded with one-step majority-logic decoding, as discussed in Chapter 8. error pattern with LY /2J or fewer errors can be corrected. Consequently, the minimum distance dmin of the code is at least y + l; that is, dmin ~ y + 1. If y is too one-step majority-logic of an LDPC code will give very poor error performance. Gallager proposed two for decoding LDPC codes [l, 2], one hard-decision and one much better error performance than the simple one-step majority-logic

U"2

lA~]Nlt!R\ GIR\Arrr!S iFOIF!. UNlEAIF!. iBlUJl<(]<( CODIE§ A graph 9 consists of a set of vertices, denoted by V = {v1, v2. · · · ) . and a set of edges, denoted E = {e1, e2, ···},such that each edge ek is identified with an unordered pair (vi, v1 ) of vertices. Such a graph g is denoted by 9 = (V, E). The vertices v; and v 1 associated with edge ek are called the end vertices of q. A graph is most ~vuu.uv,u) represented by a in which the vertices are represented as points and each edge as a line its end vertices. With this graphical -,rP11,n"'-P.T1 the two end vertices of an edge are said to be connected by the edge, and the edge is said to be incident with (or its end vertices. The number of edges that are incident on a vertex v; is called the degree, denoted by d(vi ), of vertex vi. Figure 17.3 shows a consisting of six vertices and 10 edges. Edge b connects vertices v1

h

g

FIGURE 17.3: A graph with six vertices and 10 edges.

356

Chapter 17

Low-Density Parity-Check Codes

and v2. Edges b, c, and d are incident on vertex v2, and hence the degree of vertex v2 is 3. Two edges that are incident on a common vertex are said to be adjacent or connected. Two vertices are said to be adjacent if they are connected by an edge. For example, in Figure 17.3, edges a and h are adjacent and are connected at vertex v4. Vertices v3 and vs are adjacent. A graph with a finite number of vertices as well as a finite number of edges is called a finite graph. There are many good books on graph theory, but reference [35] provides a good introduction to this subject. A path in a graph is defined as a finite alternating sequence of vertices and edges, beginning and ending with vertices, such that each edge is incident with the vertices preceding and following it, and no vertex appears more than once. The number of edges on a path is called the length of the path. In Figure 17.3, for instance, v1, b, v2, c. v4. h. vs, g. v3 is a path of length 4. H is possible for a path to begin and end at the same vertex. Such a closed path is called a cycle. No vertex on a cycle (except the initial and the final vertex) appears more than once. For example, the closed path v1. b, v2, c, v4, a, v1 in Figure 17.3 is a cycle of length 3. The closed path v1, f, vs, g, v3, j, V6, i. v1 is a cycle of length 4. An edge for which the initial and terminal vertices are the same forms a cycle of length 1 and is called a self-loop. A graph without cycles is said to be acyclic and is called a tree. Figure 17.4 shows an acyclic graph. The length of the shortest cycle in a graph is called the girth of the graph. The girth of the graph in Figure 17.3 is 3. A graph g is said to be connected if there is at least one path between every pair of vertices in Q. Both graphs shown in Figures 17.3 and 17.4 are connected graphs. A graph 9 = (V, [) is called a bipartite graph if its vertex set V can be partitioned into two disjoint subsets Vi and V2 such that every edge in [, joins a vertex in V1 with a vertex in V2, and no two vertices in either V1 or V2 are connected. H is obvious that a bipartite graph has no self-loop. Figure 17.5 shows a bipartite graph, with V1 = (v1, v2, v3) and V2 = {v4, vs, V6, v7, vs). Ha bipartite graph 9 = (V, £) contains cycles, then all the cycles have even lengths. To see that this is true, suppose we trace a cycle in 9 from a vertex v; in V1 . The first edge of this cycle must connect vertex v; to a vertex v1 in V2. The second edge of the cycle must connect v1 to a vertex vk in V1 . The third edge of the cycle connects vk to a vertex v1 in V2. Then, the fourth edge of the cycle connects v1 to a vertex in V1. This tracing process continues until the last edge of the cycle terminates at the starting vertex v;. Leaving a vertex in V1 and returning to a vertex in V1 requires two edges each time. Therefore, the length of the cycle must be a multiple of 2, an even number. The vertices of a graph are also commonly called nodes. In the following discussion, vertex and node are used interchangeably.

~iliUKI:: l/.4: An acyclic graph.

Tanner

for Linear Block Codes Cl

·v c-~--------~~::::::.___

g57,

V-1

1

---'JV;

V2 ,_

·J Ve,

FIGURE 17.5: A

in various ways [39]. The most well of codes is the trellis discussed in 12 for convolutional codes. Trellis to devise trellis-based of a code makes it to achieve MLD or near MLD with a reduction in as v.1e sho·wecl in 12 and 14. Another useful linear block codes is the of a linear block code by a Tanner which between the code bits of the code and the sums that check on them. For a linear block code of a matrix ft1J with J rows, Jlu 1 , 9r that consists of two sets of • INe construct a V1 and The first set ·)) 1 consists of n vertices that represent the n code bits of the code. These denoted vo, v1, · · · , v 11 _1, are called the code-bit vertices The second set V2 consists of J vertices that the J s 1 .s2. ··· .s 1 • given (17.2) that the code bits These vertices are called the check-sum vertices (or check A code-bit vertex v1 is connected to a check-sum vertex SJ an denoted ( v 1, if and if the code bit v1 is contained in ( or checked sum s.i. ]\To two code--bit vertices are and no two check-sum vertices are connected. H is dear that 9r is a bipartite graph. This graph was first or,onosea Tanner [3] to the structure of LDPC codes for iterative decoding and hence is caHed a Tanner The degree of a code-bit vertex is equal to the immber of the sums that contain v1. and the degree of a check-sum vertex s.i is to the number of code bits that are checked by s.i. For a regular LDPC the degrees of all the code-bit vertices in the Tanner of the code are the same and equal to y (the column weight of the and the of an the check-sum vertices are the same and ,o (the row weight of the Such a Tanner is said to be it follows from Definition 17.1 that no two code bits of an LDPC code are checked by two different sums. This implies that the Tanner graph 9r of an LDPC code does not contain cycles of 4. To decode UV~~O~HOF,

v,

858

Chapter 17

Low-Density Parity-Check Codes

FIGURE 17.6: The Tanner graph of a (7, 3) linear block code.

an LDPC code (or any linear block code) with iterative decoding based on belief propagation (IDBP) [23, 26], it is important that the Tanner graph of the code does not contain cycles of short lengths, such as lengths 4 and 6-especially cycles of length 4. Short cycles limit the decoding performance and prevent the decoding to converge to MLD [10, 25, 28, 36-39]. lEXAMPllE 17.3

Consider a (7, 3) linear block code which is the null space of the following parity-check matrix.

H=

1 0 0 0 1 0 1

1 1 0 0 0 1 0

0 1 1 0 0 0 1

1 0 0 1 1 0 1 1 0 1 0 0 0 0

0 0 0 0 1 0 0 1 1 0 1 1 0 1

It is easy to check that this code is an LDPC code. Hs Tanner graph is shown in

Figure 17 .6. The graph does not contain cycles of length 4, but it does contain cycles of length 6; one such cycle is marked by heavy lines. Therefore, the girth of this Tanner graph is 6.

11.3

A GIEOMIETRIC CONSTRUCTION Of lDPC CODIES

LDPC codes can be constructed algebraically based on the points and lines of finite geometries, such as Euclidean and projective geometries over finite fields, presented in Chapter 8. In this section, we present geometric construction of two types of LDPC codes based on a finite geometry. These two types of LDPC codes are closely related, and their Tanner graphs are dual to each other. Let Q be a finite geometry with n points and J lines that has the following fundamental structural properties: (1) every line consists of p points; (2) every point

Section 17.3

A Geometric Cons·truction of LDPC Codes

is intersected

1il!S9

y

in or intersect at one and and Jines in ((Q with {]]ll1, Icm2. · · · . [fD,i} and {£1.

be an over where the ith v; to the ith n»; of in ((} We define a vector based on the points on [, as follows:

v;

=

lf

1 if fiJi; is a 0 otherwise.

of c.·

6

~'"'

6

;,·,•c,

on£,

the incidence vector of line [, defined in 8, which the on £. It is clear that the of this incidence vector is p. V\/e form a J >< 11 matrix JI{[g) whose rows are the incidence vectors of the J lines of the finite (]Ji and whose colurnns to the n of (]Ji. H follows from the fundamental structural i.(Il and the definition of the incidence vector of a line in (]Ji that matrix (1) each row hasp ls; (2) each column has y ls ( the number of lines in (]Ji that intersect at the (3) no two rows have more than one 1 in common fundamental of ((Jl that two lines are either or intersect at one and one point in Q); and (4) no two columns have more than one 1 in A = 0 or 1 follows from The of this matrb: common; that ,rr(l) . I Jnl(()l lS r = p 11 = TIT(l). . r 01.cmrrO). 1n CQl 1s an d h ence ld(()l 1s a matnx.

lH!gi

11. This code is called the h 1 . . 1· LDPC denote db y CCQl , wmc . .11as a mmmmm distance ot at least y + 1. The Tanner graph of this code is a regular bipartite graph with n code bit vertices and J check-sum vertices. Each code bit vertex has y, and each check-sum vertex has degree p. The graph contains no of length 4. It does of 6 proof of this is left as a contain To construct the second type of LDPC we need to form vectors corresponding to the of the finite geometry (Ql. Let

The null space of

an LDPC code of (1)

I •

e

be a over GF(2) whose components correspond to the J lines of (]Ji, where the ith component v; to the ith line£; of (]Ji. Let [fD be a in (]Ji. We define a based on the lines in (Ql that intersect at [fD as follows:

860

Chapter 17

Low-Density Parity-Check Codes

where v;

={

~

if the ith line£; of()) contains the point [OJ, otherwise.

This vector vn> simply displays the lines that intersect at p, and hence it is called the intersecting (or incidence) vector of point p. We form an n x J matrix whose rows are the intersecting vectors of the n points in the finite-geometry Q and whose columns correspond to the J lines of Q. It follows from the definition of the incidence vector of a line in Q that the columns of are simply the incidence vectors of the J lines in Q. Furthermore, it follows

lH!gl

JHgl

from the definition of the intersecting vector of a point in Q that the columns of are simply the intersecting vectors of then points in Q. Consequently, the transpose of

Hg) and vice versa; that is,

)Hg) = [JHg)v, lH!g) = [JHg)v.

JHgl

JHgJ is simply

(17.4)

lH!gl

Matrix has the following properties: (1) each row has weighty; (2) each column has weight p; (3) no two columns have more than one 1 in common; and (4) no two rows have more than one 1 in common. The density of is the same as the density of

Hg>. The ranks of lH!gl and JHgl are the same.

JHgl

JHigl

gives an LDPC code of length J with a minimum The null space of distance of at least p + 1. This LDPC code is called the type-II geometry-(! LDPC code, denoted by and are called companion codes. Because and

cg). cgl

cgl JH!g) ngJ have the same rank, cgl and cgJ have the same number of parity check bits. It follows from (17.4) and structural properties of lH!gJ and Hg) that the Tanner graphs of cg) and cgJ are dual graphs; that is, the code-bit vertices of one graph become the check-sum vertices of the other graph, and the check-sum vertices of one graph become the code-bit vertices of the other graph. There are two families of well-known finite geometries, Euclidean and projective geometries over finite fields. which were presented in Chapter 8. Based on these two families of finite geometries, four classes of finite-geometry LDPC codes can be constructed: Jl.. 2, 3, 4,

11.4

type-I Euclidean geometry (EG)-LDPC codes, type-II EG-LDPC codes, type-I projective geometry (PG)-LDPC codes, type-U PG-LDPC codes.

IEG-lDP( (OIDIES

Consider the m-dimensional Euclidean geometry over GF(2·1 ), EG(m, 2s), whose structure was described in Chapter 8. This geometry consists of 2111 s points, and each point is represented by an m-tuple over GF(2·1 ). The point represented by the

Section 17.4

2(111-l)s (2ms

(17.5)

and each line consists of 2'' points. Each

Y

=

861

_ 1)

2s - 1

J =

lines in EG(m, intersected

EG-LDPC Codes

in EG(m.

21175 - 1 2" -1

is

(17.6)

lines. Two lines in EG(m. 25 ) are either or they intersect at one and point. Each line in EG(m, 2') has 2(m-lls - 1 lines to it. From can see that there are - l)/(2s - 1) bundles of parallel parallel bundle consists of 2<m-l)s lines. To construct a EG-LDPC code based on EG(m, we form the parity-check matrix whose rows are the incidence vectors of all the lines in EG(m. 2s) and whose to all the in EG(m, Therefore, consists of

IHI~b

2<111-l)s

1)

_

l=------

2s - 1

rows and n = 2111 " columns. Because each line in EG(m, 21 ) consists of 2s points, has weight ,o = 21 • Since each in EG(m, 2s) is intersected each row of (2 1115 -1)/(2 1 -1) each column of l has JI= (21111 -1)/(2·1 -1).

lHI~b

The

r of lBI~b is 0 __ 2' r _ ,__ -__ 1 -(111-lJ1 - 11 - 2111s - -

(17.7)

Form ~ 2 ands ~ 2, r 2: 1/4 and l is a matrix. The null space of hence an LDPC code of length 11 = 2111 ' , which is called an (1) 111-dimensional C EG(m. 0. s). The ( 0, s ) th-order EG-LDPC code, denoted minimum distance of this code is lower bounded as follows:

lHl~b

dmin ::'.: JI

2111s -1 ., _ l 2

+1 =

+ 1.

(17.8)

Because (2"'" -1)/(2" -1) orthogonal check-sums can be formed for each code bit, this code is capable of correcting fML = l(21111 -1)/2(2·1 - l)J or fewer random errors with one-step majority-logic as discussed in 8. To construct an m-dimensional type-H EG-LDPC code. we take the transpose of lH!~6· which gives the parity-check matrix Il-t2) _ [lH!(l) ]T EG -

EG

·

lHiib consists of J = 2 rows and = 2(m-lb(2 1)/(2· -1) columns. Each row of lHiib has weight ,o = (2 1)/(2· 1), and each column of lHiib has weight y = 2s. The null space of lHiib gives an LDPC code of length n =

l\/[atrix

111

111

11

"

111

"

-

1

-

"

-

1

862

Chapter 17

low-Density Parity-Check Codes

2<m-l)., (2 111 s - 1)/(2s - 1) with a minimum distance of at least 2s + 1. This code is referred to as an in-dimensional type-H (0, s )th-order EG-LDPC code, denoted by (2)

CE 0

(m,0,s).

EXAMPLE H.4

Let m = 2 and s = 4. The two-dimensional Euclidean geometry EG(2, 24 ) consists of 256 points and 272 lines. Therefore, is a 272 x 256 matrix with the following parameters: p = 16, y = 17, ;,_ = 0 or 1, and density r = 0.0624. The null space of gives the two-dimensional type-I (0, 4)th-order EG-LDPC code c~b(2, 0, 4) of length 256 with a minimum distance of exactly 18. The dimension of this code is 175 ( or the rank of is 81 ). Hence, it is a (256, 175) LDPC code. The two-dimensional type-II (0, 4)th-order EG-LDPC code Cfb(2, 0, 4) is the null space of = [JHI~b f, which is a 256 x 272 matrix with parameters p = 17, y = 16. ;,_ = 0 or 1, and density r = 0.0624. The length of this code is 11 = 272, and its minimum distance is exactly 17. 2b(2, 0, 4) has the same number of parity 1 bits as the type-I code b(2, 0, 4), which is 81. Therefore, c<;fb is a (272, 191) LDPC code.

lHl~b

IH!~b

JH[~b

lHlfb

ck

ck

In constructing the type-I and type-U EG-LDPC codes based on EG(m, 2·'), we can remove the origin point of the geometry and all the lines passing through the origin. In this case, the type-I EG-LDPC code can be put in cyclic form and the iype-H EG-LDPC code can be put in quasi-cyclic form. This simplifies the encoding circuit. Let GF(2 111 ·1 ) be the extension field of GF(2s). Each element of GF(21111 ) can be represented as an m-tuple over GF(2' ). As pointed out in Chapter 8, GF(2m·1 ) forms the m-dimensional Euclidean geometry EG(m, 2') over GF(2'). The elements of GF(2 111 ' ) give the 2111 s points of EG(m, 2s), and the zero element 0 of GF(21111 ) is the origin of EG(m, 2'). Let a be a primitive element of GF(2 111 s). Then, a 0 = 1, a, a 2 , ... , a 2""- 2 are all the 2ms - 1 nonorigin points of EG(m, 2·'). Let V

=

(VO, VJ,···, V2"'·'-2)

be a (2 1111 - 1)-tuple over GF(2) whose components correspond to the 2111 s - 1 nonorigin points of EG(m, 2·' ), where Vi corresponds to the point ai with O s i < 2111 •1 - I. Let £ be a line in EG(m, 2') that does not pass through the origin. Based on £, we form a (2m' - 1)-tuple over GF(2) as follows:

whose ith component

Vi

I 1 if ai is a point on = \ 0 otherwise.

£.

Section 17.4

EG-LDPC Codes

:€llS,3

The vector¥ Lis the incidence vector of the line£. H follows from (17.5) and (17.6) that there are (2(111-l)s _ 1)(2111S -1) 2·1 -1 2·1 ) that do not pass the be a matrix vvhose rows are the incidence vectors of all the Jo lines that do not pass the and whose columns n = 21171· -1 ofEG(m. This matrix has the each row has p = 2 1 ; (2) each column has weighty = (21711 -1) / (2s -1) no two columns have more than one 1 in common; that A= 0 or 1; and (4) no two rows have more than one 1 in common. The of IT-IT~~.c is 21

with 1.

n

C¥b.c(m. 0. s) is an LDPC

. 0. s) be the nuH space of - 1 with a minimum

Then, of at least

= 21111

1 2 1 -1

21111 -

y

+l

=

(17.11)

"t turns out that this LDPC code is the (0. inition 8.3 and hence is We call this code Because it is less than

2m., -

1. For a

2.1 h '-b • tu3il

((T,(1)

2m

1

-

,,~,rn,~,-..--

1. Then, it follows from Theorern 8.3

if

'li,EG.c

0< 'vvhere liV2, (hUl) is the 2 1 of hui defined (8.28). Let ho be the smallest integer that does not satisfy the condition of (17.12). H can be shown that

ho= (2' - 1) + (2' - 1)2·' + ... + = 2(111-l).1·

~~L

+ i<m-2)s+l (X)

_

_ 1)2(111-3)1

+ 2(m-2)1 +

1.

has the following sequence of consecutive powers of ot: (17.14)

0/'

as roots (42]. H follows from the BCH bound [43-45] (see Section 7.2) that the minimum distance of the 111-climensional type-I cyclic (0, s )th-order EG-LDPC code 0. s) is lower bounded as follows:

Cib.c(m,

dj}b.,

=:,: i111-l)s

+ i<m-2).1+1

_

1.

(17.15)

864

Low-Density Parity-Check Codes

Chapter 17

This BCH bound is tighter than the bound given by (17.11). The only case for which these two bounds on minimum distance are eoual ism = 2. Kn tbis cc1.se..both hounds , are 2s + 1. In fact, in Section 8.5, we proved that form = 2 the minimum distance of the (O,s )th-order EG code is exactly 2s + 1. The number of parity-check symbols of crb./m, 0, s) is equal to the degree of its generator polynomial g~b.c(X). A combinatorial expression for this number has been derived in [46]. A special subclass of type-I cyclic EG-LDPC codes is the class of twodimensional type-I cyclic (0, s )th-order EG-LDPC codes. It follows from (8.33) that the number of parity-check bits for the two-dimensional type-I cyclic (0. s )th-order EG-LDPC code 1b.c(2. 0, s) of length n = 22' -1 constructed based on the lines of EG(2, 2s) is .

c1

n - k

Therefore,

= 3s

- l.

(17.16)

crb.J2, 0, s) has the following parameters: Length Number of parity bits Dimension Minimum distance Density

n = 22s -1, n - k = 3s -1, k = 22' - 3·1·, dmin = 2' + 1,

r=

(17.17)

2' 22·1 -1 ·

For this special case, the geometry EG(2, 2s) contains 221 - 1 lines that do not pass through the origin. Therefore, the parity-check matrix lHI~~.c of the c~b.J2, 0, s) code is a (2 2s - 1) x (2 2s - 1) square matrix. c can be constructed easily by taking the incidence vector v1:, of a line [, in EG(2, 2s) that does not pass through the origin and then cyclically shifting this incidence vector v1:, 22s - 2 times. This results in 2 2s - 1 incidence vectors for the 22s - 1 distinct lines in EG(2, 2s) that do not pass through the origin. The incidence vector v1:, and its 22s - 2 cyclic shifts form the rows of Therefore, H~b.c is a square circulant matrix. Both the row weight p and the column weight y are equal to 2s; that is, p = y = 2s. A list of two-dimensional type-I cyclic (0, s )th-order EG-LDPC codes is given in Table 17.1.

JHI~b

H~b.c·

TABLE 17.1: Two-dimensional type-I cyclic (0, s )th-order

EG-LDPC codes

s

Ul

k

dmill

p

y

r

2 3 4 5 6 7

15 63 255 1023 4095 16383

7 37 175 781 3367 14197

5 9 17 33 65 129

4 8 16 32 64 128

4 8 16 32 64 128

0.267 0.127 0.0627 0.0313 0.01563 0.007813

Seclion 17 /!.

EG-lDPC Codes

:'.lliii\5

the this field as the two-dimensional Euclidean EG(2, over Let ot be a and /3 = a 5 . Then {0, L /3, /3 2 } form the subfield 22 ) consists of four The set of four (a 14 + 1701} 7 10 8 14 \;\Jith l] E {a .a ,a ,a }, which forms a line in This line does not pass of 22 ). The incidence vector of this line the the (000000011010001). This vector and its 14 l 'f" 17.1. The null space sarts rorm rne malnx IlTl(l) ciEG.c of this matrix is the two-dimensional EG-LDPC code 0. 2) of length which is the 7) LDPC code in ~;,,c,u,v,.,., the first code listed in Table 17.1. H foHows from (17.12) that the :\\;fb.,(X) of this LDPC code has 01, a 2 , a 3 , • 01 6 • a 8 , a 9 • and 12 2 a as all its roots. The roots a. a , • and a 8 are and have the same minimal
= 1+X +

e

']

'

,

The companion code of the m-dimensional LDPC code c;f c is the null space of the

EG-

b

IH[(2)

JlEG,qc

=

[IHI(l J

EG.c

]T

(17.18)

.

This LDPC code has length (2(m~l)s - 1)

-1)

n = lio = - - - - - - - - 2" -1

and a minimum distance dmin of at least 2" + 1. His not form. We call this code an m-dimensional l EG-LDPC code, denoted by C£(20) ,lj( (m, 0. s). To 0 ,qt we partition the Jo incidence vectors of lines in EG(m, the origin into

(17.19)

c;; L

2(111-l).1· -

not

,-,~uu"""'

1

l( == - - - - -

2 1 -1

classes. Each of these K classes contains 2ms -1 incidence vectors, which are obtained by cyclically shifting any incidence vector in the class 2111 s - 1 times. For each cyclic class of incidence vectors. we choose a representative, and the rest of the incidence vectors are generated cyclically shifting this representative. Now, we form a (2111 .1-1) >< K matrix IHio whose K columns are the K incidence vectors of the K cyclic classes. For 1 .::= i .::= 2111 s - 2, let IHI; be a - 1) >< K matrix whose columns are the ith downward cyclic shifts of the columns of JB.Io • Vl/e form

8615

Chapter 17

Low-Density Parity-Check Codes

the matrix Hib,qc as follows:

u,qc = r-Ho lHI1 H2

(2) HE"

(17.21)

· · · H2111,_2].

2

Then, the null space of IHI<E :~ v.qc. gives the type-II EG-LDPC code

2 cE< G) ,qc (m,

0, s) in

2

quasi-cyclic form. Every K cyclic shifts of a codeword in c£< Gl ,qc (m, 0, s) is also a codeword. Hib,qc can also be put in circulant form, which consists of K (2111 s -1) x (2111 s -1) circulants as follows: (17.22) where Gi is formed by downward cyclically shifting any member in the ith cyclic class of incidence vectors. Circulant form and quasi-cyclic form are considered to be equivalent; one is obtained from the other by a column permutation [44]. IEXAMIP'ILIE 17/ .6

Let

111

= s = 3. The three-dimensional type-I cyclic (0, 3)th-order EG-LDPC code

c~b.c (3, 0, 3) constructed based on EG(3, 2

3)

is a (511, 139) LDPC code with a minimum distance of at least 79 (using the bound given by (17.15)). Its parity-check matrix lHI~L is a 4599 x 511 matrix with p = 8, y = 72, and density r = 0.01565. Then,

Hib,qc

=

[JHI~b.cY

is a 511 x 4599 matrix with p

= 72,

y

=

8, and density

r = 0.01565. The null space of Hfb,qc gives the companion code c;b,qc(3, 0, 3) of the (511, 139) type-I EG-LDPC code.His a (4599, 4227) LDPC code. Both codes have 372 parity-check bits. The (4599, 4227) code has a minimum distance of at least nine. In quasi-cyclic form, every nine cyclic shifts of a codeword result in another codeword. In circulant form, Hib,qc consists of nine 511 x 511 circulants in a row. Recall that for the rows of

lHI~b c

111

= 2,

JHI~b,c is a (2 2s -

1)

x (2 2s -

1) square matrix. Because

are cyclic shifts of a single incidence vector of a line in EG(2,

2s) not passing through the origin, the set of columns of JHI~b_, (reading top-down) is identical to the set of rows (reading from right to left) of

Hib '1cc

=

[JHI~b .cY

has the same set of rows as

from right to left). Consequently, U.5

H~b.c· Therefore,

lHI~b ' c (in reverse order reading

c<;fb,qc and Ck1b.c are equivalent.

?G-IL/OllPC CODIE§

The structure of a projective geometry over a finite field was discussed in Chapter 8. Some structural properties of points and lines of this geometry are briefly reviewed here for the purpose of constructing the PG-LDPC codes. Leta be a primitive element of the Galois field GF(2<111 +lls), which is considered as an extension field of GF(2s). Let 7,(m+l)s _ 1 n=----

2s -1

(17.23)

Section 17.5

PG-LDPC Codes

~6?7

Then. the n elements defined in Section 8.8 form an 111-dirnensional over . 25 ). The elements (a 0 ). (01 1 ) (a 2 ), · · · . (a 11 - 1) are the points of line in PG(m, 2') consists of 2·' + 1 There are (17.24) (o/) in PG(m. 2 1 ) is intersected

lines in

2111s - 1

l/ - - - -

(17.25)

21 -1

or intersect at one and

lines. Two lines in PG(m. 2·') are either poinL Let

one

w= (vo. v1.··· .v,,_1)

to the points (a 0 ), (ot1) ..... (a;). Let L be a line in

be an

over whose ~~·M····~"' (ot of PG(m. where v; PG(m. The incidence vector of L is an 11

1)

if L contains the point (a'\ otherwise. of v [. is 2 1 + 1. V\/e form a matrix 1uJ~~ whose rows are the incidence vectors of the lines in PG(m. and whose columns to the of PG(m. 2'). Then, lH[~~ has

The

J=

(2(m-l)s

+ ... + 2s + 1)(2111.1 + ... + 2s + 1) 2s + 1

(17.26)

~~~~~~~~~~~~~~~~-

rows and n = (2(m+lls -1)/(2" -1) columns. H follows from the structural properties of lines and points of PG(m. in Section 8.8 that matrix IH[~b has the following properties: (1) every row has weight p = 2s + L (2) every column has weighty = (21111 -1)/(2s -1); (3) no two columns have more than one 1 in common; that is. A = 0 or 1; and (4) no two rows have more than one 1 in common. The density of lH[~~ is p r= 11

Form ::".

2.

- 1)(2'

2(111+ l)s -

r is relatively small. For large

very small. Therefore.

+ 1)

= ---~-111,

(17.27)

1

r is approximately

lf-lI~b is a very sparse matrix.

2-(m-l)s,

which is

868

Chapter 17

Low-Density Parity-Check Codes

H~b

The null space of gives an LDPC code of length 11 = (2<111 +lJs -1)/(2 1 -1) with a minimum distance of ai. least y + 1 = (2 111 s - 1) / (2·1 - 1) + 1. This LDPC code is called an m-dimensional type-I (0, s)th-order PG-LDPC code, denoted by c~b(m, 0, s). H follows from Definition 8.5 that c~b(m, 0, s) is simply the (1, s )th-order PG code. It is therefore cyclic. Leth be a nonnegative integer less than 2<m+l)s - 1. For a nonnegative integer !, let h(I) be the remainder resulting from dividing i h by 2<m+1)s -1. Let g~b(X) be the generator polynomial of the code c~b(m, 0, s). Let a be a primitive element of GF(2<m+ll.1 ). Then, it follows from Theorem 8.5 that g~b(X) has as a root if and only if h is divisible by 2' - 1, and

a"

0 _::: max W2,

(l/1l)

= j

(2·1

1).

-

(17.28)

0-<:/ <S

with OS j _::: m - l. Lett = a 2'- 1 . The order oft is then n = c2<m+lls -1)/(2·1 - 1). From the foregoing characterization of the roots of g~b (X). it can be shown [42] that g~b(X) has the following consecutive powers oft:

t,

e.... ,t<2'"'-l)/(2'-l)

as roots. H follows from the BCH bound that the mm1mum distance of the 111-dimensional PG-LDPC code C~1b(m, 0, s) is lower bounded as follows: dmin :::':

2111s -1 .. _ l 21

+ 1.

(17.29)

which is the same as the bound y + 1. We also can obtain this bound from (8.50) by setting µ, = 1. We can enumerate the number of parity-check symbols of them-dimensional type-I (O,s)th-order PG-LDPC code c~b(m, 0, s) by determining the roots of its generator polynomial. A combinatorial expression can be found in [46]. A special subclass of the class of type-I PG-LDPC code is the class of two-dimensional type-I PG-LDPC codes constructed based on the family of the two-dimensional projective geometries PG(2, 2s) for various s. It follows from (8.51) that the number of parity-check symbols of the two-dimensional type-I (0, s)th-order PG-LDPC code c~b(2. 0, s) is n-k=3'1 +l.

(17.30)

c~b(2, 0, s) has the following parameters: Length Number of parity bits Dimension Minimum distance

n = 22 s + 2 1 + 1, n-k=3''+1, k = 22·1· + 2·1 - 3·1·,

Density

r=----22s 2·1 1 ·

dmin

= 2.1' + 2.

+1 + +

2·1

It is a difference-set code [47]. presented in Section 8.3. The parity-check matrix JH[~~ of this code is a (2 2s + 2s + 1) x (2 2s + 2s + 1) square matrix. which can

Section 17.5

T/-\BlE 17.2: Two-dimensional

PG-lDPC Codes

~IS!':

PG-LDPC

codes

s

p,g

k

2

21 73 273 1057 4161 16513

11 45 191 813 3431 14-326

3 4 5 6 7

p

6

5 9 17 33 65 129

10 18 34

66 130

r

]I

5 0/

17 33 65 129

0.2381 0.1233 0.0623 0.0312 0.0156 0.0078

be obtained the incidence vector of a line in PG(2. and its + 2' shifts as rows. Some two-dim.ensional PG-LDPC codes are listed Table 17.2.

Let m = s = 2. The two-dimensional type-I (0,2)th-order PG-LDPC code constructed based on PG(2. is the (21, 11) code in Table 17.2. His the (1., PG code given in 8.24, whose generator polynomial ~~b (X) is +x 10 . Leta be a element of GF(2 3 x 2 ). From Example 8.23 we find that 22 ). The incidence vector w1:, of this line is

a line Lin

V£

= (0 1 0 0 0 0 0 0 0 0 0 1 0 0 11 0 0 0 0 1).

This incidence vector and its 20 11) LDPC code.

shifts form the

matrix mr~b of the the intersecting vectors of the

Note that the columns of the order PG-LDPC code c~b(m. 0. s) are of . 2'). Let -JLIT(2)

_

lllpc -

matrix of the

[TUJl.( [) ]T Jll

PG

·

(17.31)

the nun space of Ih1~i] gives an 111-dimensional type-H (0, s )th-order PG-LDPC denoted d}b(m, 0, s). has J = (2(m+l)s - 1)/(2' - 1) rows and

IBI;b

11

=

(2(m~l)s

+ · · · + 2·' + 1)(2 s + · · · + 2s + 1) 2 +1 111

1

(17.32)

mi;~

columns. The row and column weights of are p = (2ms - 1)/(2'" - 1) and y = 2s + 1., respectively. Therefore, c~b (m. 0, s) has length n and minimum distance (17.33)

870

Chapter 17

Low-Density Parity-Check Codes

The number of parity-check symbols of C~2b(m, 0, s) is the same as that of the type-I ,-,,-r!Q

r(ll (.,. (\v, PG\"i,

vVUv \..,

e\

lGo··-., 2 ' L,,(2) {'l ! Ill= , pc\"',

Jj.,.

"

_, ' "

V, C>J !C>

e-···'-v-'1e·1'·'·- ~(l) ,,-,, n ·' c4u1 ct l l LU L, PG~"', u, SJ.

,~(2) { V

pc,m, u, e

s')

is, in general, not cyclic, but it can be put in quasi-cyclic form in the same manner as for the type-II EG-LDPC code. EXAMPLE 17.8

Let m = 3 ands = 2. The three-dimensional projective geometry PG(3, 22 ) consists of 85 points and 357 lines. To construct the three-dimensional type-I PG-LDPC code 0, 2), we form the parity-check matrix JHI~b whose rows are the incidence vectors of the 357 lines in PG(3, 22 ) and whose columns correspond to the 85 points in PG(3, 22 ). The matrix JH[~~ is a 357 x 85 matrix and can be put in the following form:

c;;bC3,

lI17 117

JH[~b = [

~~ 117 117 lH!2 lBb lH!4

l ,

where 117 is the 17 x 17 identity matrix, and each lH!; is an 85 x 85 circulant matrix. The circulant matrices lH!1, lHI2, lHI3, and 1Hl4 have the following incidence vectors of lines (in polynomial form) as their first rows, respectively:

lhi.1(X) = 1 + x24 + x4o + x11 + xs4, lhi2CX) =xi+ x49 + xss + xs1 + xs4, h3(X) = x3 + x14 + x32 + x1s + xs4, 1rn4cx)

= x16 + x33 + xso + x67 + xs4.

The matrix JHI~b has row weight p = 5 and column weight y = 21. The null space of lHI~~ gives an (85, 24) three-dimensional type-I PG-LDPC code c;;b(3, 0, 2). The companion code (3, 0, 2) of this code is the null space of the parity-check matrix H;b = [H~bf. The matrix H;b has row weight p = 21 and column weight y = 5. (3, 0, 2) has the same number of parity-check bits as (3, 0, 2). Hence, it is a (357,296) PG-LDPC code and has a minimum distance of at least 6.

c<Jb

c<Jb

c;;b

!EXAMPLE 17.9J

Letm = 4, ands = 2. The parity-check matrixJHI~b of the four-dimensional type-I (0, 2)th-order PG-LDPC code c;;b(4, 0, 2) is a 5797 x 341 matrix with row weight p = 5 and column weighty = 85. C~1b(4, 0, 2) is a (341, 45) LDPC code with a minimum 4, 0, 2) is the null space of JHI;b = distance of at least 86. Its companion code [H~bf, which is a 341 >< 5797 matrix. The length of 0, 2) is 5797. Because 0, 2), which is c<J'b(4, 0, 2) has the same number of parity-check symbols as 296, 0, 2) is a (5797, 5501) LDPC code with a minimum clist;ince of at least 6.

c<Jb (

c?bc4,

c<J'b(4,

c;;b(4,

Section 17.6

of LDPC Codes

~rn

An LDPC code can be decoded in various ways, ,,_,,""_,, BF decoding, a decoding, bit-flipping (BF) decoding, (APP) decoding, and iterative decoding based on belief ·v~'""·"V'"" known as (SPA)). The first two are hard'-'"'"vuu,F,, the last two are soft-decision and the third one is in 8) is the error provide much better error nPnn.rc-n than the MLG and BF decodings. The between error ne>e-U-.nn

at the expense of some performance degradation. an LDPC code C is used for error control over an A'WGN channel with zero mean and one-sided PSD No. Assume BPSK with unit sequence energy. A codeword w = (vo, v1, · · · , v11 _1) is mapped into a I,= (x 0 , x 1, · · · . x 11 _i) before its transmission, where x1 = (2v1 - 1) = +1 for v1 = 1, and x1 = -1 for v1 = 0 with O ::S l ::S n - 1. Let y = (yo, Yl, · · · . y11 _1) be the soft-decision received sequence at the of the receiver matched filter. For 0 ::s l ::s 11 -1, YI = ±1 + 11 1, where n1 is a Gaussian random variable with zero mean and variance No/2. Let 7l = (zo. z1. · · · . z 11 _1) be the hard-decision received sequence obtained from y as follows: ,nnn,~cp

Zi

= { ~:

for for

0, ::5: 0.

YI > YI

Let IHI be the matrix of an LDPC code C with J rows and columns. Let Iln1. Iln2, · · · . Iln 1 , denote the rows of llil vvhere Ilnj

for 1 ::s j

::s

=

11

(hj,O· hj.l· · · ·, hj.11-1)

J. Then,

(17.34)

gives the

of the received sequence the check-sum

7l,

where the jth syndrome component

11-l

sJ·

= 7l · llnJ· =

'\°" Z! h J.· /. ~

(17.35)

1=0

The received vector 7l is a codeword if and only if § = @. If § -!= @, errors in 7l are detected. A nonzero syndrome component SJ indicates a parity failure. The total

1372

Chapter 17

low-Density Parity-Check Codes

number of parity failures is equal to the number of nonzero syndrome components in§. Let

=

(vo, v1, · · · , V11-1)

+ (zo, z1, · · ·

Then, ie is the error pattern in z. This error pattern condition

ie

(17.36) , Zn-1).

and the syndrome § satisfy the (17.37)

where 11-l

sJ· = ie - lln J· = ~ L,; e1h J.· 1

(17.38)

1=0

for 1 :S j :S J. 17 .6.1

Maj(Q)roty-logk ll)ewdling

MLG decoding was discussed in detail in Chapter 8. Now, we apply one-step MLG decoding to LDPC codes. It follows from the structural properties of the parity-check matrix lHI of a regular LDPC code that for every bit position l, there is a set A

I

= {lln(IJ 11nUl · · · 1 ' 2 '

'

Jht(ll} Y

(17.39)

of y rows in lH! that are orthogonal on this bit position; that is, the lth component of each row in A 1 is 1, and no two rows in A 1 have a common 1 in any other position. We form the following set of syndrome equations based on the rows in A1: (17.40) where (17.41) S1 gives a set of y check-sums orthogonal on the error digit q. As presented in Section 8.1, they can be used for estimating the error digit e1 based on the one-step MLG decoding rule. Correct decoding of e1 for O :S l < n is guaranteed if there are Ly/2J or fewer errors in the error pattern ie. All four classes of finite-geometry LDPC codes are one-step MLG decodable.

17 .15.2

/But-Fiuppnlrilg Diewidlnlrilg A!g(Q)ruitlhim

Bit-flipping (BF) decoding of LDPC codes was devised by Gallager in the early 1960s [1] /], When detectable errors occur during transmission there 'Nil! be parity-check failures in the syndrome§= (s1, s2, · · · , s 1 ), and some of the syndrome bits will be

Section 17 .6

equal to L BF

Decoding of LDPC Codes

~'7131

is based on the change of the number of parity failures in

{:z - lln1 : 1::: j ::: J} when a bit in the received sequence :z is changed (or flipped). the decoder aH the parity-check sums based on (17.34) and (17.35) and then changes any bit in the received sequence z;; that is contained in more than some fixed number 8 of failed equations syndrome bits). Using the modified received sequence :z', the decoder the parity-check sums, and the process is until all the parity-check sums are

to zero no At this the modified received sequence is a codeword in C. This is an iterative decoding The parameter 8, called the parameter that should be chosen to optimize the error performance while the number of computations of parity-check sums. The value of 8 depends on the code parameters p, y, d111 ;11 (C), and SNR. H decoding fails for a value of 8, then the value of 8 should be reduced to allow further iterations. For error whose number of errors is less than or equal to the of the code, the decoding will be in one or a few iterations. more iterations are needed. Therefore~ the number of decoding iterations is a randon1 variable and is a function of the channel SNR. A limit may be set on the number of iterations. When this limit is reached the decoding process is terminated to avoid excessive to the nature of low-density parity checks, BF decoding algorithm corrects many error whose number of errors exceeds the error-correcting of the code. A very simple BF algorithm is given here: the sums based on (17 .34) and (17.35). Hall the parity-check sums are zero, stop the decoding. Find the number of failed parity-check for each denoted by .1;, i = 0, 1. ... , 11 -1. §1lell]) 2t the set S of bits for which f; is the largest. §1e]Pl 4lo Flip the bits in set S. 5, Repeat steps 1 to 4 until all the parity-check sums are zero (for this case, we the iteration in step 1) or a maximum number of iterations is reached. In step 5, if the preset maximum number of iterations is reached and not all the parity-check sums are zero, we may simply declare a decoding failure or decode the unmodified received sequence :z with MLG decoding to obtain a decoded sequence, which may not be a codeword in C. The latter case is simply a hybrid BF/MLG decoding. This simple BF decoding algorithm can be improved using adaptive thresholds, o's. Of course, this improvement is achieved at the expense of more computations. U.'6.3

Wreo91rrritreC!l1 Mai]@ro'lty-L©igik Dec@idloru121 siru1DI lao'lt-rFin[Ol(QlDilligJ [))ce1mC!l1nri12J

The simple hard-decision one-step MLG decoding and BF decoding can be improved to achieve better performance by some kind of reliability information (or measure) of the received in their decoding decisions, as described

814

Chapter 17

Low-Density Parity-Check Codes

in Section 10.9.2. Of course, such performance improvement requires additional decoding complexity. Consider the soft-decision received sequence y = (yo, Yl, · · · , y11 _1) at the output of the receiver matched filter. For an A WGN channel, a simple measure of the reliability of a received symbol YI is its magnitude, IY1 I: the larger the magnitude IY1I, the larger the reliability of the hard-decision digit z1. This reliability measure has been used in many reliability-based algorithms for decoding linear block codes, as presented in Chapter 10. The weighted MLG decoding presented in Section 10.9.2 can be used for decoding LDPC codes. Consider an LDPC code specified by a parity-check matrix H with J rows, 1h11, lln2, · · ·, lli. 1 . For O::: l::: n - land 1::: j::: J, we define IYJl~t

~ {min{IY;I}: 0 :Si

:Sn - l,

hi_;= 1}

(17.42)

and E1 ~

v L

(2s(ll - l)lv·l(I) .1

-.1111111·

(17.43)

E1 is simply a weighted check-sum that is orthogonal on the code bit position l. Let e = (eo, e1, · · · , e 11 _1) be the error pattern to be estimated. Then, we can modify the one-step MLG decoding based on the weighted check-sum E1 as follows:

€/

=

1, { 0,

for E1 > 0. for Et ::: 0

(17.44)

for O ::: l ::: n - 1. The preceding decoding algorithm is called weighted MLG decoding [48]. The decision rule given by (17.44) can be used in BF decoding. In this case the decoding is carried out as follows: §tep 1, Compute the parity-check sums (syndrome bits) based on (17.34) and (17.35). If all the parity-check sums are zero, stop the decoding. §tep 2, Compute E1 based on (17.43), for O::: l::: n - 1. §tep 3,, Find the bit position l for which E1 is the largest. §fte]jll 4. Flip the bit z1. Step 5. Repeat steps 1 through 4. This process of bit flipping continues until all the parity-check sums are zero or a preset maximum number of iterations is reached. This modified BF algorithm is called a weighted BF decoding algorithm [17-20]. ln step 5, if the preset maximum numbers of iteration is reached and not all the parity-check sums are zero, we may decode the unmodified received sequence z with the weighted MLG decoding based on weighted check-sums computed based on (17.43) and the decision function given by (17.44). This hybrid weighted BF/MLG decoding will result in a decoded sequence. The foregoing weighted decoding algorithms require some real-number computations,

Section 17 .6 ~ 7J Ji~c

Decoding of lDPC Codes

~75

1~e 51Lfl.m-lF'm©1wcil: fil:JigJ!f:l!rn·~~uuli (SPA) is an iterative decoding based on belief 23, 25-30] that is extremely efficient for u•A,vu,u"' and its variations 12 ancl 14, it is a soft-out algorithm. h processes the received to improve the reliability of each decoded code based on the sums computed from the hard decisions of the received and the sparse matrix lHI of an LDPC code. can be measured by its uu.u,-,,rn~, of a decoded ~"'"'~~-~ ratio (LLR). or the value of its measures of code ~~s-~·~rn,~ iteration are used as for the next iteration. The iteration process continues until a certain condition is satisfied. based on the measures of code symbols. hard decisions are made. 1ve consider an LJDPC code C of length n specified a matrix JH with J rows, lln l · · · · . llu 1 , where llu.i = (/ziO· hj.1, · · · , /iin-1).

For 1 _:s j _:s J, we define the following index set for llu.i: B (Iln.i)

= {l

(17.45)

: hi.I = 1. 0 .:S I < n}.

of r111.i. of SPA decoding is based on the computation of the

for O .:S l < n, where y is the soft-decision received sequence. Then, the LLR for each code bit is P(v1 = lly) P(v1

= P(v1 = = 1, ,.P,cnF•rt,

Let P? V/

0) and

pf

P(v1

=

(17.46)

= Oly)

1) be the prior probabilities of v1

= 0 and

· d eac1 1 1kuu.i E A 1, le t q.i.1 x.Ul b e th ~or 0 <_ I < n. 1 _< J. < J , an. ie con ct··· rnonal' that the transmitted code bit v1 has value x, given the check-sums based on the check vectors in A1\llu.i at the ith decoding iteration. For 0 _:s I < n. 1 _:s j _:s J, and llu.i E A1. let be the conditional that the check-sum Sj is satisfied (i.e., s.i = 0), given v1 = x (0 or 1) and the other code bits in B(llui) have a separable distribution (qt/l : t E B(lluj )\!}: that is, ii:;

G//i)

a/t)

:::[ {1• 1 :tEB(!n 1 )\/)

P(Sj

= Oiv1

= x. {v1: t E B(Ilnj)\l)).

fl qt/). (17.47) tEB(l,i )\I

816

Chapter 17

Low-Density Parity-Check Codes

The computed values of follO"WS:

CJf t)

are then used to update the values of

q1_/+l)

as

(17.48)

where a)'.t 1l is chosen such that O.(i+l)

qi.I

+

l,(i+l)

qi.I

l

= .

The computed values of qx, U+l) are then used to update the values of CJx, (i+l) based ]. 1 ], 1 on (17.47). The updating between qx}i) and CJxz°) is carried out iteratively during ], ], the decoding process. At the ith iteration step, the pseudo-posteriori probabilities are given by pCi\v1 =xly)

=aii)P1 [1 CJ;,;U-l),

(17.49)

HiJEA1

where af is chosen such that pUl(v1 = Oly) + pCi\v1 = lly) = 1. Based on these probabilities, we can form the following vector as the decoded candidate: 'll,(i) =

c/il z(i) . . . ·O ' 1 '

{ 1,

for pU\v1 = lly) > 0.5 otherwise.

z(i) )

' ·11-l

with (i) Z1

=

0.

(17.50)

Then, compute zCil · HT. If z(i) . HT = 0, stop the decoding iteration process, and output z
§tiep 1. For O s I < 11, 1 :S j s J, and each lln1 E A1, compute the probabilities, CJf}i) and CJJ/l. Go to step 2. §tiep 2. For O :S I < 11, 1 s j s J, and each lhl1 E A1, compute the values of /·?+l) and q\(i+l) and the values of pCi+ll(v1 = Oly) and pU+i>(v1 = ], ], lly). Form z(i+ll and test zU+lJ. JHl:T. If zCi+l) .JHIT = 0 or the maximum iteration number l 111 ax is reached, go to step 3. Otherwise, set i := i + 1 and go to step 1. Step 3. Output zCi+ll as the decoded codeword and stop the decoding process. A detail exposition of SPA decoding of LDPC codes can be found in [10].

Section '!

tions in find that the number of of the order O (21 p + order O We see that the

,-.,u,"UH'U'Hu

of LDPC Codes

817/

multiplication, needed. fo implethan addition and the number of additions and subtracFrom (17.46) through (17.49), we needecl in each iteration of the decoding is needed is of the o:f SP A decoding is mainly a matrix lh1 of the LDPC

code to be decoded. A different version of SP A is based on extrinsic information at each iteration. The ex'.:rinsic information at one decoding iteration is used for the extrinsic information and the values of the code at the next iteration. The iterations continue until a ,modified hard-decision Ieceived sequence is obtained that makes all the to zero. Consider a code-bit . Let lb be a rovv i_n the ser A1 that is orthogonai on bit Then, the extrinsic information to the transmitted code bit v1 by other code bits checked by n~ the check-sum;,,:· Jlu) is [1, 2, 30, 49] 1+

tanh { IEB(Iln)\/

\

tanh IEBC,n)\i

The total extrinsic information

~ y1) 1 O

I

(17.51)

1( _3_ v,) . \ JV . O

'

to code bit

the rows in A1 is then

v1

TI11EA1

v1)

1+

tanh (·1 _3_ N. IEB(Tin)\I

1-

']

]

\

11

(17.52)

'(2 )'

f1 J[ Jl

tm1h

IEB(Iln)\I

V\/ e define a J ><

O

N Yr O

matrix

1E __ -

]

. ]O~/
-

(17.53)

·where

=

I l

£1. -

0.

s1

for l E othe:r-wise.

The matrix lE is called the extrinsic matrix. We define another J x

(17.54) 11

matrix (17.55)

878

Chapter 17

Low-Density Parity-Check Codes

where

Yp

~

I

4

for l

No YI, 0,

E

B(lhlj),

(17.56)

otherwise.

This matrix is called the channel information matrix. In the decoding, each decoding iteration updates (or improves) the extrinsic values s1 (lhlJ ), s1, and E 1,1 for O ::: l < 11 and 1 ::: j ::: J. These values computed during the i th decoding iteration are denoted by i ), s;il, and For O ::: < and 1 < j ::: J. the extrinsic value is used to modify the channel information Y1,1 by setting

s?\h

EJ'.;.

l n

y

zU+l) j,l

-

Ej'./

J,/

+ EUJ jJ·

(17.57)

We form the channel value matrix at the end of the ith iteration as follows: zUl =

[z(i)]O:skn

(17.58)

1) l:S.1:Sl'

where z(,J -

,z-

Y

1J

+ £(,-1) 1J

(17.59)

For i = 0, z(Ol = Y. The entries of zUl are used to compute the extrinsic values for the (i + l)th decoding iteration. For O ::: l < 11, is used to modify the received vector y = (yo, Yl, · · · , y11 _1). Let

s;i)

Let Ir

(i) -

( (i)

(i).

(i) )

ro 'rl · · · · 'rn-l

be the modified received vector at the ith decoding iteration. Then, the modified received vector at the next iteration is Ir

(i+l) _ ( 1.U+l) Ji+ll . . . (i+])) 0 ' 1l ' · rn-l ·

where an d r1(0) = Let

4

vz f or 0

-No·

-< l < 11. :Z:

(i) -

-

(i) (i) ) Zo ,Z1 ,···,Zn-l

( (i)

be the hard-decision vector obtained from n.Ul =

(rg>. riil, · · · . r,;;2 1) with (17.60)

Section 1

of LDPC Codes

This hard-decision vector can be tested 'Nhether it is a codeword = :l]i, 'li,U) is a codewmd and '-"···'"'-'u.'u" iteration continues until either the Jmaxilnurn ntnnber of iterations is reached. UL,v>.,u,u~ in terms of extrinsic information is carried out as follows Seti

JL,,

= 0, the maximum number of iterations to ~

JFor O ::S I < n, 1 ::; j <

and each

-

tl7'.9

the 1S

_±_\\l

f\loJ'

E

_J__ I

(i)

E:1

lfuJ ) i

illl

IEB(lli1 1 )\/

--

1IEB(fiTij )\/

~\

L

2 1U\frn1

)•

[l, 1 EA1\U,i

(i)

E:1

=

) _,_ E: (i) j.1 I I

and form the extrinsic matrix JEU>_ Go to 2. Form zU+ll = Y + l_rU+]) = n.(O) + , and 7 7 (2i{1. 2f l. · · · . 1). Test;zU+l) -lh1 . J[faU+ll -Ifl.! = the iteration number lma, is to 3. Otherwise set i := i + 1 and go to 2t z;:U+ 1 l as the decoded cocle·word.

§1iE:l[D2o

<;~

(1

).

where Jt[]) or thei-naximum iteration and go -l-lJ,

The vv,.u,.;u"rn SP A increase as the number of '--'""'"''-!!ll are not desirable in LDPC code has a large LDPC m error of the LDPC code to be decoded and the channel

sums are zero, we switch to MLG decoding to decode the hard-decision received sequence obtained at the l 111 axth iteration. If the number of residue errors left uncorrected in the hard-decision received sequence is within the M[LG of the the errors will be conected. This is often the case for with large majority-logic codes. Based on many we have observed that SPA U'-''-")''-HH" of long finite-geometry LDPC codes converges very fast. After a few

880

Chapter 17

low-Density Parity-Check Codes

five, the number of residue errors left uncorrected is small, and the MLG decoding IviLG decoding cm be implemented 3t the will correct these residue errors. In end of each decoding iteration. At the end of each decoding iteration, we check the hard-decision received sequence. If the number of unreliable positions based on a certain threshold is within the MLG error-correcting capability, we switch to MLG decoding to terminate the decoding iteration process: otherwise, we continue to the next decoding iteration until the lmax th iteration is finished. This combination of SP A decoding and J\1LG decoding is called two-stage hybrid SP.PJMLG decoding [17]. 11.6.5

Performance of finite-Geometry llDPC Codes

To demonstrate the error performance of finite-geometry LDPC codes, we select several EG- and PG-LDPC codes of various lengths and decode them with various decoding algorithms. \EXAMPLE 11.10

Let m = 2 ands = 5. The two-dimensional type-I (0, 5)th-order cyclic EG-LDPC code is a (1023, 781) code (the fourth code in Table 17.1). This code has a minimum distance of 33 and rate R = 0.763. Its error performance with various decoding algorithms is shown in Figure 17.7. We see that MLG decoding provides the least coding gain over the uncoded BPSK but requires the least decoding complexity. 1011 ~ - - ~ - - ~ - - ~ - - ~ - - ~ - - ~ - - - - . - - - = ~ • BPSK uncoded ...,._ PG-LDPC SPA bit - PG-LDPC SPA block ._ EG-LDPC BF bit """""EG-LDPC weighted BF bit - EG-LDPC MLG bit - EG-LDPC SPA bit ...,.. EG-LDPC SPA block - Shannon limit

---

0

2

3

4

5

6

7

8

E1,IN0 (dB)

FIGURE 17.7: Bit- and block-error probabilities of the (1023, 781) two-dimensional type-I cyclic EG-LDPC code and the (1057, 813) two-dimensional type-I cyclic PG-LDPC code based on various decoding algorithms.

of LDPC Codes

Section 1

~lH

The

to the order EG-LDPC r:ode. Hs error is also included in 17.7. "Ne see that the 'cvvo codes have ahnost identical error ·-~·-"·-···"'T"·"·'~'h

code with a minimum distance of 65 and rate R

EG-LDPC 0.83

=

m in 17 .8. This code with SP A from the Shannon limit at s BER of 10- 5 . trade··Off between error n,c,·,,sc,·w,·n two-dimensional code v1ith a minimum distance of 66 and a rnte of 0.825. lts error n~,·1e,·w,n almost identical to that of the EG-LDPC code. as shown in

Let m = s = 3. The three-dimensional EG-LD PC code (0. ( l) 0, 3) constructed based on code with dmin :::: 79 CEG.c and rate R = 0.272 in H is a low-rate code. Hs parity-check is a 4599 >< 511 n1atrix with ,o = 8. y = 72. and r = 0.01565. The

1 l of rr-nlEG.c~ __ rm,(l) -

is a 511 >< 4599 matrix im(2) EG.qc

]T

•

= 8. and r = 0.01565. The null space of

(0,

EG-LDPC code

code with a minimum distance of at least 9 and code. The bit- and block-error of both are shown in 17.9. We see that the (4599, 4227) very well. At a BER = 10- 5 , the performance of 1 dB from the Shannon limit.

EG (JC (3. 0. 3) that is rate R = 0.92. It is a codes with SP A

u,A,VLIU

this code is

_llHEG.c

882

Chapter 17

Low-Density Parity-Check Codes I

I

..."" BPSK unccded -- PG-LDPC SPA bit ->- PG-LDPC SPA block -1,- EG-LDPC SPA bit ...,_ EG-LDPC SPA block - EG-LDPCBFbit - EG-LDPCMLG bit - . EG-LDPC weighted BF bit - Shannon limit

j

1 ~

1 : : -

10-s -

3

2

5

4

6

9

8

7

E 1,/N11 (dB)

FIGURE 17.8: Bit- and block-error probabilities of the (4095, 3367) two-dimensional type-I EG-LDPC code and the (4161, 3431) two-dimensional type-I PG-LDPC code based on various decoding algorithms.

10"

--- --

10-1

~

10-2

E

"'

.D

80.

10-,

>-<

8 ;:; ;; 10-~ u 0

~

iii

10-:i

j

10-<>

l-+-[ii;~o;~~~1it for (4599, 4227)

!'Shannon limit for (511, 139) LDPC code

-2

-1

0

2

3

4

5

6

7

8

j 9

E1,IN" (dB)

FIGURE 17.9: Error performances for the (511, 139) type-I EG-LDPC code and the

(4599, 4227) type-H EG-LDPC code with SPA decoding.

of LDPC Codes

Section '17 .6

3:835

LDPC codes have been found with computer searches these codes do not perform as well as codes. Furthermore, have relatively small are not suitable for JVJ[LG or BF decoding.

Let m = 2 ands = 4. The two-dimensional PG-LDPC code is a 191) code third code in Table 1 rate R = 0.699, of at least 18. Its error performance 17.10. PJso included in the figure are the (0. 4)th-order cyclic (255, SPA error 191) LDPC codes with 175) EG-LDPC code JI = 3 and JI = 4, the two LDPC codes should perform but their encoding may be or quasi-cyclic. Another LDPC codes is that it is very hard to

,_,,,,,v'-''""""''

determine their minimum distances,

-------r----i---,---T--i-~-~---~-~ ~· · Uncoded BPSK PG-LDPC -~~ EG-LDPC ~"~ Gallager code y ~e= Gallager code y ec@~

0

FIGURE 17.10: Bit-error 191) type-I PG-LDPC with SP A decoding.

2

3

5

6

7

= =

3 4

8

type-I EG-LDPC code, (273, (273, 191) Gallager codes

884

Chapter 17

Low-Density Parity-Check Codes I I ~ - - Uncoded BPSKl ~

:: t1~~n~~~r-1 i

J: 10-1

-,,,..;,.. -

Max ItNum 5 Max ItNum 10 Max ItN um 20 Max ItNum 100

',,,

0

2

3

4

5

6

1

', '

7

~ -

"'·,, ,, j

8

E1,IN0 (dB)

FIGURE 17.11: Convergence of SPA decoding for the (4095, 3367) type-I EG-LDPC

code. Simulation results show that the SPA decoding process of the EG- and PGLDPC codes converges relatively fast. The difference in performance between 10 iterations and 100 iterations is very small. As an example, consider the (4095, 3367) two-dimensional type-I EG-LDPC code given in Example 17.11. The convergence of SPA decoding of this code is shown in Figure 17.11. We see that at BER= 10-4, the performance gap between 5 iterations and 100 iterations is less than 0.1 dB. We also note that the performance gap between 2 iterations and 100 iterations is about 0.5 dB. In practice, if decoding speed (or delay) is critical, 5 iterations may be good enough, or two-stage hybrid SPA/MLG decoding may be used with the maximum number of iterations set to 2 or 3. With SPA decoding, the error performance of an LDPC code depends on several important parameters: (1) the girth of its Tanner graph (or cycle distribution): (2) its minimum distance; (3) the column and row weights of its parity-check matrix; and (4) its error coefficient (i.e, the number of minimum-weight codewords). The girth should be large enough so that there are no short cycles, especially cycles of length 4, in its Tanner graph. Short cycles prevent SPA decoding from converging to MLD or near-MLD performance: however, girth should not be too large. An LDPC code whose Tanner graph has a large girth tends to have a relatively poor minimum distance [38]. The error floor in the error performance of an LDPC code is tied to the minimum distance of the code. Large minimum distance either removes the error floor or pushes it down to a much lower error probability. An LDPC code with a small minimum distance decoded with the SPA usually displays an error floor at a relatively high error rate and has a poor block-error performance, just like most of the turbo

17.

Code Co11struction

codes. For vvhose error

y=3 curve drn--

-~c,·'n•,o·,•n.

an error floor at BER

= 5x

other code contribution of information a umber of other code bits. This results in a better extrinsic ccnTvergence. The error coefficient affect; ~lTOf of the code in the rctnge of sJ..1u21H SftR~_s or error rates. Smaller er,_·or coefficients better error in the small SNR range.

extenC~ed each columns. 1Chis results m a new and hence a new LDPC code. H columns LlDPC codes can be obtained. a fevv tenths of a

decibel away fron1 the Shannon limit. Let go, g 1 , · · · • g11 _ 1 denote the columns of the matrix TEL First, we consider each column of lHi into the same number of columns. AH the new columns have the same as the 1 's) of the column is distributed arnong the new columns ..A distribution c;m be done as follows. Let q be a such that 2 :"::c q :"::c y. y q, we have )I

=q

>< Jlexi

+ b,

each column g; of IHI into q columns gi.l, g;,2, · · · . such that the first b g;.1, !l;i.2- · · · , g;,1,, have Yexi + L and the next - b columns.11;;,1,+J · g;,1,+2- · · - , 1!,i.q, have Jin,· The distribution of y ''ones" of g; into g;;_ 1, 11!;;,2, · · · • !!;i.q is carried out in a manner. In the first the first 1 of g; is in l!;i.1, the second 1 of l!;i is put in g;_2, and so on. fo the second the (q + 1)th 1 of g; is put in !!;i.l, the (q + 1 of g;; is in g; 2 and so on. This distribution of the l's of :it; continues until aH the 1 's of g; have been distributed into the q new columns. The column splitting results in a new matrix Il-Tie,1 with qn columns row has ,o; (2) each that has the structurrli any two columns have at most column has either weight Jlext or weight Jlext + 1; one 1 in common. H the of JBI is r, the of IT-ITex 1 is then r/q. Therefore, the column matrix with smaller The ""'-""'""''-,.,, TLDPC code rn1ll space of q, then the columns of have the qth extension a code bit of the extended code i:wo different Yext and Ye:a + l. is either checked Jlexr check-sums or Jlexr + 1 check-sums. In this case, the extended finite-geometry LDPC code is an LDPC code.

886

Chapter 17

Low-Density Parity-Check Codes

IEXAMPllE 17.14

Consider the (4095, 3367) two-dimensional type-I (0, 6)th-order cyclic EG-LDPC code discussed in Example 17.11. Suppose we split each column of the parity-check matrix of this code into 16 columns with rotating column weight distribution. This column splitting results in a new low-density parity-check matrix of 65520 columns with row weight p = 64, column weight Yext = 4, and density r = 0.00098. The null space of this matrix gives a (65520, 61425) extended two-dimensional type-I EG-LDPC code with rate 0.9375. This code decoded with SPA achieves an error performance that is only 0.42 dB away from the Shannon limit at BER = 10- 5 , as shown in Figure 17.12. We see that it has a sharp waterfall error perform
For m = 2 and s = 7, the two-dimensional type-I (0, 7)th-order cyclic EG-LDPC code is a (16383, 14197) code with minimum distance dmin = 129 whose parity-check matrix has row weight p = 128, column weight y = 128, and density r = 0.007813 10() F_- - , - - - , - - - - , - - - . . . , , - - , - - - , - - - - , - - - - , - - - = ,

-

· BPSK uncoded

-Bit

--mock

\ <:.:h~nnon limlt 10-(, ~ - - - ' - - - - - ' - - - - ~ - - - ' - - - - - - ' - - - ' - - - ~ - - - ' - - - - - ' 0 2 3 4 5 7 8 9 6 E1,IN0 (dB)

Bit- and block-error probabilities of the (65520, 61425) extended type-I EG-LDPC code with SPA decoding.

FIGURE 17.12:

Section 17.

Code Construction

Column and now

~~-BPSK uncoded -;c~ (16383, 14197) base EG-LDPC bit (524256. 458724) extended EG-LDPC

$:~,

~ iii

10-"

Shannon li111it for base code

ci11,ua,uu limit for extended code

10-nc_____~--~--------~--'~~-----~~-----"·-<~~~---'--~·~----"-~~~-• 6 3 4 7 8 l 2 5 E1,IN0 (dB)

FIGURE 1

3: Error '"'~" ~···,rm,,v,.,u of the

LDPC code and the

0

14197) two-dirnensional EG507873) extended EG-LDPC code with SPA decoding.

(the sixth code in Table Suppose we each column of the paritycheck n1atrix of this code into 32 columns. We obtain an extended EG-LDPC code of and rate 31/32. The of the matrix of this extended code is r = 0.000244. H is a very sparse matrix. The bitand blod>enor of this extended LDPC code with SFA are of the extended LDPC 17.13. At BER = 10-4, the 0.3 dB avray from the Shannon limit.

Let m = s = 3. Consider the three-dimensional LDPC code discussed in which is a (511, 139) code. Hs is a 4599 x 511 matrix with p = 8. y = and r = 0.01565. we extend this code each column of its matrix IHI into 24 columns. the extended code Cext is a 7665) LDPC code with rate R = 0.625. The extension results in a high-rate code. The of the and the column of IHiext is matrix JHiext of this extended code is rext = Yext = 3. The bit-error of this extended LDPC code and its base code ·with SPA are shown in Figure 17.14. The of the extended code is 1 dB from the Shannon limit at BER= 10- 4 .

ggg

Chapter 17

Low-Density Parity-Check Codes

lO()r 10- 1

E :.s

- • Uncoded BPSK 1 -- (511,139) EG-LDPC code ~ - (12264, 7665) extended EG-LDPC IDBP

--- ---

-

10--c

"

_o 0 .... 0. 10-1 .... 0 .... ....

!

i:o 10-.J

10

)

10-(, -

-3

-2

-1

3

4

5

6

FIGURE 17.14: Error performances of the (511, 139) three-dimensional type-I (0, 3)th-order cyclic EG-LDPC code and the (12264, 7665) extended EG-LDPC code.

Column splitting of the parity-check matrix of a cyclic or quasi-cyclic finitegeometry LDPC code may result in an extended code that is neither cyclic nor quasi-cyclic; however, if we arrange the rows of the parity-check matrix into circulant submatrices and then split each column into a fixed number of new columns with column weight distributed in a rotating and circular manner, the resultant extended code can be put in quasi-cyclic form. To see this, we consider the m-dimensional type-I cyclic (0, s )th-order EG-LDPC code constructed based on the lines and points of the m-dimensional Euclidean geometry EG(m, 2s). The parity-check matrix of this code consists of lo rows and 11 columns, where lo is given by (17.9), and 11 = 2 111 s - l. The rows of H~b,c can be grouped into K 11 x 11 circulant submatrices, H1, H2, ···,HK, where K = lo/11 = (2(m-lls - 1)/(2s - 1) given by (17.20). Each circulant submatrix H; is obtained by cyclically shifting the incidence vector of a line (not passing through the origin) n times. Therefore, can be put in the following form [17]:

JH!~b.,

IHI~b.,

(1)

HEG,c

[:~ l

(17.64)

JH!~b., into q columns in a manner similar to that described earlier in this section; however, the 1-components in a column of H~b.c

Now, we split each column of

Section 17.7

Construci:ion

must be labeled in a

Column and Row Splitting

For O .:': j < n. let

11119

!l;ji) be the jth

column of the ith circulant submatrix JHl;. column !];J of lh'I~i.c consists 2 !!'!OJ ~ !!'!( 'D)il(l , ·1J11i' "h nne O" of the other. VVe label the 1-components of of 'B] Oj l ... ~ Gj , .!L

O

l

I

U

_

.·",-:.

the

colurnm f,J of JHI~b.c as foHows. ~Ne label the first

column

!];\1) on or below the main of the

colurun ff.! of

of the jth

line of circulant lrl1 and inside IHI1 as the '\Ne label the first 1-component

of second

line of circulant and inside IHI2 as the of . VVe continue this process until we label the first _{'{Sfl(l,n,me>nt of '1%,t) on or below the main line of circulant IHI[( and inside IHIK as the Kth of column ·vve cmne back to circulant IHI1 and start the second round of the prncess. 'We label the second 1-component of g\1l belovv the main as the +1)th 1-component of We label the second

below the main line of circulant of . VVe continue the second round of the labeling process untii we reach the Kth circulant IT-iIK again. Then. we loop back to circulant Im 1 and continue the process. the labeling process, whenever vve reach the bottom of a circulant matrix IHI;. we wrap around to the top of the same column process continues until an the 1-components of :l¥i is completed, we distribute of '/I,J are labeled. the 1-components of g1 into q new columns in the same rotating manner as described earlier in this section. the and distribution can be carried out at the same time. Let be the nev1 n:iatrix from the column splitting.

IHI2 and inside iliI2 as the

Ih1~b.ext consists of K 11 >< nq subnrntrices · · ·. IHiext.K. For 1 :S i :S are shifts of the first rnw q bits at a time. As a result, the null space of ftJ~b.exr LDPC code in quasi-cyclic form. To extend the m-dimensional (0. s )th-order EG-LDPC code. we use the (17.21) ( or K. the rows

(17.22)). Then. ·we . .b uted in a d1stn

into q columns with column weight 1 . b nrnnner. 'We also can take tfle o f IHI(ll EG.c given y (17.64) as the matrix and then each column. For a PG-LDPC code. the number J of rovvs of its matrix may not be divisible the number n of columns of the matrix. fo this case, not all the submatrices of the rnatrix IHI~b of a PG-LDPC code can be arranged as n >< n square circulant matrices. :Some of them are nonsquare circulant as shown in 17 .8. The rovvs of such a nonsquare circulant matrix are still shifts of the first row. and the number of rows divides n. We can remove all the nonsquare circulant submatrices from lHI~b- The result is a new parity-check matrix

J

890

Chapter 17

Low-Density Parity-Check Codes

in which every submatrix JH[i is an n x n square circulant matrix.

JHI~b.c1 is still a low-

density parity-check matrix that :;atisfies aH the conditions given. ia Defiaitiou 17.1. The null space of H~b c1 is a cyclic supercode of the PG-LDPC code generated by

H~b. Then, we can ext~nd this new code by column splitting as described earlier, in a rotating and circular manner, and the resultant extended code will be in quasi-cyclic form. The null space of the transpose [JHI~b_c1Y of H~b.c1 gives a shortened type-H PG-LDPC code that is quasi-cyclic. Then, we can obtain extended quasi-cyclic LDPC codes by column splitting of [lHl~b c1 F. As an example, consider the parity -check matrix lHl~b of the (85, 24) threedimensional type-I PG-LDPC code cn(3, 0, 2) given in Example 17.8. There is only one nonsquare circulant submatrix in JHI~b. Removing this nonsquare circulant submatrix from H~b, we obtain a new matrix 0

HU~);,

~ ill [

l

Each submatrix in JHI~b.c1 is an 85 >< 85 circulant matrix. The null space of !HI~~.d gives the same (85, 24) LDPC code with a minimum distance of at least 21. We can obtain extended codes by splitting the columns of!HI~~.d· The null space of [JHI~~.c1F gives a (340, 279) LDPC code with a minimum distance of at least 6. We also can obtain LDPC codes by splitting each row of the parity-check matrix H of a base finite-geometry LDPC code into multiple rows. The resultant code has the same length as the base code but has a lower code rate. Furthermore, proper row splitting also preserves the cyclic or quasi-cyclic structure of the code. Clearly, we can obtain LDPC codes by splitting both columns and rows of the parity-check matrix of a base finite-geometry LDPC code. EXAMPLE 11.17

= 2 ands = 4, the two-dimensional type-I (0, 4)th-order cyclic EG-LDPC code is a (255, 175) code (the third code in Table 17.1). Hs parity-check matrix lHI has the following parameters: p = 16, y = 16. and density r = 0.0627. Hs error performance with SPA decoding is shown in Figure 17.15. U we split each column of its parity-check matrix I-I! into five columns and split each row into two rows. we obtain a parity-check matrix JHI' whose columns have two different weights, 3 and 4. and whose rows have the same weight 8. The null space of this new parity-check matrix is a (1275, 765) LDPC code with rate 0.6 and density r' = 0.00627. The performance of this new code with SPA decoding is also shown in Figure 17.15. His a relatively short code, but at a BER of 10-4. it performs 1.8 dB from the Shannon limit.

Form

EXAM P'llE U .18

Consider the (4095. 3367) two-dimensional type-I cyclic (0. 6)th-order EG-LDPC code c~b(2, 0, 6) given in Table 17.1. If we split each column and each row of

Section 17.

Code Construction

ancl Rovv

J?

~ 10-"

,-g

Q..

2 o/

.~

10

u 0

€

iii

Shanntn1 lin1it for { 1275. 765)

-2

-1

2

0

5

6

£ 1,/N11 (dB)

!CIGURE '17.15: Bit- Rnd block-error code and 1he

-----r·-----i------r--i----,-----r··-----

10"

I

··--- .... BPSK 1wcodeci -------· E;;lended LD?C code BER ---·::--- Base LDPC co'-k BER __

\

!O

~o Z ()

10

!

6..

10

.

."i

=

JO -r,

-

'

I

I

I \\

i.

:1

J,00"

r. . _J______ limil l ____L______

____ _L _ _ __J_)L _

0

1

2

~

4

5

L__ _ _ _-

6

7

E1.IN11 (c!B)

FIGURE 1 .16: Bit- and block--error of the code and the extended EG-LDPC code based on SPA '""-·"~''"'''u,".

892

Chapter 17

Low-Density Parity-Check Codes

the parity-check matrix of this code into 16 columns and 3 rows, respectively, we obtain a new parity-check matrix with column weight 4 and row weights 21 and 22. The null space of this new parity-check matrix gives a (65520, 53235) extended EG-LDPC code. This extended code and its base (4095, 3367) code have about the same rate. The error performance of the extended code is shown in Figure 17.16, and it performs 0.7 dB from the Shannon limit at a BER of 10-5; however, the performance of the base code is 1.40 dB from the Shannon limit. This example shows that by a proper combination of column and row splittings of the parity-check matrix of a base finite-geometry LDPC code, we can obtain a new LDPC code that has about the same rate but better error performance.

11.8

BRIEAK]NG CYCllES iN TANNER GRAIPHS

The examples given in the previous section show that properly splitting each column of the parity-check matrix H of a finite-geometry LDPC code C into multiple columns results in an extended LDPC code Cext that performs very close to the Shannon limit with SP A decoding. A reason for this is that column splitting reduces the degree of each code-bit vertex in the Tanner graph Q of the base code and hence reduces the number of cycles in the graph. Splitting a column of H into q columns results in splitting a code-bit vertex of the Tanner graph g of the base code into q code-bit vertices in the Tanner graph 9ext of the extended code Cext· Each code-bit vertex in 9ext is connected to a smaller number of check-sum vertices than in Q. Figure 17.17(a) shows that splitting a column in H into two columns results in splitting a code-bit vertex in the Tanner graph Q into two code-bit vertices in the Tanner graph 9ext· The original code-bit vertex has a degree of 4, but each code-bit vertex after splitting has a degree of 2. This code-bit splitting breaks some cycles in the Tanner graph Q of the base code C. Figures 17.lS(a) and 17.19 show the breaking

V

--,,>

+

+

+

+

(a) Column splitting

s

(b) Row splitting

FIGURE 17.17:

Graph decomposition by column/row splittings.

Section 17

in Tanner

V,

((xJ ~~<\){;/

0

~ S1

(a)

~ S2

C±)

+

Si

S,

a cycle of length 4 by column splitting operation.

(b) Breaking a cycle of length 4 by row splitting operation. F!GURE "7.1

'U,

FIGURf:: 7.1

of

V_-;

column of a base and results in an extended

LDPC code ·whose Tanner the Tanner matrix for any linear block code.

894

Chapter i 7

Low-Density Parity-Check Codes

!EXAM PLIE 11.19

Consider the (7, 4) cyclic Hamming code generated by polynomial g(X) = 1 + X + X 3 . Its parity-check polynomial is h(X) = 1 + X + X 2 + X 4 , and its dual code is generated by X 4 h(X- 1 ) = 1 +x 2 +x 3 +x 4 . A parity-check matrix for the (7, 4) cyclic Hamming code can be formed by cyclic shifting the vector (1 0 1 1 1 0 0) six times, as shown in Figure 17.20. The Tanner graph for this code is shown in Figure 17.21. It contains 21 cycles of length 4, and each code-bit vertex is on 6 of these cycles. If we split each column of IHI into two columns, we obtain a new parity-check matrix IH!ext, as shown in Figure 17.22. The null space of Hext gives a (14, 8) code. From lH!ext, we can easily check that no four l's appear at the four corners of a rectangle in IH!ext· This implies that the Tanner graph of the extended code does not have any cycle of length 4. Figure 17.23 shows the error performance of this extended Hamming code using SPA decoding, which is almost identical to MLD performance. As another example, consider the (23, 12) Golay code. A parity-check matrix lH! for this code is shown in Figure 17.24. Based on this parity-check matrix, the Tanner graph for this code has a total of 1748 cycles of length 4, and each code-bit vertex is on 152 such cycles. The error performance of this code with SPA decoding 1011100 0101110 0 0 10 l 1 l H= 1001011 1 1 0 0 1 0 l 1 1 1 0 0 l 0 0 1 1 J OO1 FIGURE 17.20:

g(X)

A parity-check matrix for the (7, 4) Hamming code generated by

= 1 + X + X 3.

FIGURE 17.21:

Tanner graph for the (7, 4) Hamming code.

Section 1

in Tanner Graphs

:€!!ll5

1000 10 10 100 00 0 0 0 10 000 10 110 0 0 0 0 0 0 0 1 0 0 l OO1 1 0

irn"" = o 1 o o o o 1 o o o 1 o o 1 100 10 00 00 10 0 l 0 0 1 10 100 0 00 0 10 0 0 0 010 10 100 0 00 1

rnatrix for the

FIGURE 17.22:

10° [---r--r---r--·-1

---,---1-

=

: 10-

i l

1

10- 2

0

.

-

8) extended

~ BPSK uncoded ~- SPA BER maximum iteration 200 ~,:,-- S. PA FER maximum iteration 200 -~ MLD BER ~:p MLD FER 0

~-----------~

~~~

t l

l5..

3 t: '\'

"D 10-, 0

~

co

_L___l_ _ _~ _____ I _____L _____. ~ - - ~ - - - ~ - - ~ l 2 3 4 5 6 7 9 8

FIGURE 'I .23:

Error

of the

8) extended LDPC code.

with 200 maximum iterations is shm,vn in 17.25. We see that at BER = 10-5 , with 200 iterations, the error with SPA for this code is 0.6 dB away from MLD, which demonstrates how of length 4 prevent the SPA decoding algorithm from converging to MLD Suppose we split each column of the matrix in 17.24 into two columns in a random manner. We find that this column splitting results in a (46, 24) LDPC code with a minimum distance of 5 whose Tanner contains only 106 cycles of length 4. The weight distribution of this code is given in Table 17.3. We see that the extended code has a very small number of codewords of low weights: 5, 6, 7, and 8. There are 82 codewords of these four low weights, which is fewer than the 253 minimum-weight-7 codewords of the 12) Golay code. We see that column

896

Chapter 17

Low-Density Parity-Check Codes

Jl-ll=

10100100111110000000000 01010010011111000000000 00101001001iill00000000 00010100100111110000000 00001010010011111000000 00000101001001111100000 00000010100100111110000 00000001010010011111000 00000000101001001111100 00000000010100100111110 00000000001010010011111 10000000000101001001111 11000000000010100100111 11100000000001010010011 11110000000000101001001 11111000000000010100100 01111100000000001010010 00111110000000000101001 10011111000000000010100 01001111100000000001010 00100111110000000000101 10010011111000000000010 01001001111100000000001

FIGURE 17.24: A parity-check matrix for the (23, 12) Golay code.

100 < = _ ~ - ~ - - ~ - - ~ - - - , - - - - , - - - - , - - - - - , - - - - - - , - - - - = -

- - BPSK uncoded SPA BER with maximum 200 iterations ~ Hard-decision BER ~MLDBER ~

10- 0

'------'--------'--------'-----'----~----'-

o

2

3

4

5

6

7

8

9

E1,/N0 (dB)

FIGURE 17.25: Bit-error performance of the (23, 12) Golay code with SPA decoding.

Section 7.8 TABLE 17.3:

Wei\1l;!httl

distribution of the extended

N@!lli!lli:Dll:Ir (\j)l[ (\'.({j)@<eW({i)Jr(lli§

0

1

23

0 0 0 0

24

,, J

4 5

6 7 8 9 10 11

12 13

14 15

16 17

18 19 20 21 22

5 18 58 280 1000 3]51 9034 24069 57390 122262 237105 417751 671322 989121 1336148 1656075 1882241

"""H'"'"·

we can

Nurnmnll:Derr

1 2

code

25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46

(u)ll ,1:illl@ll:W([J;Ira:i},

1963152 1882241 1656075 1336148 989121 671322 417751 237105 122262 57390 24069 9034 3151 1000

280 58 ]8

5 1 0 0

0 0 l

effect The bit-error ·~Q,--,-~··1·,·1,un.•vo of this and MLD are shown in

matrix JFfI. each column of JHI in a different manner and into different mnnbers many extended LDPC codes can be columns of the matrix JHI. Of course., if columns of the resultant extended code is an LDPC code. to

in Figure can also break many check-sum vertex splitting is shown in a combination of column and row will break many of the base this may result in a very good LDPC code.

898

Chapter 17

Low-Density Parity-Check Codes

I

I

-BPSK - BitIDBP - BlockIDBP ..;;,- Bit MLD ...;,.. BlockMLD

E E

10-2

"'

..0 0 .... 0..

.... 10-,

0 .... .... a)

-

~ () 0

~

iii

10-4

FIGURE 17.26: Error performance of the (46, 24) extended Golay code based on SPA decoding with 200 maximum iterations and MLD.

17.9

SHORTENIED FINiTIE-GIEOMHRY U)PC CODIES

Finite-geometry LDPC codes can be shortened to obtain new LDPC codes, by deleting properly selected columns from their parity-check matrices. For a type-I finite-geometry LDPC code, the columns to be deleted correspond to a properly chosen set of points in the finite-geometry base on which the code is constructed. For a type-II finite-geometry LDPC code. the columns to be deleted correspond to a chosen set of lines in the finite geometry. Several shortening techniques are presented in this section. First, we consider shortening type-I finite-geometry LDPC codes. We use a type-I EG-LDPC code to explain the shortening techniques. The same techniques can be used to shorten a type-I PG-LDPC code. Consider them-dimensional type-I 1 b.c.(m, 0, s) constructed based on the mcyclic (0, s)th-order EG-LDPC code dimensional Euclidean geometry EG(m, 2·1 ). Let be the parity-check matrix

ck

lH!~b ,

JH!~b.c

are the incidence vectors of the lines in of this code. Then, the rows of EG(m, 2 1 ) that do not pass through the origin of EG(m, 2 1 ), and the columns of c correspond to the nonorigin points of EG(m, 2s). For 1 :S q < 2s, let S be a set of q parallel (m - 1)-flats in EG(m, 2·1·) that does not contain the origin (i.e., the origin of EG(m, 2s) is not contained in any of the q parallel (m - 1)-flats). Because each (m -- 1)-flat in EG(m. 2 1 ) contains 2(m- l)s points, and the q (m - 1)-flats in S

lH!~b

Sr::clion 1

Shortened

are the total nun1ber of in S is q2(m-l)s. Because each S contains contains 2<m- 21s(2<m-l)s -1)/(2' -1) lines in

Ji= zem-2)scztm-l)s - l)q/(2s

-1)

(17.65)

we delete the With this deletion of to the lines contained in S become rows of zeros we the rows of zeros from the - 1 columns and Jo - Ji rovvs, column in this nevv matrix

still has the but its rows lines that are

outside of S are 2s ~ hovvever ~ the to the lines

S are less than 2s. Because each row in

line in in cornrnon, and the ,:tahix and its null space an · 1 ' I " d b rCl) ( 0 - -, c.enote y '-- EG.s m. . H is a shortened code of the EG-LDPC code 1 c(m, 0, s). The minimum distance of this shortened LDPC code is stiH at least (2ms - l)/(2s ·,v·w,...,v•u,,u,

ck b

EG-LDPC code Consider the two-dimensional (0, EG(2, third constructed based on the two--dimensional Euclidean code in Table 17.1). This code has rate 0.6863 and a minimum distance of 17. The matrix of the code is a 255 x 255 square circulant 1:1"tatrix whose rows are the incidence vectors of the lines in EG(2, that do not pass the The column and row of this matrix are both 16. Let S be a set of tvvo lines in that do not pass S contains 32 Because each line in consists of 16 that to the points in S. This deletion we delete those columns in to the two lines two rows of zeros that in S. these two ro'vvs of zeros, we obtain a 252 >< 223 matrix ]H[E(lG) ,5,. AH the columns of this new matrix still have the same but its rov,s have two different Thirteen rows of ba,,e have 14. The null space of this new matrix LDPC code with a minimum distance of at least 17 and rate 0.650. The puncturing removes 30 information bi.ts and tvm The error of this shortened EG-LDPC code is shown in 17.27. VVe see that the shortened code and the A

technique is to The

900

Chapter 17

10-1

? ii

"'0 °''" 0'"

Low-Density Parity-Check Codes

L

10-2

,.0

<-< >-<

10-1

l

!

i:r:i 10-4

'- - "- - '~~J~

10--'o 0.5

1.5

2

2.5

3

3.5

4

4.5

5

Et,IN11 (dB)

FIGURE 17.27: Bit-error probabilities of the (255, 175) EG-LDPC code and its (223, 145) punctured code.

in S are not necessarily parallel to each other; some of them may intersect. Then, the puncturing of the parity check matrix of the m-dimensional type-I EG-LDPC code is based on the points contained in S. The puncturing still does not change the column weight but results in many different row weights. The punctured matrix with the rows of zeros removed generates a shortened EG-LDPC code. One approach to shortening them-dimensional type-H quasi-cyclic EG-LDPC code constructed based on EG(m, 2") is to put its parity-check matrix in

H~b,qc

circulant form. In this case,

Hib,qc is simply the transpose of H~b.c given by (17.64),

Hib,CJC = [H~LY = [Hf, H{, ... ' Hf J,

(17.66)

where K = (2(m-l)s - 1)/(2' - 1). Because each submatrix H, is an n x n square circulant matrix, its transpose is still an n x n square circulant matrix, where n = 2ms - 1. Each column of is the incidence vector of a line in EG(m, 2·'') not passing through the origin and has weight 2·1·• Because is a square circ11lant, its rows and columns have the same weight 2·'. For O < q < K, suppose we remove q circulant submatrices from given by (17.66). The result is a punctured

H; H;

H;

Hib.q,

matrix, denoted by H(E2G) ,qc,., . , with n = 2111 s - 1 rows and (K - q)n columns. The column and row weights of this new matrix are 2' and (K - q)2', respectively. The null space of this new matrix gives a regular LDPC code of length (K - q )n and minimum distance of at least 2s + 1. The new LDPC code is still quasi-cyclic.

LDPC Codes

Shorten,ed

Section '17.9

!ll«:vi

= s = 3, the type-H EG-LDPC code constructed based on the three-dimensional Eudidean EG(3, 23 ) is a code ·with a minimum distance of at least 9 and rate 0.92. This code was considered in and its error :is shown in 17.9. The check matrix ItTii~.qc of this code is a 511 >< 4599 matrix. In circulant form, this matrix consists of nine 511 >< 511 circulant submatrices. we remove from this matrix in a 511 >< 4088, a 511 x matrix. The null spaces of these three matrices a 2694), and a (1533, shortened EG-LDPC All three codes have a minimum distance of at least 9, and their rates are 0.878, and 0. The bit-error of these three shortEG-LDPC codes and the base code are shown in 17.28.

Form

~ , u ~ u ~ u to LDPC codes results in a

with a lower rate.

10-1 "'""==

·==·=

~- - BPSK uncoded _,,,, (4088, code with Shannon limit 13,t dB , -<::- (3066, 2694 code with Shannon limit 2.89 dB : (1533. 1161) code with Shannon limit 1.68 dB ~ , - ""''"'-sz;"' - ( !599. 4227) code with Shannon lnmt 3 52 dB

3716l

=~- '"'- -

10-"

\\ \\\

~ m "'0 ..0 ,_ B

n.

,.i3 C,

10- 4

\

\

!

ii

,- 1·

\

\ \',

\

10-)

10-h

~--~~~~-~~~~~~~~,I__~~~~~~~~~~

2

3

4

5

6

7

8

E1,IN0 (dB)

Bit-error of the (4088, 3716), (3066, and (1533, shortened EG-LDPC codes and the type-H three-dimensional EG-LDPC code with SP A decoding. FIGURE 17.28:

902

17.10

Chapter 17

lovv-Density Parity-Check Codes

CONSTRUCTION Of GALLAGER UJil?'C CODES

To construct the parity-check matrix IHicA of a Gallager LDPC code as desc;ibed in Section 17.1, we first need to construct a k x kp submatrix H 1 with column weight 1 and row weight p, and then we need to find y - 1 permutations to permute the columns of H1 to form y - 1 other k x kp submatrices H2, lHl3, · · · , Hy. With these y submatrices we form the parity-check matrix He A as follows:

:J H, l

HaF [

The column and row weights of IHicA are y and p, respectively. The density of He A is 1/ k. For large k, He A is a sparse matrix. The null space of He A gives a Gallager LDPC code. The y - 1 permutations must be chosen in such a way that the code generated by the parity-check matrix lHicA has a good minimum distance and does not contain cycles of short lengths in its Tanner graph. especially cycles of length 4. Gallager did not provide any specific way of finding these permutations, and there is no known method for constructing these permutations to guarantee that no short cycles exist in the Tanner graph of the code to be constructed. Commonly, computer searches are used to find good permutations and good Gallager codes. In this section we present a method for constructing Gallager codes. This method is based on the parallel structure of lines in a Euclidean geometry [51, 52]. Consider them-dimensional Euclidean geometry EG(m, 2s) over GF(2 5 ). As described in Sections 8.7 and 17.4, the lines in EG(m, 25 ) can be grouped into c2ms - 1)/(2S - 1) parallel bundles, With each parallel bundle COnSiSting Of 2(m-l)s lines that are parallel to each other. The parallel lines in each parallel bundle contain all the 21175 points of the geometry EG(m, 2s), and each point is on one and only one of these parallellines. Let P1 , P2 ,. · · , denote the (2 111 s -1)/(2s -1) parallel bundles. For 1 :s i :s c2ms - 1)/(2s - 1), we form a 2<m-l)s X 21115 matrix H; whose columns correspond to the 2"" points in EG(m, 2s) and whose rows are the incidence vectors of the lines in the parallel bundle P;. This matrix is called the incidence matrix of the parallel bundle P;. The weight of each row of lHI; is 2s. Because the lines in each parallel bundle are disjoint, the weight of each column of lHI; is 1. The number of columns in H; is p = 2s times the number of rows in lHI;. Therefore, the matrix lHI; is in Gallager's form. For 1 :s y :s (21111 - l)/(2s 1), we form the following matrix:

IHIEG.GA

=

(17.67)

Then, HEG.GA is a low-density parity-check matrix in Gallager's form as presented in Section 17.1. All the columns of JHIEG.GA have the same weight y. and all the rows of HEG,GA have the same weight 2'. The submatrices lHI2, lHI3, · · · , Hy are actually column permutations of lHI1. The null space of lHIE(; CA gives a regular Gallager LDPC code. Because the rows of JHIEG.GA are incidence vectors of y2<m-l)s

intersect at one have more than the the matrix of length 4 in its Tanner graph. Because there are y rows in on each code no y columns can be added to zero. the mininrnm distance of the code is at least y + 1. fo the minimum distance of an LDPC code must be even (see Problem As a result ,;Ne have the lower bounds on the minimum LDPC code: for odd y. for even y.

lower bounds may be very loose for some

as we wiH see in

For each choice of tvvo m ands. the construction based on the bundles of lines in EG(m, a sequence of 2111 " for various y 's. This sequence of codes has various rates and minim urn distances. Small y 's result in high rate and y 's result in codes with lower rates but larger minimum distances. For y = (2 111 s -- 1) / (2" - 1). the (0, s )thGallager code the matrix of (17 .67) is the 111.-dimensional order EG-LDPC code oflength 21111 • For y = 1 and 2, the codes have minimum distances of 2. and 4, In code y is chosen greater than 1 to ensure that the resultant code will have a minimum distance.

The two-dimensional Euclidean consists of 72 lines m-r,aw,orO into nine bundles. Each bundle consists of For y = 2 to 9, we can construct and each line consists of codes of length 64 with various rates and minimum distances. Table 17A. The true minimum distances of these codes are their distributions. Consider the code constructed bundles. This code is a 41) code -with a minimum distance the lower bound on the minimum distance is 6.

IE1{/\fV!rU: H "23

EG(3, 23 ) over GF(2 ). This consists of 512 and 4672 lines. The 4672 lines can be grouped into 73 parallel bundles, with each parallel bundle consisting of 64 lines parallel to each other. For each parallel we can construct a 64 >< 512 matrix whose rows are the incidence vectors of the lines in the bundle. Suppose we take 6 parallel bundles to construct a Gallager code. the parity-check matrix HEG.GA of the code is a 384 x 512 matrix that consists of six 64 x 512 submatrices arranged in a column. with each submatrix constructed based on a bundle of lines. The column and row weights of HEG.GA are 6 and 8, respectively. The Let m

3

= s = 3. Consider the three-dirnensional Euclidean

904

Chapter 17 TABLE 17.4:

y

2 3 4 5 6 7 8 9

Low-Density Parity-Check Codes

EG-Gallager codes of length 64 constructed based on EG(2, 23 )

Codie

(64, 49) (64,45) (64, 41) (64. 40) (64,39) (64,38) (64,37) (64, 37)

JLoweir fuio1mllld Ollll ii1111Illl1lllii111llliim di\§fam::e

4 4 6 6 8 8 10 10

'firl!Jleminmi\lllnll!Jlm dlfat:E!llllll:te

Nl!Jlll1111lbieir of mi.llllillinnunnm· weiglhn\ lCoidiewoirdl§

4 4 8 8 8 8 10 10

18816 112 5880 3416 1680 560 12544 12544

null space of this matrix gives an EG-Gallager code of length 512. There are many possible choices of 6 parallel bundles; each choice results in an EG-Gallager LDPC code of length 512. Different choices may result in different code dimensions. One choice gives a (512. 256) EG-Gallager code with rate 0.5. Because y = 6. the lower bound, y + 2, gives a minimum distance of the code of at least 8; however, the true minimum distance of the code is proved to be 12. The bit-error performance of this code with SPA decoding is shown in Figure 17.29 (assuming BPSK signaling). Also included in the figure are the bit-error performances of three computer-generated LDPC codes of the same rate and the same (or almost the same) length. All three codes are constructed based on the rules given in [10) (see Section 17.14). The first two computer-generated LDPC codes have length 512. The parity-check matrix of the first computer-generated code has column and row weights of 3 and 6, respectively. The parity-check matrix of the second computer-generated code has column and row weights of 4 and 8, respectively. The third computer-generated code is a (504. 252) code whose parity-check matrix has column and row weights of 3 and 6, respectively. This (504, 252) LDPC code is the best-known computer-generated LDPC code of length 504. From Figure 17.29 we see that the (512. 256) EG-Gallager LDPC code outperforms all three computer-generated LDPC codes. Also included in the figure is the bit-error performance of the well-known NASA standard rate-1/2 64-state convolutional code with Viterbi decoding (listed in Table 12.1). This code has been widely used in commercial communication systems for error control. We see that all the LDPC codes outperform the convolutional code.

IEXAMPllE 11.24

Let m = 2 ands "--' 6. The lines in the two-dimensional Euclidean geometry EG(2,2 6 ) can be grouped into 65 parallel bundles, each consisting of 64 lines parallel to each other. Each line in EG(2, 26 ) consists of 64 points. Many Gallager LDPC codes of length 4096 can be constructed based on various choices of parallel bundles. Figure 17.30 shows the bit-error performances of several Gallager LDPC codes constructed based on the choices of y = 6. 8, 10, 32, and 65 parallel bundles. The codes are (4096, 3771), (4096, 3687). (4096, 3643). (4096, 3399) and (4096. 3367)

Sr~ction

~

lO"

r

-

=

[

~ 10 _,

J

I

~

L

j

'~

. -. , :1 -~

j

i

10

~C»S:

-]~'

10'~

o.

s

LDPC Codes

-,~ EG-Ga!lager (512,256) code, y = 6 . ~,'°" Computer-generated (512. 256) code, y = 3 ~-s- Computer-generated (512,256) code. y = 0! = •s, =" ~ ? 1'/lacKay (504. 252) code. y = 3 =,,,, Rate-1/~ :~4-state covolutional c o ~

r · b; =

Construction

----l---i-·--i-=-=- Uncoded BJSK

~

10-1

. 0

--1

-d

iii

JO 10

7

I.____ J___ ___L ___ ~_ _ _ J__· - - ~ - - ~ - - - ~ - 0 l 2 3 5 7 6

8

E 1,/N11 (dB)

of the 256) BG-Gallager several and a rate-1/2 64-state convolutional code.

10-'

IQ

:~' .J'..:;

"'

_s

10-1

0,_

"

§' !

10-"

m 10·-h ~- - Uncoded BPSK -;-, 6 bd( 4096. 3771) R = 0.92 L = 3.55 dB -s bcl( 4096. 3687) R = 0.90 L = 3.20 dB ""-10 bd( 4096. 3643) R = 0.89 L = 3.04 dB -,:= 32 bcl( 4096. 3399) R = 0.83 L = 2.32 dB -65 bd(4096, 3367) R = 0.82 L = 2.25 dB _

10

10

0

'

2

3

4

5

6

7

8

9

E 1,/N11 (dB)

FIGURE 17.30:

Bit-error

of several BG-Gallager codes of and bd stands for parallel bundle.

10

906

Chapter 17

Lovv-Density Parity-Check Codes

codes, respectively. Consider the (4096, 3687) code constructed based on 8 parallel bundles oflines inEG(2, 26 ). This code has rate 0.9 and a minimum distance of at least 10. With SPA decoding, it performs 1.2 dB from the Shannon limit at BER= 10-5 . The lines in a projective geometry do not have the simple parallel structure of the lines in a Euclidean geometry; hence, LDPC codes in Gallager form cannot be constructed. 11.11

MASKED IEG-GALLAGIER LDP( CODES

The parity-check matrix IHl.EG,GA of an EG-Gallager code given by (17.67) can be masked to form a new parity-check matrix for a new LDPC code [53]. For masking, we need to decompose the incidence matrix of a parallel bundle of lines in a Euclidean geometry into a row of permutation matrices. Again, consider the m-dimensional Euclidean geometry EG(m, 25 ). For O :S µ,:Sm, a µ,-flat in EG(m, 2s) is simply a µ,-dimensional subspace of the vector space V of all the m-tuples over GF(2s) or its coset (Section 8.5). Therefore, a µ,-flat consists of 2µs points. Two µ,-flats are said to be parallel if they correspond to two cosets of a µ,-dimensional subspace of V. The 2Cm-p.)s µ,-flats corresponding to the 2<m-µ)s cosets of a /..ldimensional subspace of V are said to form a parallel bundle of µ,-flats. The µ,-flats in a parallel bundle contain all the points of EG(m. 25 ), and each point is on one and only one of the µ,-flats in the parallel bundle. Let P(m, m -1) be a parallel bundle of (m -1)-flats in EG(m, 25 ). This bundle consists of 2 1 parallel (m - 1)-fiats, denoted by F1. F2, · · · , F2,. Each (m - 1)-flat F; in P(m, m -1) consists of2Cm-lJs points and 2<m-2).1

cz<m-l)s - 1)/(2' - 1)

lines. The total number of lines in P(m, m - 1) is Ii= z<m-l)sc2(111-l)s -1)/(2·' -1). The lines in P(m, m - 1) form (2(m-l)s - 1)/(2s - 1) parallel bundles in EG(m, 2s). Because there are J = 2<m-l)sc2111s -1)/(2s -1) lines in EG(m, 21 ), there are

lines in EG(m. 2 1 ) that are not contained in P(m, 111 - 1). Let W denote the set of lines not contained in P(m, m - 1). A line in W contains one and only one point from each of the 2s parallel (m - 1)-flats in P(m, m - 1), because (1) P(m, m - 1) contains all the points of EG(m, 2s), and (2) if a line has two points on an (m -1)-flat F; in P (111, m - 1), the line is completely contained in F;. The lines in W connect the parallel (m-1)-flats in P(m, m-1) together, hence, we call them connecting lines with respect to the parallel bundle P (m, m - 1), and they form 2(m -- lls parallel bundles of lines in EG(m, 28 ), called connecting parallel bundles of lines, Q1, Q2, · · · , Q 21,,,-1i,. For 1 :s i :s 2(m-l).,·, let Ai be the 2<m-l)s X 2ms incidence matrix of the connecting parallel bundle Q;. For 1 :s j :s 2·1 , it follows from the structure of

Section ·1

·1 ,

Masked

LDPC Codes

9ff7

lines in W that the 2(,n-l)s columns of A; that """'"''' ""n F; in P m - 1) form a 2(m-l)s x 2(m-l)s (a ~~,.uuu, A; consists of 2" . For 1 :::c y :::c 2(m-lls, ma trices of dimension 2 (m - l ).,·, denoted y2(m-l)s x 2 111 s matrix: ,ve form the 0

TillEG.GA

~ 'J-1. ,-

l

Ay

in such vvay that the first 2<m-l)s columns vve order the columns of the second 2Cm-l)s of the first columns to the m -1), · · ·, and the last 2 1m-l)s coimnns r,n·n'Q,c·.,~~n in we can ]H[EG.GA into

Au, [ lliIEG.GA

=

A11 A21

A2,2

J

:.

Ay.1

Ay.2

(17.69)

which has column and row weights of y and 2', For 1 :s y :s 2", the null space of an LDPC code with a minimum distance of least y + 1 y + 2) and rate of at least - y) /21 •

Consider the two-dimensional Euclidean EG(2,2 2 ) over the field GF(2 2 ). This geometry consists of 16 and 20 lines. Each line consists of 4 and each bundle of lines consists of 4 lines. The 20 lines of bundles. In Example 8.15 we showed that the Galois field as an extension field of is a realization of EG(2, 22 ). Let 0/ be a primitive 00 0/ = 0. 0/o = 1. 0/. 0/ 2 , · · · , 0t 14 form the 16 of eiement of EG (2. 4 lines: L1

L3

= =

(0, 0/4, 0/9, 0/14}, {0i2,0i10,0111,0113}.

L2

=

{Olo, Ot, 0/3, 0/7}, = {CIS,C/6,0/8,Cll2}

form a bundle P(2, 1). The other 16 lines in EG(2, 22 ) are lines bundle P(2. 1). These 16 connecting lines with respect with respect to the to P (2, 1) form four connecting Q 1, Q2, and These four bundles can be constructed Table 8.5. For 1 :s i :s 4, the matrices

908

Chapter ·17

Low-Density Parity-Check Codes

as follows: L1

0 (¥4 (Y9 (¥14 A1

1 0 0 1 0 0 0

{\ V

A2

A3

A4

1 0 0 0

0 1 0 0

1 0 0 0

0 0 0 1

1 0 0 0 0 0 0 1

0 0 0

1 0 0

1 0 0

1 0 0

0 0 1 0

L2 -:, /) av a a~ a 7

L3

0 0 1 0

1 0 0 0 1 0 0 0 1

0 0 0

0

()

()

()

V

V

1

(\

V

_J_

V

0 0 0

0 1 0 0

1 0 0 0 0 1 0 0

0 0

0 0

0

1 0

0 0 0

1 0 0 1 0

0

0

0 0

1 0 0

0 1 0 0 1 0 0

0 0 0 1

1 1 0 0 0 1 0 0 0 0

1 1 0 0 0

0 0

0 1

L4

a2 alO all (¥13

1 0

1 0

0

0 0 1

0 0 0

(\

{\

V

V

1

1

a.)C a 6

1 0 0 0

l

0 1

1

0 0 0

1 0 0 0

0 0 0 1

0 1 0 0

0 0 1 0

0 1 0 0

0 0 1 0

0 0 0 1

1 0 0 0

0 0 1 0

0 0

0 0 1 0

0 0

0 1

0

(Yo

a12

0 0 0 1

0 1 0 0

0 0 1 0

1 0 0

0 0 0

0

1

0 1 0 0

0 0 0 1

0

0 1 0 0

1 0 0

1 0 0 0

1 0 0 0 0 0 1 0

Let Z = [z;.}] be a y x 2' matrix over GF(2). We define the following product of z and lH!EG,GA:

M~Z®llil~[

z1,1A1,1

z1,2A1,2

z2.1A2.1

z2.2A2.2

Zl,2' A1,2, ] z2.2,A2.2,

. Zy,lAy,l

'

(17.70)

Zy,2'' Ay.2s

Zy,2Ay.2

where for for

=

l,

Zi,j =

0,

Zi,j

(17.71)

and O is a 2(m-l)s x 2(m-l)s zero matrix. This product defines a masking operation for which a set of permutation matrices in lH!Ec,GA given by (17.69) is masked by the zero-entries of Z. The distribution of the permutation matrices in M is the same as the distribution of I-entries in Z. H the column and row weights of Z are a and /3, then M is an (a, /3)-regular matrix with column and row weights of a and /3, respectively. If Z has different column weights or different row weights, then Mis an irregular matrix. Z is called the masking matrix. HEG,GA the base matrix, and M the masked matrix. Masking reduces the density of the base matrix and hence reduces the number of cycles in the Tanner graph of the base matrix. Consequently, the associated Tanner graph of the masked matrix has a smaller number of cycles than

Section 'I .11

Masked

LDPC Codes

:\li]!lJ

the Tanner of the base matrix 4. has no The null space of M called a masked LDPC whose Tanner of length 4 [53]. results a class of masked LDPC codes with various rates and minimun1 distances. Tl1e rnte of a masked EG-LDPC cod1;; is controHed the y and 2s, but its error is controlled the of the 1-entries in the Lw,,cs,rn,F,

1-

1 1 0 0 0 0 0 0 ., 0 1 0 1 1 0 1

I0 I

l Given a

vwith

1 1 0 1., 1 0 1 0 0 0 0 0 0 0 0

0 0 l

1 0 0 0 01,

0 0

0 0 1 0 0 1 1 0 l 1 0 1 0 0

0 0 0 0 1 0 1 1

l I

I.

I

we can form a y x y circulant matrix G Both the column and row are said to be distinct if one

matrix with column and row

Consider the 17.24. Let P

of 01 and f3 = lw, should be chosen such that

Example bundle, each consisting The incideace matrix A; of flPrnns1·on
Let y = 8. We form the

512 >< 4096 base matrix:

L.

As.64

911:l

Chapter 17

Low-Density Parity-Check Codes

which is an 8 x 64 array of 64 x 64 permutation matrices. The column and row weights of !HIECi.GA.l are 8 and 64, respectively. Suppose we want to mask IrJIEG.Cl,,l with an 8 x 64 masking matrix Z1 with column and row weights of 4 and 32, respectively. Among the seventy 8-tuples of weight 4, all but 6 are primitive. The 64 primitive 8-tuples can be generated by 8 distinct primitive 8-tuples, v1, v2, .. · , v3, and cyclically shifting them. With these 8 distinct primitive 8-tuples, we form eight 8 x 8 circulants, Gi1l. Gil), · · · , G~l), with column and row weights of 4. Then, ,.,, _ [,,-.(1),r,(l) ,,-.(1)] .&:.J ""'1 u2 · · ·

""s

gives an 8 x 64 masking matrix with column and row weights of 4 and 32. Masking HEG.GA, 1 with Z1, we obtain a 512 x 4096 masked matrix

with column and row weights of 4 and 32, respectively. The null space of M 1 gives a (4096, 3585) LDPC code with rate 0.875 and a minimum distance of at least 6. The error performance of this code with SPA decoding is shown in Figure 17.31. At a BER of 10-- 6 , the code performs 1.2 dB from the Shannon limit. To construct a lower-rate code, we need to choose a larger y. Suppose we choose y = 16. For the same geometry EG(2, 26 ), the base matrix IBl.Ec.cA.2 given by ( 17 .69) is a 16 x 64 array of 64 x 64 permutation matrices. For masking, we need to construct a 16 x 64 masking matrix Z 2 . Suppose we choose the column and row 1011

10

I

10-2

E E "'

.D 0 H

n.

6 10

.j

H H Q)

~ u 0

fl

I I I

10

I

I I

ci'i 10"

\"

10 '~

1o~L~~L ()

1

2

l~____i_1___L__.~l~~I~~ 3

4

5

6

7

8

Et,!N0 (dB)

FIGURE 17.31: The error performances of the two masked Gallager-LDPC codes given in Example 17.26.

Section 17. 'i 1

Masked

LDPC Codes

:911

To construct we take four distinct 4 and then form four 16 >< 16 these four distinct

of 4 and 16, """on.~r>,i'n,~ '''"'"'·'Hhrnatrix vvith colunm and 1-o"W we obtain a 1024 >< ~096 masked nrntrix ®

null space of distance of 1.35 dB from the Strnnnon

such that y :::: p :::: 2s. If we take the first p columns vve obtain the

(p) =

I Au

Au

,p

A2.p

IL

-1

J

E1,/N11 (dBJ

flGURE i7.J2: The error

code constructed bcised on

8961) masked

9'i2

Chapter 'i 7

Low-Density Parity-Check Codes

For masking construction, we can use JH!EG.GA (p) as the base matrix. This shortening gives more flexibility for constructing codes with various lengths and rates For example, suppose we choose EG(2, 27 ) for code construction using masking. If we choose y = 10 and p = 80, the base matrix HEG.GA(80) is then a 10 x 80 array of 128 x 128 permutation matrices. Masking this matrix with a 10 x 80 matrix Z with column and row weights of 4 and 32, respectively, we obtain a 1280 x 10240 masked matrix M. The masking matrix Z consists of eight circulants that are constructed by using eight distinct primitive 10-tuples over GF(2) and cyclically shifting each of them 10 times. The null space of M gives a (10240, 8961) LDPC code with rate 0.875 whose error performance is shown in Figure 17 .32. At a BER of 10~ 6 , it performs only 0.9 dB from the Shannon limit. Masking is a very powerful techniques for constructing both regular and irregular LDPC codes. 17.12

(OINSTRU(TiOIN Of QUASi-(YCUC (OIDIES !BY (iROJJlANl [D)IE(OMPOSmON

Consider a q x q circulant Gover GF(2) with column and row weights 8. Because column and row weights of a circulant are the same, for simplicity, we say that G has weight 8. For 1 .::: t .::: 8, let w1, w2, · · · . w 1 be a set of positive integers such that 1.::: w1, w2. · · · , w 1 .::: 8, and w1 + w2 + · · · + w 1 = 8. Then, we can decompose G into t q x q circulants with weights w 1 , w2, · · · , w 1 , respectively. Let g1 be the first column of G. We split g1 into t columns of the same length q, denoted by gil), gi2 ),. · · , gin, such that the first w1 1-components of g1 are put in gil), the next w2 2 1-components of g1 are put in gi J. · · · , and the last w1 1-components of g1 are put in For each new column l, we form a q x q circulant G; by cyclically shifting l downward q times. This results in t q >< q circulants, G1, G2, · · · , Gr, with weights w 1 , w2, · · · , w 1 , respectively. These circulants are called the descendants of G. Such a decomposition of G is called column decomposition of G. Column decomposition of G results in a q x tq matrix.

git).

gf

gf

(17.73) which is a row oft q x q circulants. If w1 = w2 = · · · = w 1 = w, then 1H! is a regular matrix with constant column weight w and constant row weight tw. If t = 8 and w 1 = w2 = ... = w(8) = l, then each descendant circulant G; of G is a permutation matrix, and IH! is a row of 8 permutation matrices. The parameter t is called the column splitting factor. Figure 17.33 shows a column decomposition of an 8 x 8 circulant of weight 3 into two descendants with weights 2 and l, respectively. Similarly, we can decompose G into descendants by splitting its first row into multiple rows and cyclically shifting each new row to the right q times. Let l .::: c .::: max{w; : 1 .::: i .::: t). For 1 .::: i .::: t, let wi.l, w;.2. · · · , w;.c be a set of nonnegative integers such that O.::: wi.l, w;.2, · · · , w;., ::: w;, and w;.1 -l-w;,2 +· · ·-1-w;., = w;. Each descendant G; in IH! of (17.73) can be decomposed into c descendant circulants with weights wi.1, w;.2, · · · . w;.,, respectively. This is done by splitting the first row g;,1 of 2) " mto · · u; crows o f t l1e same lengt h q, d enote db y !k(])1 , gi.( l, · · · , '/!,;,(c)J, w11ere g;_(l)1 contams the first w;.1 1-components of g;,1, g;21l contains the second w;.2 1-components of
0

.L

0

.1.

0 ..11....11.

.1.1...A.1-.1.0....,

.1

Section '17 .1

Construction of

Codes

10000110 01000011 10100001 11010000 =>01101000 00110100 00011010 00001101

FIGURE

10000010 01000001 10100000 01010000 00101000 00010100 00001010 00000101

.33: A column

Circulant

9Ji3

00000100 00000010 00000001 10000000 01000000 00100000 OOOlOOOOJ 00001000 ·2 ..J.

of a circulant of

Ee-vv row 1 to the we obtain c circulants < q matrix: 0

~·~,~~

0

'"'

SJ~•JH,-.c.

iilJ;

=

which is a column of c q >< q circulants. We call iilJ; the row lT" c J.lke u1 row . "~n row we a 11ow w;.k = 0. .LI q >< q zero as a circulant. If each circulant G; in lHI of (17.73) is its row we obtain the c >< t array of circulants: G;ll G(2l I

w;.k

l

=

of
(17.75)

(c)

Gr

which is a cq >< tq matrix. H each G?l has weight 1, then I!]) is an array of F-""H'V•W" matrices. The nun space of I!]) a code of length n = tq that can be put in quasi-cydic form Section 5.12). H G is a sparse matrix, I!]) is also a sparse matrix with smaller G. H no two rows two in G have more than one in common. then no two rows ( or two columns) in I!]) have more than one 1-component a quasi-cyclic LDPC code whose in common. In this case, the null space of lDl Tanner graph is free of cycles of length 4. H an the circulants in I!]) have the same weight. then the code is a regular quasi-cyclic LDPC code. As shown in Section 8.5 and Sections 17.4, 17.5, and 17.7, sparse circulants can be constructed from the incidence vectors of lines in either a Euclidean geometry or a projective geometry. Furthermore. no two rows two either from the same or from two different circulants have more than one 1-component in common. quasi-cyclic LDPC codes can be constructed decomposing one or a group of these geometry circulants [54. 55].

914

Chapter 17

Low-Density Parity-Check Codes

The incidence vectors of all the lines in EG(m, 2s) not passing through the origin can be partitioned into K = (2(m-l)s - 1)/(2s - 1)

cyclic classes, Q1, Q2, .. ·, QK. Each cyclic class Q; consists of 2ms - 1 incidence vectors and can be obtained by cyclically shifting any vector in the class 2ms - 1 times. Hence, for each cyclic class Qi, we can form a (2111 s - 1) x (2111 s - 1) circulant Gi using any vector in Q; as the first row and then cyclically shifting it 2 111 s - 2 times. The weight of Gi is zs. Consequently, we obtain a class of K circulants, (17.76) In code construction, we can take a subset of k circulants with 1 ::: k ::: K, say G1, G2, · · · , Gk, from g and arrange them in a row,

We choose column and row splitting factors t and c, respectively, and decompose each circulant G; in H into a c x t array .!IJ\ of descendants. Replacing each G; in H by its array decomposition llJi;, we obtain a c x kt array Jl) of descendant circulants. Jl) is a c(2 111 s - 1) X kt c2ms - 1) matrix. The null space of JI) gives a quasi-cyclic LDPC code C whose Tanner graph does not contain cycles of length 4 and hence the girth of its Tanner graph is at least 6. If D has constant column and constant row weights, then C is a regular LDPC code.

EXAMPLE H.27' Consider the three-dimensional Euclidean geometry EG(3, 23 ) over GF(2 3 ). This geometry consists of 511 nonorigin points and 4599 lines not passing through the origin. The incidence vectors of the lines not passing through the origin form nine cyclic classes. From these cyclic classes we can form a class of nine 511 x 511 circulants of weight 8, Suppose we take eight circulants from g and arrange them in a row to form the matrix We choose column and row splitting factors t = 2 and c = 4, respectively, and decompose each circulant G; into a 4 x 2 array D; of eight 511 x 511 permutation matrices. Replacing each circulant Gi in H by its array decomposition Di, we obtain a 4 x 16 array JD> of 511 x 511 permutation matrices. JD is a 2044 x 8176 matrix with column weight 4 and row weight 16. The null space of D gives an (8176, 6135) quasi-cyclic LDPC code with rate 0.75 and a minimum distance of at least 6. Its error performance with SPA decoding is depicted in Figure 17.34. It has a beautiful waterfall error performance all the way down to a BER of 10- 10 without error floor. At a BER of 10- 10 , it performs 1.4 dB from the Shannon limit.

Section 1 .1

Construction of

Circulant

E;,IN11 (dB)

]Srror 17.27.

of the

code

in

LDPC codes can aiso be constructed based on the incidence [17, Consider the mdimensional even and odd m. For even m, the incidence vector of each line and the incidence vector~ of all the lines can be

1n

into

- 1)

classes. Each class consists of (2(m+lJs - 1)/(2' -1) incidence vectors that can be obtained any vector in the class. For odd m :::: 3, there are c2< 111 +lJs -1)/(22' -1) lines in whose incidence vectors are not and the incidence vectors of all the other lines are The incidence vectors can be - 1)

classes. LDPC codes can be constructed based on the classes of incidence vectors of lines in a

Consider the three-dimensional projective over This consists of 585 and 4745 lines. Each line consists of 9 are 65 lines whose incidence vectors are not and the incidence vectors of the other 4680 lines are The 4680 incidence vectors can be

916

Chapter 17

Lovv-Density Parity-Check Codes

partitioned into eight cyclic classes, with each class consisting of 585 vectors. Based on these eight cyclic classes of incidence vectors, we can form eight 585 >< 585 G1, G2, · · · , Gs. Each circulant G; has weight 9. Suppose we want to construct a quasi-cyclic LDPC code of length 9360 with rate 0.875. First, we decompose each circulant G; by column decomposition into three descendant circulants, G;.1, G;.2. and Gu, with weights 4, 4, and 1, respectively. Removing the descendant with weight 1, we obtain the 585 x 1170 matrix which consists of two 511 x 511 descendant circulants of G;. We decompose each circulant of D; by row decomposition into two descendants., each having weight 2. The result is the following 2 x 2 array of 511 x 511 circulants:

* = [ G;1] G(2)

G;1] G(2J

,.1

,,2

D;

J,

which is a 1170 >< 1170 matrix with both column and row weight 4. We then form the following 1170 >< 9360 matrix: D -- [D*1 D*2 D*3 D*4 D*S D*6 D*7 D*] 8·

which has column and row weights 4 and 32, respectively. The null space of D gives a (9360, 8192) quasi-cyclic LDPC code with rate 0.875. The error performance of this code is shown in Figure 17.35. At a BER of 10- 6 , it performs 0.95 dB from the Shannon limit.

10"

=--- ____

10-1

....

10-2

- - Uncoded BPSK .-BER ""*"'FER - Shannon limit

---

-- -- --

t

..... .,.~,

E

...... ......

"'

.D 0

ii

.... 10-e

e

-

... .,,,_

....... ...

C

... ... ...

.. .

a.>

"" ~

"' ...

"'

<.)

0

10'

co 10

1,

-

10- 7 10 ,

I 0

2

3

4

s

6

7

8

9

£,,/N11 (dB)

FIGURE 17.35: Error performance of the (9360, 8192) quasi-cyclic code given in Example i7.28.

Construction

LDPC Codes Ba~ed on r:inite Geometries

'91

on finite however, LDPC codes also can be constructed over the field where the same as those ove:t

over ). A line in or its coset of the vector space of all the There are p(m-l)s

- 1)

into

be

K=

--- 1)

to each other. Each is on one and - 1)

The incidence vectors of all the lines not into ls --

,vuvmu~

the

1)/(V' -- l)

of - 1 vectors, f.~H the constructi011s c.f TLJC:p,c codes based on the lines and of to the construction of LDPC codes based on the li.nes and Vle use the construction of for ,>~La,,v,cush

over ?[,=(VO, VJ.···. Vn_i),

to the n points of EG(m, p 1·) and Vj = 1 if and and Vj = 0 otherwise. Therefore, the of Y L bt1 ndle P; '.),f lines in vve can form a )s x whose rmvs are the incidence vectors of lines in P;. Jt follows of a bundle that the column and row JI :S K. 'vVe form a yp<m-l)s >< p 711 s mati-ix in a column as foHovvs:

9'i8

Chapter

n

Low-Density Parity-Check Codes

where each submatrix H, is formed based on a parallel bundle P, of lines in EG(m. p'). The column and row weights of JHIEG,GA are y and , respectively. The null space over GF(2) of this binary matrix gives a Gallager LDPC code of length n = p 1115 and minimum distance of at least y + 1 for odd y and y + 2 for even y. Clearly, for p = 2, we obtain the class of BG-Gallager LDPC codes given in the previous sections. IEXAMl?llE 17 .29)

Let m = 2, s = 1, and p = 43. The two-dimensional Euclidean geometry EG(2, 43) consists of 1849 points and 1892 lines. The 1892 lines can be grouped into 44 parallel bundles, each consisting of 43 lines parallel to each other. Each line consists of 43 points. Taking 4 and 6 parallel bundles, we can construct two EG-Gallager LDPC codes that are (1849, 1680) and (1849, 1596) codes with rates 0.9086 and 0.8632, respectively. With y = 4 and 6, the lower bound y + 2 on the minimum distance of an EG-Gallager code gives their minimum distances as at least 6 and 8, respectively. The bit-error performances of these two BG-Gallager LDPC codes with SPA decoding are shown in Figure 17.36 ( assuming BPSK signaling). At a BER of 10- 6 , the (1849, 1680) code achieves a 5.7-dB coding gain over the uncoded BPSK system and is 1.5 dB from the Shannon limit for rate 0.9086, and the (1849, 1596) code achieves a 6-dB coding gain over the uncoded BPSK system and is 2.1 dB from the Shannon limit for rate 0.8632.

IEXAMPllE 11.30

Suppose it is desired to construct a rate-1/2 LDPC code with length around 6400 based on geometry decomposition and masking. To satisfy the length constraint, we must choose m, s, and p such that p 1111 :::::: 6400. There are many possible choices of m, s, and p for which p 111 s :::::: 6400. One such choice is m = 2, s = 4, and p = 3. With this choice, the geometry for decomposition is the three-dimensional Euclidean geometry EG(2. 34). This geometry consists of 6561 points and 6642 lines, and each line consists of 81 points. The 6642 lines can be grouped into 82 parallel bundles, with each parallel bundle consisting of 81 parallel lines. We decompose this geometry based on a parallel bundle P(2, 1) of lines. Then, there are 81 connecting parallel bundles of lines, Q1, Q2. · · · . Qs1, with respect to P(2, 1). The incidence matrix A, of a connecting parallel bundle Q, is an 81 x 6561 matrix consisting of eighty-one 81 x 81 permutation matrices. To achieve the desired rate 1/2 and length equal or close to 6400, we set p = 80 and y = 40. With this choice of p and y, we construct a base matrix IH!EG,GA (80) based on (17.72). HEG,GA is a 3240 x 6480 matrix with column and row weights 40 and 80, respectively. It is a 40 x 80 array of 81 x 81 permutation matrices. Next, we need to construct a 40 x 80 masking matrix Z. Suppose we choose the column and row weights of Z to be 3 and 6, respectively. To construct we choose two distinct primitive 40-tuples over GF(2). Cyclically shifting these two distinct primitive 40-tuples, we form two 40 x 40 circulants IG1 and IG2. Then, Z = [G 1 G2]. Because the weight of each circulant is 3, it is easy to construct G 1 and G2 su.:'.h tlu1t the Tanner graph of Z is free of cycles of length 4. Masking the base matrix JHJ.EG,GA (80) with Z, we obtain a 3240 x 6480 masked matrix

Construction of LDPC Codes Based on Finite Geometries

Section 17 .1

I

I

9li!0

I

Uncoded BPSK -,i~ 4 bd (1849.1680) block -=4bd(l8°!9,]680)bit :-, ~o= 6 bcl (18°!9, 1596) block -@- 6 bcl (1849, 1596) bit = =

0

3

2

5

I

I

I

I

6

7

8

9

10

E;,IN, 1 (dB)

FIGURE 17.36:

Error

of two EG-GaHager codes of iength 1849.

1 =

(!,

10-1

~--=::~~.t~.\ &

10-2 ~\

,, :1

B

8 ~

=-""=

=-,,,,,

I u 'I !u

\ '!

10·-+

0

C

bI!

jw

\l~

~ rii

-o- FER

\\

10

H

..<.) ~

r

j:

~ncoclecl ~PSK BER :

\ !

"'

..D

0 H 0..

=

cc€~

I

10-0

u

I I!,

2

3

4

5

6

7

8

9

FIGURE 17.37: Bit- and block-error probabilities of a (6480, 3240) LDPC code.

920

Chapter 17

Low-Density Parity-Check Codes

M = Z ® lH!Ec.GA (80) with column and row weights 3 and 6, respectively. The null space of IVE give~ a (3, 6) regula:i~ (6480, 3240) LDPC cod(: vvith rate 1/2. The e1ror performance of this code decoded with the SPA is shown in Figure 17.37. It has both good bit- and block-error performances. At a BER of 10- 6 , it performs 1.5 dB from the Shannon limit.

Construction based on finite geometries over GF(p 8 ) results in a very large class of finite-geometry LDPC codes containing all the finite-geometry LDPC codes constructed in the previous sections of this chapter as subclasses. 11.14

!RAN IDOM U)P( (ODIES

In addition to being formed by the geometric construction presented in the previous sections, LDPC codes also can be constructed by computer search in a pseudorandom manner based on a set of guidelines satisfying the conditions given in Definition 17.1. This construction results in an ensemble of random codes that have been proved to contain good LDPC codes [10]. Suppose it is desired to construct an LDPC code of length n with rate k/n. To construct a parity-check matrix for this desired code, we need to choose an appropriate column weight y and an appropriate number J of rows. H is clear that J must be at least equal ton - k, the number of parity-check symbols of the desired code. In computer construction, J is usually chosen to be equal ton - k. For JI-II to have constant row weight p, the condition y x n

=p

(17.81)

x (n - k)

must hold. Otherwise, lHl cannot have constant row weight. In this case, we simply try to keep all the row weights close top. H n is divisible by n - k, it foHows from (17.81) that pis a multiple of y; that is, p = yn/(n k). For this case, we can construct a regular low-density parity-check matrix with column weight y and row weight p. H 11 is not divisible by 11 - k, we divide y x n by 11 - k and obtain y x n

=

p(n - k)

+ b,

where p and bare the quotient and remainder. respectively, with O < b < expression of (17.82) can be rearranged as follows: y X 11

=

(11 -

k - b)p

+ b(p + 1).

(17.82) 11 -

k. The

(17.83)

which suggests that we can construct a low-density parity-check matrix lHl with two row weights, p and p + 1, respectively. For convenience, JI-II is constructed in such a ,vay that its top b rovvs have \veight p ~1- 1, and its bottom n - k - b rows have weight p. The construction of lHl is carried out step by step. At each step, one column is added to a partially formed matrix. Each added column must satisfy certain constraints. For 1 :::: i :::: n, at the ith step, a binary (n - k )-tuple of weighty is chosen as a candidate column lhti and is added to the partial parity-check matrix lHii-1

=

[!l:R1, lht2, · · · , iini-d

(17.84)

Section 17 .14

obtained at the (i For i == 1~ IHio is constraints must be satisfied:

to forn1 the next

Random LDPC Codes

912'1

matrix IT-IT;. lln; to TI-E,- -1 ~ the

JL Choose llil; at randorn :froni.

that are not

used in 2, Check v,hether 1.'ll; has more than one inmI;-1.Hnot,goto 3;

in common with any column Iln; and go back to 1 to choose

another candidate colunm.

add lm; to IT-lI;-1 to form the construction process. If a:c1y of the b rows of IE1; has p + t or any of the bottom n - k - b rmvs of ]Hi; has lm; and go to 1 to choose another car:cdidate colunrn. The construction process continues until a matrix IT-IT with n columns is formed. H b = 0. lh1 is a matrix with rovv and column v,._,n,,n~ p and y. H b =I= 0, then 1Y1 has two row weights, p + 1 and p. For a given 11, k. and y. it is that all the (n are either used or rejected before IH is formed. To reduce this we need to choose 11, k. and JI such that the total number of (11 is much larger than the JI code 11. or we can relax the row constraints in 3 to allow rovv that a chosen candidate column that m 3. Of course, we can restart the construction process by another sequence of candidate columns. l[f the row rank of JH[ is n - k, the null space of IHI an (n. k) LDPC code with rate k/n. H the rank of IHI is less than n -- k, then the null space an (11. k') LDPC code with k' > k and rate k' /11 > k/11. We can show that the rate R of the constructed code is lower bounded as fo!lovvs: R > 1-

}1__,

p

The constraint at step 2 of column selection ensures that the Tanner graph of the code does not contain ,my of 4. the girth of the Tanner graph is at least 6. The construction is efficient for small y, usually 3 or 4. For large y, to check the constraints at 2 and 3 can be computationally 1 of the construction a column is chosen at random from expensive. Because at the remaining available (n - k )-tuples, the code constructed is not unique. The construction gives an ensemble of random LDPC codes, With this construction it is very hard to determine the minimum distance of the code constructed. For small y, 3 or 4, the lower JI + 1, on the minimum distance can be very poor.

Suppose we want to construct a JI = 3. Because n = 504, and n - k =

252) LDPC code vvith rate 1/2. V\/e choose 11 / (n -- k) = 2, so we choose row weight

922 p

Chapter 17

Low-Density Parity-Check Codes

= 2 x 3 = 6. The number of binary 252-tuples with weight 3 is

which is much larger than the code length 11 = 252. Following the construction procedure, we obtain a regular 252 x 504 parity-check matrix H with column and row weights 3 and 6. respectively. The density of lHl. is 6/504 = 0.012. Its row rank is 252, and hence its null space gives a (504, 252) LDPC code with rate l/2. Because y is 3. the minimum distance of the code is at least 4. The true minimum distance of this code may be greater than 4. The bit-error performance of this code with SPA decoding is shown in Figure 17.29. We see that this code performs very close to the rate-l/2 (512. 256) EG-Gallager code given in Example 17.23. Random construction results in a large ensemble of LDPC codes that contains finite-geometry LDPC codes as a subclass. Clearly. there must exist random LDPC codes that outperform finite-geometry LDPC codes in error performance, especially long random codes. Computer-generated random LDPC codes. in generaL do not have the structural properties of the finite-geometry LDPC codes, such as cyclic or quasi-cyclic structure. Consequently, encoding of a random LDPC code in hardware is much more complex than encoding a finite-geometry LDPC code; that is. its encoding cannot be implemented with linear shift registers. Because computer-generated LDPC codes. in general. have relatively small column weight, the number of check-sums orthogonal on a code bit that can be formed is small. As a result. these codes perform poorly with one-step majority-logic decoding or bit-flipping decoding. Furthermore, computer-generated random LDPC codes with SPA decoding do not converge as fast as finite-geometry LDPC codes do. For finite-geometry LDPC codes with SPA decoding, usually 5 iterations give a performance only a fraction of a decibel from their performance with 100 iterations. as demonstrated in Figure 17.11 for the (4095, 3367) cyclic EG-LDPC code. With all these disadvantages, long random LDPC codes do perform very close to the Shannon limit. For example, very long random LDPC codes (10 7 bits long) have been constructed and shown to perform only a few thousandths of a decibel from the Shannon limit [14].

An irregular LDPC code is defined by a parity-check matrix lHI with multiple column weights and multiple row weights. In terms of its Tanner graph. the variable nodes (code-bit vertices) have multiple degrees, and the check nodes (check-sum vertices) have multiple degrees. H has been shown that long- random irregular codes perform arbitrarily close to the Shannon limit [8. 12-14]. Irregular LDPC codes are most commonly designed and constructed based on their Tanner graphs. One such approach is to design these codes in terms of the degree distributions of the variable and check nodes of their Tanner graphs [12. 13]. Consider the Tanner graph g of an irregular LDPC code with parity-check matrix IHI. The variable nodes in g correspond to the columns of ~rrd the check nodes of g correspond to the rows of )BL The degree of a node in g is defined as ~

Section 17 .15

LDPC Codes

column in JH. and the row in ff-IT. Let

of

21

check aode is

-]

i=i

I,

v,here y; denotes the fraction denotes the 1TJ1axin1uE1

,rnd

Pi

-]

of the check nodes of

where Pi denotes the fraction of denotes the maximum check-node of an variable- and check-node degree distributions of its LD JPC codes can be v,.,u,,cp•A• centered on the evolution between the two types of nodes in [l2] when the code cannot be made for codes of finite distributions designed for infinite code vviH not LDPC

m,nir,,>'1P

the error floor

in [21] and [22]. This constraint variable nodes must be smaller than the number of the number of colurrms of the check matrix be sn,,dler than the number of rovvs of IT-IT). ,__,.,c,,-;,-,c,,-,·, that the number of variable nodes be smaller than the bits of the code to be can be met vvith a redistribution. Let nc1 and be the desired code and rate, the degree distribution y(X) for the desired rate based on the evolution of probability densities. From the fnKtion of the variable nodes given in y(X). we the number JV2 of variable nodes the desired code nc1; that = J/2 x nc1. H is smaller than the nun1ber nc1(1 ~ ) of the parity-check bits of the desired then y and are used for constructing the desired code. H 1s

924

Chapter 17

Low-Density Parity-Check Codes

than nc1(l - Ki), then we convert the excess degree-2 variable nodes to variable nodes of next-higher degree, say, degree 3 if it exists. The result is a 111odified variable-node degree distribution y*(X). In a Tanner graph, the sum of check-node degrees (equal to the total number of edges of the graph) is equal to the sum of variable-node degrees. If we modify the variable-node degree distribution y (X), we must change the check-node degree distribution p(X) accordingly to make the sum of check-node degrees the same as the sum of variable-node degrees in the new variable-node degree distribution. The next step in code construction is to construct a Tanner graph by connecting the variable nodes and check nodes with edges under the constraints given by the degree distributions. Because the selection of edges in the graph construction is not unique, edges are selected randomly. During the edge selection process, effort must be made to avoid having cycles among the degree-2 variable nodes and cycles of length 4 in the code graph. As a result. computer search is needed. Once a code graph is constructed, we form the corresponding parity-check matrix lHL Then, the column and row weight distributions of !HI are the same as the variable- and check-node degree distributions. The null space of 1Hl. gives an irregular code of the desired length and rate. The described construction gives a random irregular LDPC code. The masking technique presented in Section 17 .11 can be used to simplify the construction of irregular LDPC codes based on the degree distributions of variable and check nodes of their Tanner graphs. For masking, the array HEG,GA (p) of permutation matrices given by (17.72) is used as the base matrix. Suppose the degree distributions y(X) and p(X) have been designed for a given code rate Rc1 and length nc1. Suppose geometry EG(m, v') is used for constructing the base matrix. We choose parameters y and p such that pp(m-l)s is equal or close to the desired code length nc1, and (p - y)/y is equal or close to the desired rate Rc1. We construct a y x p masking matrix Z = [z;.J] with column and row weight distributions \C(X) and d(X), respectively, identical to the degree distributions y(X) and p(X) of the variable and check nodes, respectively, where dv :S y and de :S p. In constructing the masking matrix z. we put columns with weight 2 in the parity-check positions and make them cycle free among them. The cycle-free condition can easily be achieved by using a set of weight-2 columns obtained by downward shifting two consecutive l's from the top of Z until the second 1 reaches the bottom of Z as shown: 1 0 0 1 1 0 0 1 1 0 0 1

Z=

l

0 0 1

0 0 0 0

0

nV

0

n V

0

1 J.

0

0

0

0

1

Masking the base matrix masked matrix

lHl.Ec,GA (p)

M = Z ®

other columns

(17.87)

given by (17.72) with Z, we obtain the

JH!EG,GA(P)

with column and row weighi disiribuiiuns <1:(X) and all(X), respectively. Cunsequenily, the associated Tanner graph of M has variable- and check-node degree distributions

1 .15

Irregular LDPC Codes

925

y(X) and

the 1mH space of iiYiI an irregular LDPC distributions. Because the size of fl in general, it is much easier to construct since the associated Tanner graph of the base matrix of length 4, we The masking construction do not have to worry about ~H"'"'""-' and search effort of large random edge reduces the construction of the code as is needed in LJCi1PiC code 1

v;o,c.,u,rJHcd

to illustrnte the construction of irregular LDPC

the variable- and check-node (X) = OAOS2X

p

+

J

+ 0.0555X 7 .

= 0.3109X 18 +

The maximum variable- and check-node we choose

l

n r1 = 4000 and rate 0.82 is to we find the asymptotically distril1utions for rate 0.82:

are 8 and 20, respectively. Suppose EG(2. 26 ) over GF(2 6 ) for Using this geometry, we bundle of lines into sixty-four 64 x 64 matrices. To achieve a code dose to the desired code length 4000, we choose p = which is than the maximum check-node degree 20. To achieve a rate close to the desired code rate we choose y = 12, which is C'1l'l'Cl1CP1' than the maximum variable-node 8. For the choice of y = 12 and the rate of the code is at least (p - y) /,o = -12) /64 = 0.81, which is close to the desired rate 0.82. The of the code is 63 >< 64 = which is close to the desired code It follows from (17. that the base matrix IHIEG,GA(63) is a 768 >< 4032 which is a 12 >< 63 array of 64 x 64 matrices. For the desired code, the numberof bits is 4000 x (1-0.82) = 720; however, the number of degree-2 variable nodes the first coefficient J/2 = 0.4052 of y (X) is nr1 x Y2 = 4000 >< 0.4052 = ·which is than 720, the number of parity-check bits of the desired code. the variable-node degree distribution y (X) 903 variable nodes of degree 2 to degree 3. The degree redistribution results in a new variable-node degree distribution, =

0.1798X

+

~r"' 0

"''"

+ 0.1466X 6 + 0.0555X 7 .

There is a small change in the coefficient of each degree in the check-node degree distribution p(X). Next, we need to construct a 12 x 63 masking matrix Z with column and row weight distributions identical to the modified degree distributions (X) and (X). Table 17.5 gives the desired column and row weight distributions of the masking matrix Z. By computer we find the desired masking matrix Z. Masking the 768 x 4032 base matrix IHIEG.GA (63) with Z, we obtain the masked matrix M = Z ® IHIEG.GA (63). The null space of M gives a (4032, 3264) irregular code

926

Chapter 17

Low-Density Parity-Check Codes

TABLE 17.5: Desired column and row weight distributions of the

masking matrix Z for Example 17.32. Colll!l!m.llll weiiglhlt diistirilbn11tfoID1

Row weiigM cdliistiriib1111tfo1111

Cohl!m.l!ll. weight

No. of coll1111m.lllls

Rowweiiglhl1!

No. IThlf rnws

2

11

19

4

3

39

20

8

7

9

8

4

100 - - , Uncoded BPSK ' FER C 1 -es-· FER C, -~· FER 1 -.BERC 1

-©-·

10-1

==---

c~

10-2

BER Co -BERC~ 1

& :E 10-,

~

o:l

.D 0 0.

,.,

'"0 .... '" (l)

~ u

Shannon limit

',,,,,

10-· 10-5

0

~

i:o

10-6 10-7 10-8 10-9 0

1

2

3

4

5

6

7

8

9

FIGURE 17.38: Error performances of the three (4032. 3264) irregular LDPC codes given in Examples 17.32 and 17.33.

C 1 with rate 0.81. The error performance of this code with SP A decoding is shown in Figure 17.38. At a BER of 10-5, the code performs only 1 dB from the Shannon limit; however, it has an error floor starting around a BER of 10- 6 . Figure 17.38 also includes the error performance of a (4032, 3264) irregular code C2 constructed based on the original asymptotically optimal degree distributions without degree redistribution. We see that C1 performs much better than C2. The error floor of C2 occurs above a BER of 10-5 .

Section

LDPC

\A/hose code of finite An ~Iariable has an error floor distance. The error floor variable nodes or ,,axiable nodes is of variable nodes converted into and so on [53].

'\!Ve rnust also of cl,eck-node

the SlU11 in the new

variable-node p1 distribution: fJ+l

we construct a 12 >< 63 distributions rn the same c1s the one

the base matdx ,~,,....,,.,,_. "'' 17.32. The masked matrix IMI+ 1 LDPC code C+J · The error 17.38. 1Ne 1 has much better error floor ''"o.r'c,'w,,-n than codes C 1 ;:md rn 17 .32. there is no error floor down to a BlSF<. of 5 >< than 15 above a BER of 17.32 and 17.33 slmw thc1t the error floor clown by the of variable nodes of the code the error of the code moves aw<1y from the Shannon limit in the ,,vaterfall TABLE n.6: Column and row

matrix for

distributions of the

17.33.

CC11,Ilm1m Yefenglh11[ i!:llns/rrnilDuntlfollil I ~(ij)V/ -· I I',fo, ([])[ 1:([J)Ilnummm-vs----j-J.P'-.,
3

f

2s

f----------:~~-- 2:

I

23

_____[ _ 24

I

--+-+-------~ 8 --,~·

F~ -----+----:---1------- (._ 9

~.-

4

--·---- _____J_______

928

Chapter 17

Low-Density Parity-Check Codes

IEXAMPlE 11.34

The following optimal degree distributions of variable and check nodes of a Tanner graph are designed for a rate-1/2 irregular LDPC code of infinite length [12]:

+ 0.28057X2 + 0.034996X 3 + 0.096329X4 + 0.009088X 6 + 0.0013744X 13 + 0.100557X 14, 0.982081X 7 + 0.0179186X 8 .

y(X) = 0.47707681X

p(X) =

Suppose we want to construct a rate-1/2 irregular LDPC code with length around 10000 using masking based on these degree distributions. To construct such a code, there are many geometries that can be used for constructing the base matrix for masking. We choose the two-dimensional Euclidean geometry EG(2. 27 ) for constructing the base matrix HEG.GA(P) given by (17.72). Using this geometry, we can partition the incidence matrix of a parallel bundle of lines into one hundred twenty-eight 128 x 128 permutation matrices. Suppose we choose y = 40 and p = 80, which are greater than the maximum variableand check-node degrees, 15 and 9, respectively. With the choice of y = 40 and p = 80, 80 x 128 = 10240. which is close to the desired code length 10000, and (p - y)/y = (80 40)/80 = 0.5, which is the same as the desired rate 1/2. It follows from (17.72) that the base matrix HEG,GA (80) is a 40 x 80 array of 128 x 128 permutation matrices. For masking, we need to construct a 40 x 80 masking matrix Z with desired column and row weight distributions given in Table 17.7. This is done by computer search. Masking JHIEG.GA (80) with Z, we obtain a 5120 x 10240 masked matrix M = Z ® lHIEG,GA (80). The null space of M gives a (10240, 5102) irregular LDPC code C with rate 1/2. The error performance of this code is shown in Figure 17 .39. At a BER of 10- 6 , it performs 1 dB from the Shannon limit.

Column and row weight distributions of the masking matrix for Example 17.34.

TABLE 17.7:

Colmmnm weight «fo,tiributfoim

Row weight irllistribution

Coh]lmJlll wenglht

No. of collumllll§

R((])W wenglhit

No. olfrnw§

2

38

8

39

3

22

9

1

4

3

5

8

7

1

15

8

Section 1 16

Theoretic LDPC Codes

-----------~--~-- .. ·11 =Shannon lin1it

g

(I:). :::I:_ E

2

gJ;'.2'.'31

,

10-·' ;--

~

2

s ii3

I io "le-

: 10-7 I ~

l

I

I 11

!

I

lO ·' _ _ _i_!l__ _ L _____L _ _ _ J _ _ _ _L _ _ _ ~ - - ~ -l O l 2 3 4 6 5 E,,IN 11 (dB)

FIGURE 17.39:

rn

Error 17.34.

of the two

LDPC codes

form a branch of combinatorial mathematics. Besides finite there are other branches in combinatorial mathematics that can be used LDF'C codes. These branches include combinatoric 58]. Xn Section 8.3 we showed that codes constructed based on difference sets are JVJ[LG and we also showed in Section 17 .5 that the codes constructed based on a class of difference sets are LDPC codes whose Tanner 4. Constrnc1ion of LDPC codes based on random (or graphs was discussed in the section of this chapter. Construction based on random results in an ensemble of LDPC codes. fo this section we nr,C'N'n'o" another to the construction of LDPC codes [59]. The construction is based on a set of of the same length in a given graph that satisfies certain constraints. Let g = (V. be a connected graph with vertex set V = {v1. vz. · · · . edge set E = {e1 . ez, · · · . eL}. "\Ne that Q not contain two vertices in Qare connected by at most one edge (i.e., no multiple edges between two Some basic structural of a graph were discussed in Section 17.2. in g are said to be disjoint if they do not have any vertex in common. Two are said to be each other if they have one and only one vertex in common: that intersect (or cross each other) at one and only one vertex. For 1 < y ::: q. let P be a set of paths of y - 1 in that satisfies

930

Chapter 17

Low-Density Parity-Check Codes

the following constraint: Any two paths in P are either disjoint or singularly crossing each other.

This constraint is called the disjoint-crossing (DC) constraint. Let n = IPI denote the number of paths in P and q0 denote be the number of vertices in 9 that are covered by ( or on) the paths in P. For simplicity. we call the vertices in 9 that are covered by the paths in P the vertices in (or of) P. We form a q0 x n matrix lHl = [hi.J] whose rows correspond to the q0 vertices in P and whose columns correspond to the n paths in P. where hi.J = 1 if and only if the ith vertex Vi of P is on the jth path in P: otherwise. h;,J = 0. This matrix is called the incidence mntrix of P. The columns of this matrix are called the incidence vectors of the paths in P with respect to the vertices in P: the jth column simply displays the vertices on the jth path of P. Because the length of each path in Pis y ~ 1, there are y vertices on each path in P. Therefore. each column of lHI has weight y. The weight of the ith row of IHI is equal to the number of paths in P that intersect at the ith vertex vi of P. It follows from the DC constraint that no two columns ( or two rows) of IHI can have more than one I -component in common. If y is much smaller than the number q0 of vertices in P, IHI is a sparse matrix. Then, the null space C over GF(2) of lHl gives an LDPC code of length n whose Tanner graph does not contain cycles of length 4. The minimum distance of C is at least y + 1. If the rows of lHI have the same weight p (i,e., each vertex of Pis intersected by p paths in P). then C is a (y. p )-regular LDPC code. The path set P can be constructed by using a trellis T of y - 1 sections with y levels of nodes, labeled 0. 1, · · · , y - 1. Each level of T consists of q nodes that are simply the q vertices of Q. For O ::= k < y ~ 1, a node vi at the kth level of T and a node v.i at the (k + l)th level of Tare connected by a branch if and only if (vi. VJ) is an edge in Q. This ( y - 1)-section trellis T is called the path trellis of 9 of length y ~ 1. A path of length y - 1 in 9 is a path in T starting from an initial node at the 0th level of T and ending at a terminal node at the (y - l)th level of T such that all the nodes on the path are different. Figures 17.40(a) and 17.40(b) show a complete graph (i.e .. every two vertices are connected by an edge) with seven vertices and its path trellis of length 2, respectively. To find a set P of paths of length y - 1 in 9 that satisfies the DC constraint, an extend-select-eliminate (ESE) algorithm [59] can be devised to parse the path trellis T of Q. Suppose we have parsed the trellis T up to the kth level of T with the ESE algorithm, with O ::= k < y - 1. For 1 ::= i ::= q. let Nu denote the number of survivor paths of length k terminating at the ith node of the kth level of T (i.e., the paths that satisfies the DC constraint). Let Pi.k denote this set of survivor paths. Now. we extend all the paths in Pi .k to the (k + 1)th level of T through the branches diverging from node i. At node i of the (k + l)th level of T. we select a set Pi.k+l of paths of length k + 1 that satisfies the DC constraint and eliminate all the other paths that terminate at node i. The result is a survivor path set Pu+1 at node i of level k + 1 of T. This ESE process continues until the (y - l)th level of Tis reached. Then, the union of the survivor sets, Pr.y-1, P2.y--l, · · · , P,1.y-1, gives the path set P. For 1 ::= i ::= q and O ::= k < y, all the survivor paths in Pi.k start from different initial nodes at the 0th level of T and intersect only at node i of the kth level of T. Two ~11n1ivnr p,ith~ tPrmin,iting ,it turn r1ifferPnt nnr1es at thP Hh 1PvP1 nf T ,irp Pill1pr disjoint or cross each other only once at a node of an earlier level of T. Selection of

Theoretic LDPC Codes

!9131

s (a)

of

(c)

(b)

of seven its the DC constraint found

of

trellis of length 2, and a set the ESE algorithm.

survjvo:c one node of :rhe kth level of T affects the selection of survivor at the other nodes of the same level of T. To maintain all the sets of survivor at each level of T at about the same size, we create a list of the nodes at each leveL The node ·with the smallest number of survivor of survivor path selection at level k + 1 of whereas the one with the number of survivor at it at level k of T has the lowest for the survivor path selection at level k + 1 of T. WCHURrn.cu.sh

graph 9 of of length 2 that in terms of the

the ESE seven vertices shown in satisfies the DC constraint in vertices (1. 2.

We see that the matrix for this set of

(L 3, 7), (L 5, 6) CL

6,

4.

(4. 5, 7).

in P covet ciU seven vertices of the graph. The incidence 1s

lli1 ==

l 1 1 0 0 1 0 0 1 1 0 l 0 1 0 1 0 0 0 0 0 0 1 1 0 0 0 1 0 1 0 1 0 0 1

0 0 1 1 0 1 0

0 0 0 1 1 0

1

(17.88)

J

which has constant row and column The null space of lflI gives a (3, 3)regular (7, 3) LDPC code ·with a minimum distance of 4 whose Tanner graph does not contain cycles of length 4, but it does contain of length 6. One such cycle is depicted by the lines shown in 17.41.

932

Chapter 17

Low-Density Parity-Check Codes

2

3

4

s

6

7

FIGURE 17.41: The Tanner graph of the (3, 3)-regular LDPC code generated by the parity-check matrix given by (17.88).

The graph-theoretic construction of LDPC codes given here is very general and can be applied to any connected graph Q. Because the selection of survivor paths at a node of each level of the path trellis Y of 9 is not unique, the path set P obtained at the end of the ESE processing is not unique, and hence the code constructed is not unique. !EXAMPLE 11.315

Suppose we start with a complete graph 9 with 400 vertices. We choose y = 6. The path trellis Y of 9 is a 5-section trellis with six levels of nodes, with each level of Y consisting of 400 nodes. Each path of length 5 in g is a path in T. Applying the ESE algorithm to process Y, we obtain a set P of 2738 paths of length 5 that satisfies the DC constraint. The paths in P cover all 400 vertices of the graph. The incidence matrix Hof Pis a 400 x 2738 matrix with column weight 6. The null space of IHI gives a (2738, 2339) LDPC code with rate 0.8543 and a minimum distance of at least 7. The bit- and block-error performances of this code with SPA decoding are shown in Figure 17.42. At a BER of 10~ 6 , the code performs only 1.7 dB from the Shannon limit. Note that the number of parity bits of the code is one less than the number of vertices of the code construction graph. For a connected graph g = (V, E) with a large number of vertices, especially a complete graph, processing the path trellis of Q of length y - 1 to construct a set of paths of length y - 1 in g to satisfy the DC constraint with the ESE algorithm may become very complex. To overcome this complexity problem, we can take a divide-and-conquer approach [59]. Consider a complete graph g with q vertices. Suppose we want to construct a set P of paths of length y - 1 in g that satisfies the DC constraint. We first divide Q into y blocks, 91, g1, · · · , Qy, of equal (or nearly equal) number of vertices. Each block Q; is a complete subgraph of Q, and any two blocks are connected by edges. For each block Q; we construct a set P; of paths of lengthy - 1 in Q; that satisfies the DC constraint using the ESE algorithm. Clearly, Pi, P 1 , .. · , Py are disjoint, and two paths from two different sets, P; and Pj, do not have any vertex in common. Next, we form a trellis Y" with y levels of vertices, labeled 0, 1, ... , y - 1. For O :s i < y, the i th level of ye consists of q; +1 nodes that correspond to the q;+1 vertices in block 9;+1· For O :s i < y - l, the nodes at the ith level of ye are connected to the nodes at the (i + l)th level of ye by

£,/N11 (clB)

vertices in Q1+~.

branches thc:t 8re the Ti,is - I r;rocess ';re ,,vith the ESJE

trellis for the bjocks of Q,

J-.:-\

YVf':

8HiC{

cross each otheT. For each CrOSSc-:s \Nith it. rfhc union U ···U

Py

UP'

Q that satisfies the DC constraint The

r

Of

treHis ye vvith tb_e ESE is about the same 8S y -- 1 of each b]ock ,vith the ESE trellis of "i/Ve forn1 the incldence rnatri;[ JH1 for P. the null space of Im of 4. an LDPC code Yvhose Tanner does not contain

Consider the vertice~. Let y = 6. Vle divide Q into six each block is a with 97 vertices. the divide-anclconquer ESE algorithm to process the path trellis of S for each block and the trellis for the blocks . we obtain a set P of 10429 of 5 that satisfies the DC constraint and coveKs all 582 vertices of the graph. We form the

934

Chapter 17

Lovv-Density Parity·-Check Codes

incidence matrix !HI for P, which is a 582 x 10429 matrix. The null space of lH gives a (10429., 9852) LDPC code with rate 0.945 and a minimum distance of at least 7 whose Tanner graph does not contain cycles of length 4. The bit- and block-error performances with SPA decoding are shown in Figure 17.43. At a BER of 10 6 , the code performs 1.05 dB from the Shannon limit. We also can construct a path set P of lengthy - 1 in a connected graph Q that satisfies the DC constraint by choosing one path at a time through the ( y -1 )-section path trellis T of Q. We begin by choosing any path in T as the first path in P. We delete all the branches on this path from T. Then, we trace the modified path trellis and choose the second path that either has no common vertex with the first path in P or intersects with the first path in Pat one and only one vertex (DC constraint). Again, we delete the branches on the second path from the modified path trellis and obtain a new modified path trellis. Then, we trace this new path trellis and choose the third path that satisfies the DC constraint with the two paths in P. We delete the branches on the third path from the path trellis and then start a new path selection. We continue this process of path selection and removal of branches on a selected path from the path trellis. Each time we select a path it must satisfy the DC constraint with the paths already in P. When a node in the path trellis has no incoming or outgoing branches, it is removed from the path trellis together with all the branches still attached to it. We continue this process until no path in the path trellis can be chosen without violating the DC constraint, or the path trellis 1011

,,.----.----~----,--s----,---~----,-----,---~ 582 vertices, L

=

5. (10429, 9852)

6 subgraphs of 97 vertices each SPA bit ---e,--. SPAblock Uncoded BPSK

---- ---

_

10- 7

l

10-R

1

2

3

4

5

6

7

8

9

£,/No(c!B)

FIGURE 17.43: Error performance of the (10429, 9852) LDPC code constructed hased on the complete graph of 582 vertices using the divide-and-conquer ESE algorithm.

Sec(ion 1 .1

Construction of LDPC

Based on BIBD

935i

an LDPC code. This code constru.ction n10de! of the first random LDPC code construction in Section 17.14- with the ~-~,mun,•~ of n -· k 'Vertices as the code construction Other constructions of LDPC codes based. classes of with have also been lS

·u 7 ~ i 71 «:C~li\DS. r~ucc_~·-,r:ccJJ~\,D r:i[~ tro)rr CGJ:!i5 r:s ~.l,10""sr~[2J 1

1

(


r~3L(JC:~{ con,binatorial m2thematics. and with a view to

of LJ[:iPC codes. A BIBD of X is a collection of 11 such that the ,o of the n and the number in XO n, q, JI. p, and 'A. Instead of a BJ[BD can be described a q x n matrix li-JI = [h;.J] io the q of X and whose columns to the n = 1 if and if the ith block.

i.r1 com1non. If matrix of an LDJPC LDPC

coadition of 8 >.. = 1. then lSI meets aH the conditions of a code Definition :i7.1. n 1;;.1lJose 'JCanner <=":7

u

~-~'"'',~'-·"~"~'~·•uu

:-'l ~;')

~~_:;n

blocks: {Xj. x2. .x5,.,6},

X3, x5}, {x2,X6,X7}.

..Y4 ..'C6}, {x1.x3.x7}.

form a EJBD for the set X. block consists of y appears in ,o = 3 and eYery two appear block. Hs incidence matrix Im is

IHl

The nuH space of IHI ~~~·~=----~-

==

1 1 0 1

0

0

0

1 0 0 1 1 0 0 1 1 0 1 0 1 0 0 1 0 0 0 0 1

1 0 1 0 0 1 0 1 1 0 1

0 0 0

1 0

1 0 0 0 1

a (7, 3) LDPC code with a minimum distance of 4.

936

Chapter 17

Low-Density Parity-Check Codes

Combinatoric design is a very old and rich subject in combinatorial mathematics. Over the years. many BIBDs have been constructed with various methods. Extensive coverage can be found in [41. 58]. There are many classes of BIBDs with ii. = 1, which can be used for constructing LDPC codes whose Tanner graphs do not contain cycles of length 4 [64. 65]. Construction of these classes of designs requires much combinatorial mathematics and finite-group theory background and will not be covered here; however. we do present one special class of BIBDs with parameter 1c = 1 constructed by Bose [63]. Let t be a positive integer such that 20t + 1 = pm, where p is a prime; that is. 20t + 1 is a power of a prime. Suppose the field GF(pm) has a primitive element a that satisfies the condition a 41 - 1 = ac, where c is a positive odd integer less than pm. Then. there exists a BIBD for a set X of q = 20t + 1 objects with n = t (20t + 1) blocks, each block consists of y = 5 objects, each object appears in p = St blocks, and every two objects appear together in exactly 1c = 1 block. Let the elements of G F(p 111 ). 0, a 0 = 1, a. a 2 , ... , a p"' - 2 • represent the 20t + 1 = pm objects of the set X. Then, the BIBD for X is completely specified by the following t base blocks: Bi

= {0t2i. a2i+41, a2i +81. 0t2i+121. a2i+l61},

(17.89)

where O :s i < t. All the 11 = t (20t + 1) blocks of the BmD are obtained by adding each element of G F(pm) in turn to each of the t base blocks. The incidence matrix IHI of this BIBD is a (20t + 1) x t(20t + 1) matrix with column and row weights 5 and St. respectively. The density of His 5/(20t + 1). which is very small fort :::: 2. Because 1c = 1. it follows from Definition 17.1 that the null space of H gives an LDPC code of length n = t (20t + l) whose Tanner graph is free of cycles of length 4. In fact. H can be put in circulant form. For O ::: i < t, let vi be the incidence vector of the base block B; which is a p 111 -tuple with 1's at locations 2i, 2i + 4t. 2i + St, 2i + 12t, and 2i + 16t. Let Gibe a (20t + 1) x (20t + 1) square circulant matrix obtain by shifting Vi downward cyclically 20t + 1 times (including the zero shift). All the columns ( or rows) of Gi are different and are the incidence vectors of 20t + 1 different blocks. Then. 1HI can be put into the following circulant form: H

=

[Go G1 · · · G1-1].

(17.90)

With lHI in circulant form, the null space of H gives a quasi-cyclic BIBD-LDPC code oflength 11 = t (20t + 1). For 1 ::: k ::: t, we can choose k circulants. Go, G1, · · · , G1c-1. to form the following matrix: H(k) = [Go G1 · · · G1c-1]

(17.91)

with column and row weights 5 and Sk. respectively. Then, the null space of lHI(k) gives a quasi-cyclic LDPC code of length n = k(20t + 1). Suppose ,x.,e decompose each circulant Gi in JH(k) into five {20t -1-1) >< (20t -l-1) circulant permutation matrices by row decomposition as presented in Section 15.12,

D,

~

I lJ G1.,

G2., G3.,

-

(;;, ..... ,,

LGs.i

(17.92)

Section 1

Construction

LDPC Codes Based on BIBD

each circulant

IT});,

931

we obtain the

5 >< k aiT3Y of

[ll

l

=

(17.93)

The nuH space of JU a a minimum distance of at least 6 and rate of at k. H .ve remove one row of permutation matrices from IO\ the resultan,t submatrix :Jf TIJJ a code also with a minimum distance of at least 6 but of at least (k - 4) / k. If we move two rows from lOJ, then the resultant subrnatd,: a LDPC code with a minimum distance of at least t!. and rate of at least (k - 3) / k. 1

vvhich is a 20t + 1 = element a that satisfies th'c condition integer less than 421 (see Problem 421 This BJfBD consists of 11 each appears in Lett

= 21.

The field GF(421) does have a - l = o/, vvith c as a positive odd there is a BIBD for a set X of each block consists of y = 5 and >c = 1. The 21 base blocks are

lO"

10-1

~ .o

10-2

"'

.D 0

& 10-1 6

'"'

'?'"'

.'<

u

10-4

..s

sci3

10-' \\

I

10··<,

\t

10-7

2

0

3

4

s

6

7

E1JN11 (dB)

FIGURE 17.44: The error

"=~1·,,:,,.,,~r\,[,'.D,v,"

code given in Example 17.39.

of the

quasi-cyclic BIBD-LDPC

938

Chapter ·17

Low-Density Parity-Check Codes

for i = 0 to 20. Based on these 21 base blocks, we can form the incidence matrix lHI for the BIBD in the following circulant forms: H

=

[Go, G1. · · · , G20],

where G; is a 421 x 421 circulant. Suppose we choose k = 20 and decompose each G; into five circulant permutation matrices by row decomposition. We obtain a 5 x 20 array D of 421 x 421 circulant permutation matrices, which is a 2105 x 8420 matrix with column and row weights 5 and 20, respectively. The null space of llJJ gives a (8420, 6319) quasi-cyclic BIBD-LDPC code with rate 0.7504 and a minimum distance of at least 6. The error performance of this code is shown in Figure 17.44. At a BER of 10- 6 , it performs 1.4 dB from the Shannon limit.

17.18

CONSTRUCTION Of U)?( CODIES IBASIED ON SHORTIENIED RS CODIES WffH TWO !NfORMAT!ON SYMBOLS

In earlier sections of this chapter we used several branches of combinatorial mathematics as tools for constructing LDPC codes. In this section we present an algebraic method for constructing LDPC codes based on shortened RS codes with two information symbols [66]. This method gives a class of LDPC codes in Gallager's original form. Let a be a primitive element of the Galois field G F( q) where q = v' is a power of a prime. Let p be a positive integer such that 2 ::: p < q. Then, the generator polynomial of the cyclic (q - 1. q - p + 1. p - l) RS code C over G F( q) of length q - 1. dimension q - p + 1, and minimum distance p - 1 is given by (7.2), g(X)

=

(X - a)(X - a 2 ) ... (X - aP- 2 )

=go+ g1X

+ g2X 2 + · · · + xP- 2.

where gi E GF(q ). The generator polynomial g(X) is a minimum-weight code polynomial in C, and hence all its p - 1 coefficients are nonzero. Suppose we shorten C by deleting the first q - p - 1 information symbols from each codeword of C. Then, we obtain a (p, 2, p -1) shortened RS code Cb with only two information symbols. A generator matrix of this shortened code is given by 1 All the linear combinations of the two rows of
c1: 1l =

{cv: c E GF(q))

of a codewords in C,, forms a one-dimensional subcode of C,, with mmmmm dist~nce p. Two codewords in l differ at every location. We p;,rtition Cb into q

cf

Section 1 . 1

Construction crf LDP(:

"o~r~~ c.101,A~t

Based

1 '" .c,.e ,.~,U-' J·coC \

code·-.;;vords in ar;.y different cosets" d_lffer '.:rt least at p ·- 1 locations. ·y;/ e cnTcn1ge ths q code-\A/Ords of /)

I .,

n·· 1Ll1 ,uJIJ!.

~~

21H p

~

t\NO

)==(0.0,·

Lei

.O,l,0.0_

(v 1• u2 . · · ·• vi') be 81 codeword in VJ of'? 'iNith lts loc01t~on vector ;;z( vi) .

(1

Fo:;- l <

Y=

a

Vie

JBeca\Jse the location

at coost one 1

Le1 ( 17

location vectors i-:"1

cf

,.·1

codev'!ords

tn_r-::;

common . Nhose ro\vs are the q ec1ich vsctor in /J ) ) 1

I

.. ,,. , ) ]--"' •!,.

....

du

y

]. Ivfatrix location

lowtion motrix of the CO'.>:t }.

(17.98)

940

Chapter 17

Low-Density Parity-Check Codes

has the following structural properties: (1) no two rows in the same matrix H; have any 1-component in common; and (2) no two rows from two differen( matrices, lfff; and H 1, have more than one 1-component in common. !EXAMPLE 11.41()

Consider the Galois field GF(2 2 ) constructed based on the primitive polynomial p(X) = 1 + X + X 2 . Let a be a primitive element of GF(2 2 ). Then, the four elements of G F(2 2 ) are O = a 00 , 1 = a 0 , a, and a 2 = 1 +a. The location vectors of these four field elements are

zo = (100 0).

:ii1

=

(0 100),

Za

=

(001 0).

Let p = 3. The cyclic (3, 2, 2) RS code C1, over GF(2 2 ) has generator polynomial g(X) = X + a and generator matrix

G=[~ !

~l

The code has 16 codewords, and its minimum distance is 2. Adding the two rows of G, we obtain a codeword w = (a, a 2 • 1) with weight 3. The following set of four codewords: {/h:

f3

E

GF(2 2 )} = {(O, 0, 0), (a, a 2 , 1). (a 2 , 1. a). (1, a. a 2 )} 1

forms a one-dimensional subcode C1~ J of C1, with a minimum weight of 3. We 1 partition C1, with respect to C1~ l and obtain the following four cosets:

ci1) = c1;2l =

{(a.1. 0), (0, a. 1). (1, 0. a). (a 2 • a 2 • a 2 )},

cfl =

(a 2 , a, 0). (l, 1, 1). (0, a 2 , a). (a, 0, a 2 ),

4

(1. a 2 , 0), (a 2 , 0, 1). (a. a. a). (0, 1, a 2 ).

Ci

) =

{(0, 0, 0). (ot, a 2 , 1). (ct2, 1. a), (1, (X, a 2 )},

The symbol location matrices of these four cosets are

H1 =

~w

zcct )) = 1

~c(?))

U2 = L

In;

L1,

=

l~

1 l~ ~

~ Z(c,;''l ~ I~

Lo

0 0 0 1

0 0 1 0 0 1 0 0

0 1 0 0 0 0 1 0 0 0 0 1 0 1 0 0

0 0

1 0

0 1

()

0

1 0 0 0 0 0 0 ] 0 0 0 1 0 0 1 0

0 1 0 0

1 0 0 0 1 0 0 0 0 0 0 1

0 0 1 0 0 1 0 0 0 0 0 1 1 0 0 0

1 0 0 0 1 0 0 0 1 0 0 0

0 1 0 0

0 0 1 0

1 0 0 1 0 0 0 0

0 0

1 0 0 0

(\ V

n ~l

~J

n 1_!_

I

J

(onstruction o-f LDPC

Section 1 .1

r· I

l.

Let y

== 3,

0 0 0

1 0 0

0

l

1

1 0

0 0

0 0 0

0

0

0

0 0 0

1 0 0 r,

1 0 0 0 0 1 0 0 0 0 1 0 u 0 0 1

u"

1 0 0

I

J I I

L

0 0 1

0 0 1 0 0 1

1

0

0 0

0 0 0 1 0 0 1 0

0

0 0

1 0

0 1 0 1 0 0 0 0 1 0 0 0 0 0 1

0 0 1 0

1 0

1

0 0 0

0

1 0

1

(\

J.

V

0 0

0 0

V

0

I0

L

1

0 r, u l 0

0 l 0 0

0

0 0 0 'I J.

"

\J

0 0

1 l

0 0 0

0 1 0 0 0 0 1 0

1 (\ V

(3) RS-·based The null spsce a rate l/3 and a rninhnu1n distaEce of 6. The lovver bound JI dist~n1ce is ~.. ~ - - . . . . , , - . - - . ~ ~ ~ ~ - - · - " · · · -..-..

~~1

"l

I

J

rru~trix:

I r

0 0

1 0

~Ale forrn the

I

Codes

Ba sec! on Shortened

-......,.--..==-~=---------··==~-

1 0 0 1 0

0 0 0

0 0

1

0

0 ·-1 0 0 1

I

1 0 0 0 0 1 0 0 0 0 l 0 0 0 0 1 1 0 l

0 0 0

0 nV

+ 1 on

0 0 l 0

0 0 0

1

I

J

code with the minimum

-~----~~~.--,~-·~·-~=-~=·

-~~~~~~~-,~.~-

such that 1 ::: JI ::: q. VVe form the

JICJ >< pcJ

ILnatrix over

(y)

=

J

fr'Jc A (JI) consists matrix with column and row oflh1c.4(J1)have more than one is 1 / q ., which is small for is a sparse matrix that has all the structural of the LDPC code in Definition 17.L form (1 the null space of an LDPC code CaA(J!) of 11 = pq with a minimum distance of at least JI + 1 for odd JI and JI + 2 for even JI. The rate of this code is at least (p - JI)/ p. and JI, we can construct a sequence of and rates vvith p = JI. y + L · · · . CJ - L For any choice of CJ and p, we can construct a sequence of codes of n = pq with V8Iious rates and minimum distances for JI = L 2. · · · . p. For p = q - l, ()I) is Because the construction is based on the (p. 2, p -1) shortened RS code

'~M,2

Chapter 17

Low-Density Parity-Check Codes

C1, over the field GF(q ), we call Cb and GF(q) the base code and the construction field, respectively. rEXAMl?LE 17.41

Let GF(2 6 ) be the construction field. Suppose we choose p = 32. Then, the base code is the (32, 2, 31) shortened RS code over GF(2 6 ). The location vector of each element of GF(2 6 ) is a 64-tuple with a single 1-component. Suppose we choose y = 6. Then, the RS-based Gallager-LDPC code Cc A (6) constructed is a (6, 32)regular (2048, 1723) LDPC code with rate 0.841 and a minimum distance of at least 8. Hs error performance with SPA decoding is shown in Figure 17.45. At a BER of 10- 6 , the code performs 1.55 dB from the Shannon limit and achieves c1. 6-dB coding gain over the uncoded BPSK. If we choose p = 63, the base code is then the (63, 2, 62) shortened RS code. We set y = 60. The RS-based Gallager-LDPC code is a (60, 63)-regular (4032, 3307) quasi-cyclic code with rate 0.82 and a minimum distance of at least 62. The error performance of this code with SPA decoding is shown in Figure 17.46. At a BER of 10- 6 , it performs 1.65 dB from the Shannon limit. Owing to its large minimum distance, no error floor is expected. This code can also be decoded with one-step majority-logic decoding to correct 30 or fewer random errors. Jf it is decoded with weighted BF decoding, an effective trade-off between error performance and decoding complexity can be achieved.

10°

- - Uncoded BPSK lO

I

._ FER (2048, l 723) BER (2048, 1723)

---- --- -- .... --

-

Shannon limit

lO " -

lO Xc__~-"-~~-'--'---'-~~-'-~~I~~-"-~~-'--~·--'-~~~ 6 2 3 7 8 0 5 9

E 1,/N0 (dB)

FIGURE 17.45: Error performance of the (2048, 1723) RS-based Gallager (6, 32)regular LDPC code with construction field GF(26 ) given in Example 17.41.

Cons·LfUCtion

Section 17.'I

10" ~ - - - · - - ,

~

-

l.DPC

on Shortened RS Codes

9l43

-- 10/ --~-~1----i---------~

"

i\l

'

~CTo ~

''V'

-

Uncoded BPSK FER (4032, 3307) BER (4032, 3307) : Shannon limit

of the 3307) RS-based Gallager ( 60, LDPC code with construction field GF(2 6 ) given in Exam-

11

10

~,---~--~-""r---r---1 - - - - - - - - - ~ \

--·· Uncoded BPSK ~~? FER (8192, 6754) --,:.,, BER (8192, 6754) = ·~ Shannon limit

ii) 'J

";

0

2

7

3

8

9

Et/N11 (dB)

FIGURE 17.47: Error of the regular LDPC code with construction field

6754) RS-based Gallager (6, 32)given in Example 17.42.

944

Chapter 17

Low-Density Parity-Check Codes

!EXAMPLE 17.42

Suppose we use GF(2 8 ) as the construction field and the (32, 2, 31) shortened RS code over GF(28 ) as the base code. We set y = 6. Then, the RS-based GallagerLDPC code is a (8192, 6754) code with rate 0.824 and a minimum distance of at least 8. Its error performance with SPA decoding is shown in Figure 17.47. At a BER of 10-6 , it performs 1.25 dB from the Shannon limit and achieves a 6.7 dB gain over the uncoded BPSK. The foregoing algebraic construction of LDPC codes is simple and yet very powerful. H gives a large class of regular LDPC codes. Codes in this class can be decoded with the SPA, weighted BF, BF, or one-step majority-logic decodings to provide a wide range of trade-offs between error performance and decoding complexity. The matrix given by (17.99) is an array of permutation matrices that is in exactly the same form as the matrix given by (17.72). Hence, it can be masked to generate new LDPC codes. 17.19

CONCATENATIONS WiTH llDPC AND TURBO CODES

In most applications of concatenated coding for error control, RS codes are used as the outer codes, and they are, in general, decoded with an algebraic decoding algorithm, such as the Berlekamp or Euclidean algorithm presented in Chapter 7. Ha 1011 ..- Bit - Block

10-1

E

10-2

'\

13

"'

..D 0 H 0..

10-3

H

0

H H
~ u

10-4

0

~

ill

10-0 -

10-6 Shannon limit

10-7 0.5

1.5

2

2.5

3

3.5

4

4.5

E1/N11 (dB)

FIGURE 17.48: Bit- and block-error performance of a concatenated LDPC-turbo coding system with a turbo inner code and an extended EG-LDPC outer code.

the inner tuibo decoder can be of st•'ch a con1bina!ion

code as the

f\!VO

constituent

is 0.75. 'I'he bit- and block~. of this concatenatecl LDPC·turbo are shown ' it 0.7 dB a·way from the Shannon limit This schen-ie used the l_\TA:SA, TDRS whose overall rate is 0.437. P":i_notbe:r forrn of concatenation. of L.JDP·C and turbo codes is to use u.n LDP(: as code as the

,;odes to achieve very !'::rror fioor 21t c:1 very Io-lN error

used as the constituent error floor with an

without

Jl7.:!L Does the

matrix the conditions of the matrix of an Definition 17.l? Determine the nrnk of this matrix and the codevvords of its null space. ·-

IHI=

! 0 1 l 0 l 0 0 1 0 l 0 l 0 0 0 l 1 0 0 0 1 0 0 0

l 0 0 0

0 0 1 0 L L 0 0 0 0 1 1 0 0 l 0 1

0 0

-i

I .J

946

Chapter 17

Low-Density Parity-Check Codes

172 Form the transpose lHl:T of the parity-check matrix IHI given in Problem 17.1. Is !HIT a low-density parity-check matrix? Determine the rank of !HIT and construct the

code given by the nuli space of HT. 17,,3 Prove that the (n, 1) repetition code is an LDPC code. Construct a low-density parity-check matrix for this code. 17A Consider the matrix H whose columns are all them-tuples of weight 2. Does lHl satisfy the conditions of the parity-check matrix of an LDPC code? Determine the rank of H and its null space. 17,5 The following matrix is a low-density parity-check matrix. Determine the LDPC code given by the null space of this matrix. What is the minimum distance of this code? 1 1 0 1 0 0 0 0 1 1 0 1 0 0 0 0 1 1 0 1 0 0 0 0 1 1 0 1 H= 1 0 0 0 1 1 0 0 1 0 0 0 1 1 1 0 1 0 0 0 1 17Ai Prove that the maximum-length code of length 2111 17.7 17.8 17/;;J

17.10 17.11 17.12

17.13

17.14 17.15

17.16

17.17 17.18

17.19 17.20

17.21

1 presented in Section 8.3 is an LDPC code. Construct the Tanner graph of the code given in Problem 17.1. Is the Tanner graph of this code acyclic? Justify your answer. Construct the Tanner graph of the code given in Problem 17.2. Is the Tanner graph of this code acyclic? Justify your answer. Construct the Tanner graph of the code given by the null space of the parity-check matrix given in Problem 17.5. Does the Tanner graph of this code contains cycles of length 6? Determine the number of cycles of length 6 in the graph. Determine the orthogonal check-sums for every code bit of the LDPC code given by the null space of the parity-check matrix of Problem 17.5. Prove that the minimum distance of the Gallager-LDPC code given in Example 17.2 is 6. Determine the generator polynomial of the two-dimensional type-I (0, 3)th-order cyclic EG-LDPC code constructed based on the two-dimensional Euclidean geometry EG(2, 23 ). Determine the parameters of the parity-check matrix of the three-dimensional type-I (0,2)th-order cyclic EG-LDPC code c2b.c(3, 0, 2). Determine the generator polynomial of this code. What are the parameters of this code? Determine the parameters of the companion code of the EG-LDPC code given in Problem 17.13. Decode the two-dimensional type-I (0, 3)th-order cyclic EG-LDPC code with one-step majority-logic decoding and give the bit- and block-error performance for the A WGN channel with BPSK signaling. Repeat Problem 17.15 with BF decoding. Repeat Problem 17.15 with weighted majority-logic decoding. Repeat Problem 17.15 with weighted BF decoding. Repeat Problem 17.15 with SPA decoding. Decode the three-dimensional type-H (0, 2)th-order quasi-cyclic EG-LDPC code given in Problem 17.14 with SPA decoding, and give the bit- and block-error performance of the code for the A WGN channel with BPSK signaling. Consider the parity-check matrix JHI~b.c of the three-dimensional type-I (0, 2)thorder cyclic EG-LDPC code given in Problem 17.13. Split each column of this

r!latrix into five resuit is a new code. Decode this code 1'722

IL7,2~117,25

Jl7J;(6

its bit- and block--error

matrix of the 6, p = LJ., and y = 3. Choose column submatrices such that >,. is no greater than l. Prove that the Tanner of a finite-geometry LDPC code contains of 6. Enun,erate the number of 6. P:;:ove that the minimum distance of an LDPC code must be even. Use the result to prove the lower bound on minimum distance (17.68). Construct an LDPC code six bundles of lines in the two-dimensional Euclidean its bitand block-error ne,.-t,w;r,Pw,""'"0 Construct a masked EG-Gallager LDPC code of length the incidence matrices of eight parallel bundles of lines in set p = 32, and form an 8 >< 32 matrices. To construct such a LJ. and 16, using four 111

Jt7o23

distribution. The

=

using SPA. J1'1\Z7 The incidence vectors of the lines in 25 ) not passing the origin form a 1023 >< 1023 circulant G with weight 32. Construct a rate-1/2 quasi-cyclic code of length 8184 decomposing G into a 4 x 8 array of 1023 >< 1023 circulant the bit- and block-error performance of the code 172Jl Prove that there exist a primitive element 0/ in and an odd positive c less than 2.41 such that a 64 + 1 = 0/c. Jl'7o29 a concatenated turbo coding system with a LDPC code of your choice as the outer code. Construct the inner tmbo code using the secondorder HJ\lj[ code as the code. Give the bit-error

1L R. G.

Check

'" lRE Trans.

Check

IVHT

1962.

3,, R.

M. Tannec "A Recursive IT-27: 533-47, and R. M.

"IEEE Trans.

"Near Shannon Limit Performance of Low " Electron. Lett., 32 (18): 1645-46, 1996.

~t M. 42

"IEEE Trans. Inform.

Nover11ber 1996. "Linear-Time Encodable 42 1723-31, November 1996.

7, M. C.

1963.

and D. Jr. C. MacKay, "'Low Density "IEEE Commun. Lett., 165-67, June 1998.

"IEEE Trans. Check Codes over

948

Chapter 17

Low-Density Parity-Check Codes

~t M. G. Luby, M. Mitzenmacher, M. A. Shokrollahi, and D. A. Spielman, "Improved Low-Density Parity-Check Codes Using Irregular Graphs and Belief Propagation," Proc. 1998 IEEE Intl. Symp. Inform. Theory, p. 171, Cambridge, August 16-21, 1998. 9. D. J. C. Mackay, "Gallager Codes That Are Better Than Turbo Codes," Proc. 36th Allerton Conj Commun., Control, and Computing, Monticello, HI., September 1998.

JW. - - - , "Good Error-Correcting Codes Based on Very Sparse Matrices," IEEE Trans. Inform. Theory, 45 (2): 399-432, March 1999.

JL:L - · - , "Sparse Graph Codes," Proc. 5th Intl. Symp. Commun. Theory and Applications, pp. 2-4, Ambleside, UK July 11-16, 1999. U. T. Richardson, A. Shokrollahi, and R. Urbanke, "Design of Capacity-

Approaching Irregular Codes," IEEE Trans. Inform. Theory, 47 (2): 619-37, February 2001.

1:?t T. Richardson and R. Urbanke, "The Capacity of Low-Density Parity-Check Codes Under Message-Passing Decoding," IEEE Trans. Inform. Theory, 47: 599-618, February 2001. Jl4l. S. Y. Chung, G. D. Forney, Jr., T. J. Richardson, and R. Urbanke, "On the Design of Low Density Parity Check Codes within 0.0045 dB of the Shannon Limit," IEEE Commun. Lett., 5: 58-60, February 2001. Jl5. Y. Kou, S. Lin, and M. Fossorier, "Low Density Parity Check Codes Based on Finite Geometries: A Rediscovery," Proc. IEEE Intl. Symp. Inform. Theory, Sorrento, Italy, June 25-30, 2000. Jl6. S. Lin and Y. Kou, "A Geometric Approach to the Construction of Low Density Parity Check Codes," presented at the IEEE 29th Communication Theory Workshop, Haines City, Fla., May 7-10, 2000. 17. Y. Kou, S. Lin, and M. Fossorier, "Low Density Parity Check Codes Based on Finite Geometries: A Rediscovery and More," IEEE Trans. Inform. Theory, 47 (6): 2711-36, November 2001. Jl8. Y. Kou, S. Lin, and M. Fossorier, "Construction of Low Density Parity Check Codes-A Geometric Approach," Proc. 2nd Intl. Symp. Turbo Codes and Related Topics, Brest, France, September 4-7, 2000. 19. S. Lin, Y. Kou, and M. Fossorier, ''Low Density Parity Check Codes Construction Based on Finite Geometries," Proc. GLOBECOM 2000, San Francisco, Calif., November 27-December l, 2000. 20. S. Lin, Y. Kou, and M. Fossorier, "Finite Geometry Low Density Parity Check Codes: Construction, Structure and Decoding," Proceedings of the Forney Fest, Kluwer Academic, Boston, Mass., 2000.

and Evaluation of Some "Proc. 2001 IEEE vol. 2, pp. Encodable Moderate"to appear in IEEE Trans. Commun, 2004.

2tlo S. L. I.,aurjtzen and D. J. Structures and Their 1988.

" J. R. Stat.

on "Eur. Trans. Teleconmmn., 6: 2;(6i,

R. J. McEliece, D. J. C. an Instance of Pearl's Belief Propagation Commun., 16: 140-52, 1998.

1955. ''Turbo " IEEE J. Select. Areas

"lterative of Models," IEEE J. Select. Areos v~uuv~

J. " IEEE Trans. 2001.

"Reduced Herafrve Check Codes," IEEE Trans. Commun., 47: 3@,

Ivi. Fossorier, Y. Kou, and S. Logic Decodable Codes Based on Belief Commun., 48 931 June 2000.

:3]_,

C. A. and P. "Near Shannon Limit ErrorCorrecting Coding and Decoding: Turbo Codes," Proc. 1993 IEEE Intl. Conf Commun., pp. 1064-70, 1993.

of '' IEEE Trans.

23:t C. Berrou and A. "Near Error and Turbo-Codes," IEEE Trans. Commun., 44: 1261-71, October 1996.

3:t S. Benedetto and G.

"Unveiling Turbo Codes: Some Results on Parallel Concatenated Coding Schemes," IEEE Trans. 42 (2): 409-28, March 1996.

:34L J. Hagenauer, E. Offer, and L. Papke, "Iterative Decoding of Convolutional "IEEE Trans. Inform. Theory, 42: ]§,

N. Deo, Theory with Applications to Engineering and Prentice Hall, Englewood N.J., 1974.

Science,

950

Chapter 17

Low-Density Parity-Check Codes

360 S. M. Aji and R. J. McEliece, "The Generalized Distributive Law," IEEE Trans. Inform. Theory, 46 (2): 325-43, March 2000. 370 N. Wiberg, "Codes and Decoding on General Graphs," Ph.D. Diss., Dept. of Electrical Engineering, University of Linkoping, Link oping, Sweden, April 1996

38. T. Etzion, A. Trachtenberg, and A. Vardy, "Which Codes Have Cycle-Free Tanner Graphs," IEEE Trans. Inform. Theory, 45 (6): 2173-81, September 1999. 3'\t G. D. Forney, Jr., "Codes on Graphs: A Survey for Algebraists," Proc. 13th

Intl. Symp. Applied Algebra, Algebraic Algorithms and Error-Correcting Codes, Honolulu, November 15-19, 1999. 4l!t R. D. Carmichael, Introduction to the Theory of Groups of Finite Order, Dover, New York, 1956.

411. H.B. Mann, Analysis and Design of Experiments, Dover, New York, 1949. 412. T. Kasami, S. Lin, and W. W. Peterson, "Polynomial Codes," IEEE Trans. Inform. Theory, 14: 807-14, 1968. 413. R. C. Bose and D. J. Ray-Chaudhuri, "On a Class of Error Correcting Binary Group Codes," Inform. Control, no. 3: 68-79, March 1960. 4141. W. W. Peterson and E. J. Weldon, Jr., Error-Correcting Codes, 2d ed., MIT Press, Cambridge, 1972. 4150 E. R. Berlekamp, Algebraic Coding Theory, McGraw-Hill, New York, 1968. 46. S. Lin, "On the Number of Information Symbols in Polynomial Codes," IEEE Trans. Inform. Theory, IT-18: 785-94, November 1972.

4170 E. J. Weldon, Jr., "Difference-Set Cyclic Codes," Bell Syst. Tech. J., 45: 1045-55, September 1966.

41~1o V. D. Kolesnik, "Probability Decoding of Majority Codes," Prob[. Peredachi Inform., 7: 3-12, July 1971.

419. R. Lucas, M. Bossert, and M. Breitback, "On Iterative Soft-Decision Decoding of Linear Block Codes and Product Codes," IEEE J. Select. Areas Commun., 16: 276-96, February 1998. 5Q]). S. Lin "Shortened Finite Geometry Codes," IEEE Trans. Inform. Theory, IT-18 (5): 692-96, September 1972.

51. H. Tang, J. Xu, Y. Kou, S. Lin, and K. Abdel-Ghaffar, "On Algebraic Construction of Low Density Parity Check Codes," Proc. 2002 IEEE Intl. Symp. Inform. Theory, p. 482, Lausanne, Switzerland, June 30-July 5, 2002. 52. H. Tang, J. Xu, Y. Kou, S. Lin, and K. Abdel-Ghaffar, "On Algebraic Construction of Gallager and Circulant Low Density Parity Check Codes," to appear in IEEE Trans. Inform. Theory, June 2004.

and K. Abdel-Ghaffac ''Construction of

":Near Sh.annon Liinit

2004. K. A,bdel-Ghaffar, "On Circulant Lo,v IEEE Intl. p. 200, 5, 2002. 57q C.L,

Introduction to Combinatorial

1968. 5i't C. J. Colbourn and l'-l J. Boca CFzC

Combinatoriai

1

1 ')

Theoretic Construe!IEEE Co11u11un. 7: 171

2003. ''Construction of Intl.

and LDPC " Proc. 2001 IEEE 2001.

6], P. 0 Vontobel and R Ml. Tanner, "Construction of Codes Based on Finite vv·.,o.HH_," Proc 2001 IEEE Intl. 2001.

,s;;t P. 0 Vontobel and H. Proc. 2002 IEEE Intl. June t:., 2002. cti:!lo R C. Ann. 1§t1Jo

the Construction of Balanced 1939.

,,u,,v,u~
Block Designs,"

B. Pkm1nar, B. I-Ionary, Y. J. and S. "Construction of Low Block Designs," to appear Check Codes Based on Balanced ~''"'-''"F in IEEE Trans. June 2004.

1/i!'.io B. Vasic and 0.

"Combinatorial Construction of Parity Check Codes for Herative " to appear in IEEE Trans. Inform. Theory, 2004. K. A,bdel-Ghaffar, and S. Lin. "Construction of LowCodes Based on Shortened Reed-Solomon Codes with ·• IEEE Commun. no. 7: 317-19. 2003.

( H IA P T fE R

18

All the coding schemes discussed so far have been designed for use with binary-input channels; that is, the encoded bits are represented by one-dimensional BPSK signals according to the mapping O -,. -./E., and l -,. +JE;, or O -,. -1 and 1 -,. +1 for unit energy signals. (We note here that even nonbinary codes, such as RS codes, are usually transmitted using binary signaling by representing each symbol over GF(2111 ) as a binary m-tuple.) In this case the spectral efficiency ;7 of the coded system is equal to the code rate R; that is, 17 = R ::s 1 bit/dimension or 1 bit/transmitted BPSK symbol, and at most one bit of information is transmitted each time a BPSK symbol is sent over the channel. Thus, since the bandwidth required to transmit a symbol without distortion is inversely proportional to the transmission rate, combining coding with binary modulation always requires bandwidth expansion by a factor of 1/ R. In other words, compared with uncoded modulation, the coding gains resulting from binary modulation are achieved at the expense of requiring a larger channel bandwidth. For the first 25 or so years after the publication of Shannon's paper, research in coding theory concentrated almost exclusively on designing good codes and efficient decoding algorithms for binary-input channels. In fact, it was believed in the early 1970s that coding gain could be achieved only through bandwidth expansion and that coding could serve no useful purpose at spectral efficiencies 17 ::: 1 bit/dimension. Thus, in communication applications where bandwidth was limited and large modulation alphabets were needed to achieve high spectral efficiencies, such as data transmission over the dial-up telephone network, coding was not thought to be a viable solution. Indeed, the modulation system design emphasis was almost exclusively on constructing large signal sets in two-dimensional Euclidean space that had the highest possible minimum Euclidean distance between signal points, given certain constraints on average and/or peak signal energy. In the next two chapters we introduce a combined coding and modulation technique, called coded modulation, that achieves significant coding gain without bandwidth expansion. Indeed, coding gain without bandwidth expansion can be achieved independently of the operating spectral efficiency of the modulation system. Thus, coded modulation is referred to as a bandwidth-efficient signaling scheme. In this chapter we discuss trellis-coded modulation (TCM) [1 ], a form of coded modulation based on convolutional codes, and in the next chapter we discuss block-coded modulation (BCM), based on block codes. Basically, TCM combines ordinary rate R = k/(k + 1) binary convolutionai codes with an M--ary signal constellation (M = 2k+l rel="nofollow"> 2) in such a way that coding gain is achieved without increasing the rate at which symbols are transmitted, that is, without increasing the required bandwidth, compared with uncoded modulation. For example, a rate R = 2/3 convolutional code can be combined with 8-PSK modulation by mapping the three encoder output bits in each T-second time interval into one 8-PSK symbol. This TCM scheme can then be compared with uncoded QPSK modulation, since

or 1 hit/din1ension

2S

thr; tb.e

TCMl was introduced in a paper and the essential elements of the idea \Vere later pal)ers [L 3. 4]. Other contributions to of TCM include those i\/J.assey [5]. Anderson ancl and [7]., Calderbank 1:•ncl l'/Iazo

[10. U]. has also been devoted channels. As in the case of

achieve lhe best

Divsal::ii, 8lso c::m be included in bandwidth-efficient versions of con e.g .. [16, 17]) and turbo e.g., [18, For the various aspects of TCM u-, more detail than 1s contains a view of the up to 1990. O'Jeniev1 of the state of the art in 1998.

are drawn from consteHation in either one- or two-dimensional Euclidean space, constellations appear in Figure 18.1. Some modulation (Ah;j[), constellations are shown in 2--AM, is to BPSK Figure 18.l(b) illustrates several

fa our treatment of TCM we assume that the transmitted

1 Throughoul this chapter we denote spectral efficiency in units of bits/symbol, where one signal (symbol) is irnnsmittecl in each T-second time interval. With this notation, the required bandwidth is proportional to I/ T, and higher spectral efficiencies are thus more bandwidth efficient.

9S4

Chapter 18

Trel!is·,Coded ~v1odulation

" " "' ~

0

Q

Q

0

0

0


el

" "' .+ " " """ " " """

a)

a)

[/) [/)

·E

0 r:,::

~

+

u

CN

N CC)

0

©

(l)

0

0

Q

0

0

0

0

"'

0

Q

©

0

0

@

0

0

0

0

0

G

0

0

" """ " "' "' " " "' 0

(J)

w

©

G

@

N

0

Q

G

:§

0

" "v " "" " " """ " " "., " """" " 0

G

'-0

.g

©

0

0 0 0

~

[/)

0

p..

uo

vuo

+ uo

"

"

"

"

"'

-aC

0

~

@

®

®

~

" -aC ·3f 0


N

t

~


.+

~


co

"':g C

.s a)

E

C

Cl

VO

~

oD

~

C

@

(j)


0 '-0

@

0

:aa', 0

H

-;

.~ ~

'-0 ,--<

®

©

@

®

@

(F,)

" 0

Cl

u

[/)

cil

co

C

-~ C

a)

0

@l)

@

®

Ci)

" :aE

0

G

0

()


0

3

~

a) h

p..

"' C

-~ ~

Cl'l

p..

~

~

6

~

e [/) [/)

0r:,:: --(J,;r-~~1--~--;;,~

u

co

FIGURE 18.1:

Typical signal constellations.

two-dimensional signal constellations thc1t exhibit a combination of amplitude modulation and phase modulation (AM/PM). Rectangular constellations with M = (2P)2 = 4P signal points, p = 1, 2, · · ·, are also referred to as quadrature amplitude modulation (QAM) signal sets, since they can be generated by separately applying amplitude modulation to two quadrature carriers (a sine wave and a cosine wave)

Section 1

Introduction to Trellis-Coded Modulation

95:5

and then the two modulated All one-dimensional AM and two-dimensional AM/Pl\![ constellations can be viewed as subsets of a an infinite array of translated to its minimum average energy subset of the one-dimensional and lattice '1!}, whose tvva dimensions. some two-din1ensional sets are shown in 18.l(c). MPSK and thus modulation. The 0uu,Jn.. ," denoted lS Because TCM schemes use rather than additional transmitto accommodate the redundant bits introduced must be made with uncoded modulation sets but have the same 0 n~~-,CT~ the same nurn.ber of inforrnathat tion bits per transmitted have the same average energy per transmitted .As an we can the average energy of the three sets shown in Figure where we have assumed that the minimum Euclidean distance between is dmin = 2, as follows: y~,crnc;-,~,,

1(+1)2J

= 2E.,

=2 =

2

.

=l

f(+1)2 + (+3)2] 4

(2-AM)

=5

(18.la) (4-AlVI)

2[(+1) 2 + (+3) 2 + (+5)2 + (+7)2] 8

· - - - - - - = 21.

(8-AM)

in a TCM scheme with one information bit and one redundant bit that uses 4-AM modulation with an uncoded scheme 2-AM YNe must reduce the energy of each point in the coded a factor of 5. or almost 7 to maintain the same average energy per transmitted that we must scale the amplitude of each the factor Ha TCM scheme with one information bit and two redundant bits using 8-AM is with uncoded the energy in the coded system must be reduced by a factor or more than 13 dB. This reduced signal energy results in a reduced minimum distance between signal that must be overcome coding for TCM to achieve a coding with an uncoded with the same average energy. To minimize the reduction in signal energy of coded practical TCM schemes that is, rate R = k / (k + 1) codes. Thus, TCM codes with just one redundant involves the use of binary convolutional codes. In Table 18.1 we list the average energies of each signal set shown in Figure 18.1, where the minimum distance between signal points dmin = 2, and each constellation is in its minimum average energy configuration. The energy requirements of one signal set compared

956

Chapter 18

Trellis-Coded Modulation TABLE 18.1: Average energies for the signal sets in FignrP 1 I( 1.

(a) Oime d!imem,fonaB §ign!llU §d§ 0

§ignmB §<et

Es

(dB)

2-AM 4-AM 8-AM

1 5 21

0.0 7.0 13.2

(lb>) Two·idlimel!ll§ROIIl!.all ired!lll1llgru11air siigllllaR sets §igllll!lli §1£':ti

4-QAM 8-CROSS 16-QAM 32-CROSS 64-QAM

Es

(dl!R)

2 5.5 10 20 42

3.0 7.4 10.0 13.0 16.2

Q<e) JP'§]]{ §Dgl1ll!llH set§

§igllll!llll sel1

4-PSK 8-PSK 16-PSK

Es

(dlR)

2 6.8 26.3

3.0 8.3 14.2

with another can be determined by simply taking the difference (in decibels) of the values listed in Table 18.1. For example, if uncoded 8-PSK is compared with coded 16-PSK with one redundant bit. we say that the constellation expansion factor Ye of the coded system relative to the uncoded system is Ye = 14.2 dB - 8.3 dB = 5.9 dB. Now, consider the transmission of a signal sequence (coded or uncoded) from an M-ary signal set S = { so. s1, · · · , s M ~l} over an A WGN channel. Let y(D) =Yo+ y1D + yzD 2 + · · · be the transmitted sequence, where yz ES for all/, and let r(D) = ro + r1D +r2D 2 + .. · = y(D) +im(D) be the received sequence, where ll1l(D) = no+ n 1 D + n2D 2 + · · · is the noise sequence, nz is an independent Gaussian noise sample with zero mean and variance No/2 per dimension for all l, and rr and 111 belong to either one- or two-dimensional Euclidean space, depending on whether S is one- or two-dimensional. We can also represent the transmitted, noise, and received sequences by the vectors y = (yo, Yl, yz, · · · ), ll1l = (no, n1, n2, · · · ), and ir = Cro, r1, r2, ···),and for two-dimensional signal sets, we denote transmitted, noise, and received signal points by yz = (Yi!. Yi!), nz = (nil. /1il ), and rz = (rii. rjl ), respectiveiy. Throughout the chapter we assume that Jr is unquantized; that is, soft demodulator outputs are available at the receiver. To compute the symbol-error probability P., of an uncoded system, we can consider the transmission of only a single symbol. For example, for the QPSK signal set shown in Figure 18.l(c), if each signal point has energy E.,, that is, its distance from the origin is JF,";, we can approximate its symbol error probability on an A WGN channel with one-sided noise power spectral density No with the familiar

!ntroduct:ion

Trellis-Coded

union upper bound as follovvs: / <

\\ !1

I

+ \£ n

1·\·

\

\

\1

"

;.I

is the n1ini:11un1

distance bet,Neen

maximum-likelihood two-dimensional the distance in the twodimensional Euclidean coded sequence :/, the (y) becomes

\

<

i

) ·;:vhere

[[Yi -

vf [ 2 =

+

Euclidean distance between the coded sequences y and y'. Now. Euclidean distance between :I as the minimum coded sequence i, and as the number of nearest we ~"-'""'"'-'·"·~ the bound on (:y) as <

The for event-error conditioned on the transmission of a

m (18.3) and (18.5) are in general, TCM

958

Chapter 18

Trellis-Coded rv1odu!atlon

systems are nonlinear: however. most known schemes have many of the symmetry properties of linear codes. Typically, dJ;ee is independent of the transmitted sequence. but Ac11,,e can vary depending on the transmitted sequence. The error analysis of TCM schemes is investigated more thoroughly in Section 18.3. Because the exponential behavior of (18.2) and (18.5) depends on the MSE distances of the uncoded and coded systems. respectively, the asymptotic coding gain y of coded TCM relative to uncoded modulation can be formulated as follows: y

=

d(i-ee!cuded/ Ecoded ~?~·~~--~~~~

(18.6)

d,~·,in/1111coded/ E,111cuded

where Ecoded and E,mcoded are the average energies of the coded and uncoded signal sets. respectively. We can rewrite ( 18.6) as ( 18.7)

)I=

where Vi is the constellation expansion factor, and Jld is the distance gain factor. We proceed by letting cl,,,;,, and f:.. 111 ;11 represent the minimum distance between signal points in the uncoded and coded respectively, and by assuming that the minimum distance between points in the coded (expanded) constellation is reduced, so that the average energies of the coded and uncoded signal sets are equal; that is, f:.. 111 ;11 < d111 i 11 , and JI, = l. Then, the TCM system must have a free distance between coded sequences that is greater than the minimum distance between signal points in the uncoded system to achieve coding gain. In other words, even though f:.. 111 ;11 < d 111 ;11 • a TC!Vi system must achieve dtree > d,,,;,,. In the design of codes for binary modulation, the MFSE distance between two signal sequences y and y' is given by (see Problem 18.1) clj;-ee

= 4E,c!H.fiec,

(binary modulation)

where dH.f,ee is the minimum free Hamming distance of the convolutional code. Thus, for binary modulation, the best system design is achieved choosing the code that maximizes cl fl.free. We will shortly see that, in gene ml. this is not true for TCM system design. In the TCM case, consider a rate R = k/(k + I) convolutional code with minirnum free Hamming distance c/ 111 ,cc in which we denote the k + 1 encoder 1 · at any time · h output ,tl1ts umt· I· 1JY t l1e vector v1 = (v1U '. u1(k-lJ • · · · , v1(OJ ). (crl i 1roug out the remainder of this chapter, when it is not necessary to specifically denote the time unit I of a vector. the subscript I will be deleted; that is. vectors such as v1 will be denoied simply by v.) Then, assume that the i,·i 1 binary vectors v are mapped into elements s; of an M-ary signal set S using a one-to-one mapping fimction f (v) -+ s;, where M = 21c 1- 1 • We see from (18.3) that the performance of a TCM system depends on the squared Euclidean (SE) distances between signal sequences. Thus. the set of SE distances between all possible pairs of signal points must be determined. Denoting the binary labels of two signal points by the vectors v and v', we define the error

Introduction to Trellis-Coded Moduiadon

modulo-2 sun-1. H foHows that WH(e) = dH the (abe)is. For each error vec1:or e, sue:h that ,/ = Y E8 e. and thus M SJE d]str;nces set of h1troduce defir;ed as fcHmNs:

constellation 5 and the the MFSE dis'wnce d[icc

funct:i.010

the minimum Euclidean

o2 ( "")

Oc

A

==

·,;

.h.

-;;_;;1here 6

0

=lLTI!l1 V

be used and the IVJ:FSE distance 'We l3hel each brcinch of a convention8; convolutional code with vector :1 bits. vve can label e21d: reference code,vorcL tlrnt !bis error trellis is j,dentical to the con error trellis

error vector e, where::,,s the the minimum distance to a ie, are monon1i8tls. 813 in the case of convolutional codes. To when the function method can he to TCM we now i1ntroduce the notion of a into two Q (0) and Q (1 ), labeled a vector w with v(OJ = 0, nnd to labels with v(OJ = L Next, we let be the /-\EVlE for the :;ubset Q (0), and be the AEWE foI the subset 1 02 02 0 ) c. t 1'11.~ \Tr,~\;V-Jr' ,: uc.O ~,-,rl V) , ca1·e r\pfi11ecl 1'n can 'v'V'''Y' Q_; r,.l ~i[, '1'",1P" "<,Jl~J'~ '·, "''• ~,,, L, u,,.O ( "· _ ., ~ _.. ._, 0 ]

0

0

%0

Chapter 18

Trellis-Coded Modulation

DEnNnrmN ll8Jl A one-to-one mapping function .f (w) ---+ Si from a rate R = k/(k + 1) convo!utional encoder output vector w = (v("J. v
to a signal points; belonging to a 2k+ 1-ary signal set Sis uniform if and only if D-~. 0 (X) = D-~.l (X) for all error vectors e.

IEXAMPllE UU

Uniform Mapping

Consider the three 8-PSK signal sets shown in Figure 18.2 along with their associated labels. Using the signal point labeled by w = (000) as a reference and assuming unit energy signals, we see that there are four distinct Euclidean distances between 8-PSK signal points: a

2

= [1;v2J2

+ [1 -

( 1/h) J2 = 1/2 + (3/2 - h) = 0.586,

b2 =1 2 +1 2 =2. 2

c = [ 1;hJ2

(18.lOa) (18.10b)

+ [ 1 + (1/h) J2

= 1/2 + (3/2 + h)

= 3.414.

d 2 = 22 = 4.

(18.lOc) (18.10d)

We now examine the eight possible SE distances corresponding to the error vector e = (001) for the labeling of Figure 18.2(a). We see that there are a total of four code vectors w for which ~~(e) = ll.fM--.f(wEBe)ll 2 = 2: the vectors v = (000), (001), (110), and (111), and four code vectors w for which D-~(e) = 3.414. that is, the vectors v = (010), (011), (100), and (101). Thus, the AEWE for the error vector e = (001) is D-~(X) = (l/2)X 2 + (l/2)X 3 .4 14 . If we now partition the 8-PSK signal set into two subsets Q (0) and Q (1), depending on the value of the bit vCO) in the label vector, we see that for the error vector e = (001), each subset contains exactly two signal points for which ~;(e) = 2, and two signal points for which ~;(ie) = 3.414; that is, ~~. 0 (X) = D-; 1 (X) = (!)X 2 + C!)X 3414 . Repeating this calculation for each possible error vector ie gives us the AEWEs listed, along with the corresponding MEWEs, in Table 18.2(a) for each subset Q (0) and Q (1). Because ~~. 0 (X) = ~~,l (X) for all ie, the mapping is uniform. 001 100, ,,--,-- . 010 ,~-------b ,. , c~ O l l ~ d ~ O O, O llO ®~, __

100

001 ,,--~----. 010 ~ (Q

'

eooo

1100

@'

~'

100@

~000

'

' IE{

l';il\

111

010 ,---®---- . 001

011

Ole'--, -0 _ , , , = 111

fi

th,

101

101

111 '---®---- 101 110

(a)

(b)

(c)

'-o--

FIGURE 18.2: Three 8-PSK signal sets with different labelings.

TABLE 18.2: The AEWEs cind MEVJEs

ie

to the three 8-PSK

18.2.

r;.::,v'

xo

vO

000

.A

x2 + l2Ly3.414

001

l 2"

010

1xoss6 2 lxOS86 2 1 xJ.-+P 2

+ 12 x4

101

ixoss6

+ !x2

110

!x2

111

1xo.ss6 2

011 100

+ 2_(!_ 1 v3.414

2l

+ 1x2 2 +}

[ v0.586

[

2A

-, 2

1 v0.586

+ 2l

2"''

-

l

2

2-

!xo.ss6

2 .I

L

2''

1x-1 2 1

2L

v0.586

xo.ss6

xoss6

--"

V""'I

ro

Fl

.-<-

I

i5" :::;

~,

v0.586 .IL

~-0

l '(3.414 + 2' 1xoss6 + lx3AJ4 2 2 Ix· 2

+l2

1 y3At4

2

I

+

2 ' -, 2 t v0.586 + l '{2

+ 2"'' 1 v2

I y0.586

2L

I v3.4l4

,

1xoss6 2 l )73A14

1 ·111 I 4 x---.-. + 2X

+ !x3AI-+ _L I

L

!

T 2L'i 2 l ;(0.586 _L l

2'

+

v0.586

.d1..

:i

F:~

0n. C n

C

ie

~·. 0

001

lx2 -f -

+

010 011

lry0.586 4 \..(

+

110 111

+

1

l v3.414 l 2A - 4 xo.ss6 v2

L'l..

+

v3.414

A

"'

[ x0.586

2"

1x2 2 lxOS86 + l 4 2

I

1

+

4

1

, 1 v3.414

2./':;,

21 v0.586 .I)\.

+

l.:...~ ( ./l v0.586

T

I

]

2

E5'

+ 1.11. l v4

+ 2,(-:,._ l v3A14 yJ.J,14

_L I

l x2

v3.414

100 101

:::i

XO

000

_L

./:;.

l

~:

::r

iil I

+

l(xoss6 4

l

1 v0.586 4..11._

_L I

+ 2 x2 l

L_ v3.4]4 - i 2_,lj;__

+ ±xH14

ro

±

Q.

:s;:

+

0

Q_

+

1 v0.586

4"

0

Q_

+

!xoss6 4\_!l

A

+

+

1+

2 I 2 1 1 xo.ss6

2 l v0.586 2.11..

.;;;·

+::r

+

I

1

v0.586

A

xoss6

2 uruuiuea

C Ill

.-:·

c· :J

,© ©it d

!J:)

(i:!'I I\.! ()

::r Ill u,-,. (I)

·-s ()0

TABLE 18.2:

(continued)

...--l

(l)

{1:)

e

.&;o(X)

000

XO

001

1 xo.ss6 4 Jxo.ss6 4 1xo.ss6 4

XO

xo

XO

XO

XO

Q_

xo.ss6 x2

xo.ss6 x2

xo.sss

() Q_

x2

rur+

xo.ss6

xo.ss6

xo.sss

::,

1 xo.ss6 4

+ JxH14 4

1xoss6 4

+ 4Jx3AI4

+ 14 xH14 x2

Jxoss6 4

+ lx3A14 4 x2

1 xo.ss6 4

xz

110 111

&;,o(X)

x4

100 101

+ 1x3.414 4

&;

.&; 1(X)

x2

010 011

+ 14 x3.414

tn·

h ()

,&Z,(X) ,C\

+ 14 x3.414 x4

1 xo.ss6 4

+ 14 x3414

1 xo.ss6 4

+ 14x3.414 x2

1xoss6 + lxH14 4 4 1x2 + lx4 2 2 1xo.ss6 + JxJ.414 4 4 1x2 + lx4 2 2 1.xo.ss6 + JxJ.414 4 4

l

(X)

&;;(X)

Q_ (I)

x4

xz

xz

xOS86

xoss6

xo.sss

xz

x4

xz

xo.ss6

xo.ss6

xo.sss

< .::,, C

o·

in the t\V() the 8-PSTK. and for Q (0) ,md Q (1),

The

rernarks relate

~~o

UU:

and

For the unifor1n all error vectors e. This is true o

In the subsets one subset can be obtained fron1 the

the set of distance0 betvveen c For a the subsets a sufficie:nt condition for

i0 nec:Gssctry that there be an the existence of the case m

a

between the subsets Q (0) and Q (1). cannot be

·vve nmv let code

be any twc sequences in the

where -1~ ...

(18.11b)

and

+-··. is a nonzero path the error trellis of that is. w(D) and differ in at most L + 1 branches. The term then represents an error event L + 1. H and are the two channel. sequences corresponding to and that = f (7o) + .f (71)D + f (v2)D 2 + · · · and y'(D) = j(,::ro) + f(Y~)D -I- /(Y~)D 2 +···,then the SE distance between and y' (D) is given by

95~.

Chapter 18

Trellis-Coded Modulation

d}[y(D), y'(D)] =

L di [f (vi), f (v;)J

(/,:5/:5/,+T.,)

"' L

II

f(v1) - .f(\v/ EB ie1)

11

2

(h:5/:5h+L!

(18.12)

L

L'-.~/ ie1)

L

L'-.2(ie1)

(h:5/:5h+L)

>

g L'-.2[e(D)].

(h:5/:5h+L)

where the inequality follows from the definition of Euclidean weight given in ( 18.9c ), and [e(D)] is called the Euclidean weight of the error sequence e(D). We now prove a lemma that establishes the conditions under which the Euclidean weights can be used to compute the MFSE distance d{iee of a TCM system. lLJEMMA UtJl (lR'.ATJE R = + Jl) ICOJIJ>JE LJEMMA) [1] Assume the 1m:pping from the output vector v of a rate R = k/(k + 1) binary convolutional encoder to the elements of a i'+ 1 -ary signal set S is uniform. Then, for each binary error sequence e(D) in the error trellis. there exists a pair of signal sequences y(D) and y1 W) such that (18.12) is satisfied with equality.

Pmo_f. From the definition of Euclidean weight, L'-. 2 (e1) = min.'I/--( VI,1" . ___ .l;,, 1, ,V;ro,

1

L'l.;1 ( ie1) for all time units I. Because the mapping is uniform, minimizing over

the k-bit vector [vJl0 . · .. , v; I)] yields the same result in the subset Q (0) with 01 = 0 as in the subset Q (1) with 0l = l; that is, the Euclidean weight is 0 independent of the value of v; ), and L'-. 2 ( e1 ) = minr,,;1,, _ _v; 1, L'l.~1 ( e1 ). Further, 1 a rate R = k / (k + 1) encoder can produce any sequence of k-bit vectors l l] ( only one bit is constrained); that is, every such sequence of k-bit vectors corresponds to a path through the trellis. Thus, for each binary error sequence e(D), there exists an encoder output sequence v( D) such that (18.12) is satisfied with equality. CQJJEJI)),

v;

v;

[v?'l. ··· ,v;

Lemma 18.1 implies that the MFSE distance d!i·ec between signal sequences can be computed by replacing the binary labels on the error trellis with the MEWEs and finding the minimum-weight path through the trellis: that is,_

l

12 = mme([))/0([)) cjicc fl 2[ e(D)_.

(18,

A similar argument can be used to show that the average weight enumerating function Am,(X) can be computed by labeling the error trellis with the AEWEs and finding lhe transfer function of the modified state diagram (see Problem 18.4). H the mapping is nonuniform, the rate R = k / (k + 1) code lemma does not hold, and the computation of A(lv(X) and ctJ,.ee becomes much more complex. An example illustrating this point is given later in this section, The technique for using the AEWEs to compute A(lv(X) will be presented in Section 18.3. In the remainder of this section we present a series of examples illustrating the basic principles of designing a TCM system to maximize dT,ee·

Introduction to Trellis-Coded Modulation

convoiluti,onal code with minimum free

assumed to have unit energy., so fhe IVISE distan(:e we see that the SE distance between all four the error vector e = ) is to 2, and thus , the MEWE is of0 ll (X) = Table 18.3 we WH

yillylli)

01

00

___ --11 __ --0]

,C)

---

., ,

I

.

,-,~---··----· --~---·-·---~-··

(a) Gray n1apping:

(b) Natural mapping

FIGURE 18.3: Two

set

rules for the

T,,0J:ll.E rn.3: Euclidean distance structure for

00 01 10 11

0 1 1 2

XO

vO

A

and

xo '

x4 x2

966

Chapter 18

Trellis-Coded Modulation

From Table 18.3 we see that all the AEWEs are monomials and that .6.~(X)

=

8~(X) for all e. Such mappings are called regular mappings, which implies that they are also uniform mappings. Further, in the case of Gray-mapped QPSK, the two

error vectors for which WH(e) = 1 result in 6.*(e) = 2 for all v, and the error vector for which WH(e) = 2 results in 6.*(e) = 4 for all v; that is, .6.*(e) = 2wH(e) for all e and v. In other words, there is a linear relationship between SE distance and Hamming distance. Thus, the rate R = 1/2 convolutional codes with the best minimum free Hamming distance dHJree will also have the best l\1FSE distance d},.ee when combined with Gray-mapped QPSK. For example, the optimum free distance (2, 1, 2) code with dH,Jree = 5, when used with Gray-mapped QPSK, results in an MFSE distance of cl},.ee = 10. Compared with uncoded BPSK with unit energy signals and d1~ 1in = 4, this (2, 1, 2) code results in an asymptotic coding gain of y = lOlog 10 (dJ,.eeld;,; 11 ) = 10log10 (10/4) = 3.98 dB, exactly the same as when this code is used with BPSK modulation. Thus, designing optimum TCM schemes for Gray-mapped QPSK is identical to finding optimum binary convolutional codes for BPSK modulation. For naturally mapped QPSK, however, the situation is different. The two error vectors for which WH(e) = 1 give, for all v, 6.*(e) = 2 in one case and .6.;(e) = 4 in the other case, and the error vector for which WH(e) = 2 gives ,6,~(e) = 2 for all v. In other words, there is no linear relationship between SE distance and Hamming distance when natural mapping is used. Thus, traditional code design techniques will not give the best codes for use with naturally mapped QPSK. Continuing with the naturally mapped case, let us now consider two different (2, 1. 2) code designs: Code 1 : G1 (D)

= [ 1 + D2 1 + D + D2

(18.14a)

Code 2: G2(D)

= [ 1 + D2

(18.14b)

D ].

Code 1 is the optimum free distance (2, 1, 2) code with dH.Jree = 5, whereas code 2 is suboptimum and has dH.free = 3. The encoder diagrams for these two codes are shown in Figure 18.4(a), and their error trellises with binary labels are shown in Figure 18.4(b). Now, replacing the binary labels with the MEWEs of naturally mapped QPSK from Table 18.3(b ), we obtain the modified error trellises of Figure 18.4(c). Examining the modified error trellises for the minimum-weight error events, we see that d},ee = 6 for code 1, resulting in a coding gain of y = 1.76 dB compared with uncoded BPSK, whereas code 2 achieves d}iee = 10 and y = 3.98 dB. Thus code 2, clearly inferior to code 1 for binary modulation or for Gray-mapped QPSK, is the better choice for naturally mapped QPSK. The following comments apply to Example 18.2: e

The linear relationship between Hamming distance and Euclidean distance in Gray-mapped QPSK is unique among nonbinary signal sets. 1n all other cases, no such linear relationship exists, and the best TCM schemes must be determined by jointly designing the code and the signal set mapping.

15· ci -

,'i

::J C ~

0

0

,-0

m

~

o~ co

~ :i':>

tn

:J 0

(Naturally mapped QPSK)

0 0..

(D

s""' "'p:, p:,

::J

Q.. (D

(Naturally mapped 4-AM)

{;pDl

""'

0.

p.;· (Jc,

(Gray-mapped 4-AM)

I

~

J;Ur>DJ--Jy

V, (i)

"o· .-,.

>

::i

l

~

co v({))(D) ---~>

Code 1

""'""'

::; r>

0,..,

-~""'

0 Q_

y

{fl

(1) Cf)

1-h

0 ,..,

:;f]

0

r°'

N

C

,>~)

~>I

,>Dr ----

" o·

>

,-1-

::;

8" ·-I

iiJ en·

vl"\D)

>

Cocle2

n 0

(a) Encoder diagrams

o_

f-'

(j)

Q_

t-,)

__,,,

(b) Error trellises

(c) Modified error trellises

(cl) Modified error trellises

(e) Modified error trellises

~

0

Q.

C

w

0

0 =1

~0 :J

<:

0

~

'

® g; ~

Chapter ·1 s

968 0

®

®

0

0

Trellis-Coded Modulation

The nonlinearity of most TCM systems arises from the mapping function f (-), which does not preserve a linear relationship between Hamming distance and Euclidean distance. Both signal mappings in Example 18.2 are regular; that is, each error vector e has a unique SE distance associated with it, and the AEWEs are equal to the MEWEs for all e. For regular mappings, the Euclidean weight enumerating function A (X) of the code is independent of the transmitted sequence. Thus, A(X) and djree can be computed in the same way as for linear convolutional codes with binary signal sets, that is, by assuming that the code sequence corresponding to the all-zero information sequence is transmitted. The critical step in the design of code 2 for naturally mapped QPSK was to assign the error vector e = (10) with maximum Euclidean weight to the two branches in the trellis that diverge from and remerge with the all-zero state So. This assignment guarantees the best possible Euclidean distance in the first and last branches of an error event and is one of the key rules of good TCM system design. Each of the coding gains quoted in this example came at the expense of bandwidth expansion, since the coded systems have a spectral efficiency of 17 = 1 bit/symbol = 1/2 bit/dimension, and the spectral efficiency of uncoded BPSK is 17 = 1 bit/dimension. Most of the comparisons with uncoded systems in the remainder of this chapter will involve TCM schemes that do not require bandwidth expansion; that is, they are bandwidth efficient. The design of good rate R = 1/2 codes for use with naturally mapped QPSK will be considered again in Section 18.4, when we take up the issue of rotationally invariant code designs.

'" The QPSK signal set is equivalent to two independent uses of BPSK, denoted by 2 x BPSK. This can be considered a simple form of multidimensional signaling, a subject that will be covered in Section 18.5. IEXAMPllE 1r!L3

Raite R

= 1/2 Trrems-Codled 4-AM

In this example we consider the same two rate R = 1/2 convolutional codes as in Example 18.2, but this time with the encoder output vector v = (v< 1lv<0>) mapped into the one-dimensional 4-AM signal set. Both Gray mapping and natural mapping of the 4-AM signal set are illustrated in Figure 18.5, where the signal amplitudes are assigned in such a way that the average signal energy Es = 1. Using the signal point labeled v = (00) as a reference, we see that there are three distinct SE distances between the 4-AM signal points: a

2 2

=

[(-1/-Js) - (-3/-J5)] 2

=

[21-Js]2

(18.15b)

c2 = [(3/-J5) - (-3/-J5)] 2 = [6/-J5]2 = 7.2.

(18.15c)

=

[4/-J5]2

(18.15a)

= 0.8,

3.2,

b =

[(1/-J5) - (-3/-J5)]

2

=

Clearly, the MSE distance between signal points in this case is 6-~,in

= 0.8.

Section 1

Introduction


C

Trellis-Coded Modulation

!lll69J

----------c;,-

- <--{I--,-

-3/VS

+ 1/\/5

-11vs ( a)

mapping

ylliylil)

01 11 00 10 c-------,~-~~::--------o - 3;vs - 11\ls + 11vs (b) Natural mapping FIGURE 18.5:

nntural

of the 4-AM

TABLE 18A: Euclidem1 distnnce structure for

set.

and

In Table 18.4 we hst for both error vectors e and the four

of 4-AM, the four (X) and MEWEs (X) in both cases, and thus '-''-'~,_,,,-,u,,,o are uniform. VJ e note that in each case, , there is exactly one error vector 112; for which is not a and thus the mappings are not regular. If we now labels on the error trellises shown in Figure 18.4(b) with the MEWEs of 4-AM from Table 18.4, ·we obtain 18.4( d) and 18.4(e ), Examining the modified error trellises for the error events, we see that for Gray-mapped 4-AM (Figure df,-ee = 7.2 for code 1, resulting in a coding gain of y

=

l01og 10 (cl}i.eeld1~ 1 ; 11 )

= 2.55 dB

with uncoded 2-AM with unit

energy signals and c1;1; 11 = 4, whereas code 2 achieves d?,.ee resulting in a coding loss of y = -2.22 dR code 1 is the better choice for Gray-mapped 4-AM. For 4-AM with natural (Figure 18.4( e )), the situation is reversed, and the best choice is code 2, which results in a coding of y = 2.55 dB vvith uncoded 2-AJVI.

Trellis-Coded Modulation

Chapter 18

910

The following observations relate to Example 18.3: " fo both cases, the mappings are nonregular; that is, for some error vectors e, the MEWE does not equal the AEWE. This implies that the weight enumerating function A(X) of the TCM system changes depending on the transmitted sequence; however, since L'.1~ 0 (X) = L'.1~.l (X) for all e in both cases, the mappings are uniform, and the MFSE distance d!i-ee can be computed by replacing the labels e in the binary error trellis with their corresponding MEWEs 8~(X) and using the transfer function method. e By definition, all regular mappings must be uniform, but the reverse

is not

true. o

0

As in Example 18.2, the critical step in designing the best codes for both mappings was to assign the error vector with maximum Euclidean weight to the branches in the trellis that diverge from and remerge with the state So. In Example 18.3, unlike in Example 18.2, coding gain is achieved without bandwidth expansion, since the coded signal set, 4-AM, has the same dimensionality as the uncoded signal set, 2-AM. This explains the somewhat smaller coding gain, 2.55 dB versus 3.98 dB, achieved in Example 18.3 compared with Example 18.2.

IEXAMIPllE 18.4

Now. consider a rate R = 2/3 convolutional code with 8-PSK modulation in which we denote the three encoder output bits by the vector v = (v(2)v0lv< 0 J). In Figure 18.6 these three bits are shown mapped into the 8-PSK signal set according to the natural mapping rule. Each signal is again assumed to have unit energy. but in this case the MSE distance between signal points, computed in (18.lOa), is ti~,;11 = 0.586. Thus, compared with the QPSK signal set with the same average energy, the MSE distance of 8-PSK is reduced from 2.0 to 0.586. In Table 18.5 we list the eight possible error vectors e and the eight corresponding AEWEs, L'\~. 0 (X), L'.1~,l (X). and L'\~(X), and MEWEs. 8~. 0 (X), 8~.l (X), and TABLE C

18.5: Euclidean distance structure for naturally mapped 8-PSK. A~_l(X)

A~o(X)

~~--~----··

000

XO

XO

XO

001

xo.5sc,

x11 ..,s6

xo'>xc,

x2

010 011

4xo5sc,

x2

x2

+ 4xu1cJ -

4x115sc, -

f"IXI

+ ~x.1A1cJ -

~x115sc,

+ ~x,1 ..i1cJ

li~ 1un

of(X)

xo

xii

XO

xO:iS6

xo.5s6

xo ..,sc,

x2

x2

x2

x(J.586

xo5s6

x11.,s6

100

x-i

x4

xcJ

x.i

x-'

xcJ

IOI

x,1"114

x.1-11"1

xUlcJ

X·' .JlcJ

x3Al4

x·'.41"1

110

x2

x2

x2

111

+ ~x3.-+i-+

~xo.)1'16

~XiJ'i,% -t- ~ X 3 .Ji.J

~Xli'iXb

+ ~x,\414

x2

x2

x2

x11.osb

xU.)86

xo.)s6

I

Introduction to Trellis-Coded IVloolulation

sin

010

--0-',,

Oll

001

u"

\ \

I

\

I

100

\

I I

I

I

¢ 000

\ yl'lyi l iyiOI

c: I I

I

I

\

I \ \

101

111

110

I'Tatural

set

rule fot the 8-PSK

Problern 1K8 it is shown that uniform 8-JPSK shown in Figure two error vectors that result in different Euclidean distances; that is, for natural the error vectors ,e = (011) and e = (111) result = 0.586 or 3A14. whereas the other six error vectors to a ")!/e novv consicleT several and evaluate their lV[FSE dist,mces

for 8-PSTK the error trellis labeled with MEV'/Es. H1

=[

+D+l)

+ 1l / ( D 2 +

D + 1)

1 ].

2. l) code with constraint v = 2 and shown in 18.7(a). the 2" = ~-state and the modified error trellis labeled

This is the

with ·which to compare is QPSK ·with an average energy mapped TCM suffers a = -0.56 dB in this case! ·without bandwidth 8-JPSK? Because the better TCIVI scheme

a rate /R == coinbined

code with one uncoded information bit as that is. both have a efficiency of 17 = 2 8··PSK modulation. To illustrate this latter

loss of

972

Chapter 18

Trellis-Coded Modulation

(a) Encoder diagram

(b) Error trellis

(c) Modified error trellis

FIGURE 18.7: Encoder diagram and error trellises for rate R

= 2/3 coded 8-PSK.

the same two (2, 1, 2) codes as in Example 18.2, although this time we include an uncoded information bit and use the systematic feedback form of the encoders. Thus, the two rate R = 1/2 generator matrices are given by Codel:G1(D)=[ 1

(D 2 +D+l)/(D 2 +1) ],

(18.17a)

Code2:G2(D)=[ 1

D/(D 2 +1) ].

(18.17b)

Section 18.1

lrrtroduction to Trellis-Coded Modulation

97:!

----~~---r l t ), (~--D1-->C~)-->D~>U

~ v111(D)

'{

L __ -~

L__CJ

L--·----~~-----··

L

-->

ylO}(D)

(a) Encoder diagram

(b) Error trellis

(c) Modified error trellis

FIGURE 18.8:

Encoder diagram and error trellises for rate R

1/2 coded 8-PSK

(code 1). The encoder diagrams for these two codes are shown in Figures 18.8(a) and 18.9(a), their binary error trellises are given in Figures 18.8(b) and 18.9(b ), and the modified error trellises labeled with MEWEs for mapped 8-PSK are shown in Figures 18.8( c) and 18.9(c ). respectively. The uncoded information bit is handled by adding a parallel transition to each branch in the binary trellis of the rate R = 1/2

974

Chapter 18 u(

2

1(D)

Trellis-Coded Modulation

-------------------------<>- vm(D)

(a) Encoder diagram

(b) Error trellis

(c) Modified error trellis

FIGURE 18.9: Encoder diagram and error trellises for rate R

1/2 coded 8-PSK

(code 2). code. Thus, there are two branches connecting each pair of states in the binary error trellis, one for each of the two possible values of the uncoded bit. We follow the convention that the first bit listed on each branch of the binary error trellis is the uncoded bit. In the modified error trellis, we show only one br<1nch conrn~cting Pach pair of states; that is, it has the same structure as the trellis for the rate R = 1/2 code,

Section 18.1

!ntroduction to Trellis-Coded Modulation

'.9l75i

but its label is the i:he two error trellis are labe),ed ,-,~·-·-"'"',.., MEWEs are and

For any TCM scheme ·with the calculation of the MFSE involves two terms: (1) the MFSE distance o}.ee between distinct trellis than one branch and the MSE distance o;,;in between distinct trellis one branch in Because o!i·ee is the free distance between trellis associated with the coded bits. it can be from the error trellis labeled ·with the MEWEs. Because . on the other hand. is the minimum distance between transitions with the uncoded it must be the overall MFSE distance is 'n{ 002free·· == -· u.H

002 ,nin

} ·

transition distance o;,,i11 is used. From 18.8(b) and it is clear that the branch lcibels differ the error vector (100). from Table 18.5 we conclude that o~,in == 4.0. Now. we can see from 18.8(c) and 18.9(c) that = 1.758 for code 1. and 81;-ee =

The

02 02 . ) = nun, · . ! ofree. . {1 .7S8 om 11, - n11n _ .

= 1.758.

1)

and --

02 02 ) n11·1··{ 1 .,. 0 ji·er!' 0 min -

4.0)

= 4.0.

(code 2)

and the losses with uncoded are y = -0.56 dB for code l and y = +3.01 dB for code 2. for the three different the best performance. and the coding codes considered in this is achieved the terms of dH.free) rate R = 1/2 code with one uncoded bit. This simple 4-state code achieves a 3.01-dB gain compared with uncoded QPSK without bandwidth 18.9 illustrates that oiher rules for 8-PSK result in less coding gain than natural The o

remarks relate to

18.4:

All for the 8-PSK signal set are nonregulor. the enumerating function A(X) depends on the transmitted code sequence for all 8-PSK.-based TCJVK however. if the is uniform. the average can be labeling the branches weight enumerating function Auv of the error trellis with the AEWEs and using the transfer function method.

976

Chapter 18 ®

@

e

0

@

®

Treilis-Coded Modulation

Virtually all signal sets and mappings used in practical TCM systems are nonregular. although symmetries usually exist that allow a uniform mapping. The MEWEs can be used to compute the MFSE distance d},.ee of TC:M systems with uniform mappings, as shown in Examples 18.2, 18.3, and 18.4; however, to determine the average weight enumerating function Aav(X), the AEWEs must be used, as will be illustrated in Section 18.3. The critical advantage of naturally mapped 8-PSK over other uniform mappings for 8-PSK is that the error vector for all parallel transitions, e = (100), is assigned to the largest possible EW, ,6, 2 (e) = 4.0, by the natural mapping rule (see Problem 18.9). In other words, for 8-PSK, the MSE distance between signal points on parallel transition paths is maximized by natural mapping, thus minimizing the probability of a one-branch (parallel transition) error event. An exhaustive search of all possible 8-PSK TCM schemes with 17 = 2 bits/symbol and 4 states indicates that the best scheme is code 2 in Example 18.4, that is, the suboptimum rate R = 1/2 code with one uncoded bit, combined with natural mapping. This illustrates that, unlike code designs for binary modulation, the best TCM designs often include uncoded information bits resulting in parallel transitions in the trellis. (If uncoded bits are employed in the design of codes for binary modulation, the minimum free Hamming distance can never exceed the minimum Hamming distance between the parallel transition branches, which equals 1.) All the encoders in Example 18.4 were given in systematic feedback form. Equivalent nonsystematic feedforward encoders exist that give slightly different BER performance because of the different (encoder) mapping between information bits and code bits. Systematic feedback encoders are usually preferred in TCM system design because they represent a convenient canonical form for representing minimal rate R = k/(k + 1) encoders in terms of a single parity-check equation. This canonical representation simplifies the search for the best encoders. Larger coding gains can be achieved by employing more powerful codes, that is, longer constraint lengths. Tables of the best TCM code designs for a number of important signal constellations are given in Section 18.2.

The rate R = k/(k + 1) code lemma guarantees that if the mapping is uniform, any error sequence e(D) in the binary error trellis with a given Euclidean weight ,6, 2 [e(D)] corresponds to a pair of signal sequences y(D) and y'(D) in the trellis separated by a free squared Euclidean distance of t. 2 [e(D)]. In this case, the MFSE distance dJ,.ee of a TCM system can be computed using the method of Euclidean vveights~ however, if the mapping is not uniform, the rate == k/(k + 1) code lemma does not hold, and the method of Euclidean weights will, in general, give only a lower bound on the actual dT,·ee· This point is illustrated in the following example. Jl~

EXAMPLE 18.5

No1m.m1ifo1rm Mappill'ilgs

Consider the two nonuniform mappings of 8-PSK shown in Figures 18.2(b) and (c), along with their AEWEs and ME'.:VEs listed in Tables 18.2(b) and (c). If these

Section 18.1

Introduction to Trellis-Coded l\llodulation

'9J7!7i

whose encoder are used we obtain error trellis are shown in the modified error trellises shown in and Table consider the nonuJJ_iforna (0) and Q (1). Let e(D) in '':Nhich there is no ,o·,.uL•A' + (110)D 3 be a -\eo + 121D + + 18.9(b) that starts and ends in state So. From the the e,Tor trellis of we can the EVv of as follows: modified error trellis of

= 0.586 + 0.586 + 0.5E:6 + 0.586 = 2.34A. For the rate J?_ == exist a of 4-branch trellis in the same state, that differ sequences y(D) and are distance 2.344 18.2(b) and Table 18.2(b) we see that the desired must start with the branches vo = (101) and vS = (011). since is the of labels such that :eo = vo EB vS = (110). and

(a) Mapping of Figure 18.2(b)

(b) Mapping ofFigure 18.2(c)

FIGURE 18.10: Modified error trellises for two nonuniform mappings.

978

Chapter 18

Trellis-Coded Modulation

dHf(vo). f(vo)] = d~;[yo. Yo]= 0.586. (The branches assigned to Vo and "o can also be reversed without changing the result.) Thus, from Figure 18.9(b ), the pai.t must start either from state S2 or from state S3. Similarly, the next three pairs of branch labels must be v1 = (001) and v~ = (010), v2 = (111) and = (000), and v3 = (101) and = (011) (or the reverse of these labels) to satisfy the distance conditions; but a close examination of Figure 18.9(b) reveals that no pair of paths with these labels and starting either from state S2 or from state S3 exists in the trellis. Thus, it is impossible to find a pair of paths v(D) and v' (D), starting and stopping in the same state, that differ by the error path e(D) and whose corresponding signal sequences y(D) and y'(D) are distance 2.344 apart; hence, the rate R = k/(k + 1) code lemma is not satisfied. Next, consider the nonuniform mapping of Figure 18.2(c) and Table 18.2(c), in which there is an isometry between the subsets Q (0) and Q (1). Let e(D) = eo + e1D + e2D 2 + e3D 3 + e4D 4 = (110) + (lOl)D + (100)D 2 + (101)D 3 + (110)D 4 be a path through the binary error trellis of Figure 18.9(b) that starts and ends in state So. From the modified error trellis of Figure 18.lO(b) we can compute the EW of e(D) as follows: ll 2[e(D)] = 2.0 + 0.586 + 2.0 + 0.586 + 2.0 = 7.172. (18.21)

v;

v;

For the rate R = k / (k + 1) code lemma to be satisfied, there must exist a pair of 5-branch trellis paths, v(D) and v' (D), starting and stopping in the same state, that differ by the error path e(D) and whose corresponding signal sequences y(D) and y'(D) are distance 7.172 apart. From Figure 18.2(c) and Table 18.2(c) we see that the desired path pair must start either with the branch pair vo = (000) and Vo = (110) Or with the branch pair VO = (100) and VO = (010), Since these are the only pairs of binary labels such that eo = Vo EB Vo = (110), and 4Lf(vo), f(vo)] = dHyo, Yo] = 2.0. From Figure 18.9(b) we see that the path pair must start either from state So or from state S1 in both cases. As in the previous case considered, the next four pairs of branch labels are similarly constrained to satisfy the distance conditions. It is easily seen that there is only one possible branch pair corresponding to the error vector e 1 = e3 = (101). but there are two possible branch pairs corresponding to the error vectors ie2 = (100) and e4 = (110). Again, a close examination of Figure 18.9(b) reveals that no pair of paths with these labels and starting either from state So or from state S1 exists in the trellis. Thus, it is impossible to find a pair of paths v(D) and v' (D), starting and stopping in the same state, that differ by the error path e(D) and whose corresponding signal sequences y(D) and y' (D) are distance 7 .172 apart; hence, the rate R = k / (k + 1) code lemma is again not satisfied. Example 18.5 leads to the following observations: ®

o

When the mapping is nonuniform, there are still many error sequences for which the rate R = k/(k + 1) code lemma is satisfied; however, Example 18.5 illustrates that this is not true for all error sequences. Example 18.5 shows that an isometry hetween the subsets Q (0) ::ind Q (1) is necessary, but not sufficient. to guarantee that the rate R = k/(k + 1) code

Section 1R 1

Introduction to Trellis-Coded Modulation

!ll1!ll

lemma is satisfied. See Problem 18.10 for an example illustrating this fact that uses a different constellation. o

o

Because the rate R = k/(k + 1) code lemma is not satisfied for nonuniform mappings, the method of Euclidean a lower bound on c!J,.ee in this case. This is also true of the method to be presented in Section 18.3 for determining the AWEF Aav(X) of a TCM from the AEWEs. Using a nonuniform does not an inferior TCM one that is more difficult to In this case, a supertrellis of states must be used to determine the set of distances between all '"""'""c path result in the best designs for

uniformity, was introo A more duced by this condition is the computation of weight enumerating functions is but many practical TCM systems are not geometrically uniform.

18.3, and 18.4 illustrnte two basic rules of good TCM system design: Rule 1: Signal set mapping should be so that the MSE distance between transition branches is maximized. Rule 2: The convolutional code should modified error trellis leaving and possible MSE distance.

A general block time unit l, a total of k information

so that the branches in the the same state have the largest

un1

(kl U1

=

is shown in Figure 18.11. At each (k) (k-1) (1)) h ,··· ,u 1 , enter t e ( u 1 ,u 1

>

0

-- ->

(k)

- - > Signal mapper ,___>- Y1

lit

Convolutional encoder

u,

rate

(I)

k!(k + 1)

FIGURE 18.11:

General TCM encoder

and signal mapper.

980

Chapter 18

Trellis-Coded Modulation

system. Of these, a total of k :': k bits, namely, u;kl, u}k-l), ... , u}1l, enter a rate R = k/(k + 1) systematic feedback convolutional encoder, producing the output (k) (k-1) (1) (0) h . (0) . ,, ', b', , (k) (k-1) (1) . b its v1 , v1 , · · • , v1 , v1 , w e1e v1 1s me paruy H, ana v1 , v , •.. , v 1 1 are information bits. These k + 1 bits enter the signal mapper along with the k - k . formatlon . b'its u (kl = v (kl , u (k-1) = v Ck-ll , · · · , u (ic+ll = v (k+ll . p·ma ll y, unco d e d m 1 1 1 1 1 1 the k + 1 bit vector v1 = (v;k), v?-1), · · · . v;1), vf 0 l) is mapped into one of the M = zk+l possible points in the signal set S. If k = k, then there are no uncoded information bits and no parallel transitions in the trellis diagram. In the next section we will study a technique called mapping by set partitioning [1] in which the k+ 1 coded bits vfkl, v;k-1), · · · , v?l, vf 0 l are used to select a subset of

v?

size zk-k from the signal set S, and then the k - k uncoded bits 0 , v?-l), ... , vfk+ll are used to choose a particular signal point from within the selected subset. Thus, a path through the trellis indicates the particular sequence of selected subsets. and the zk-k parallel transitions associated with each trellis branch indicate the choice of signal points within the corresponding subset. This mapping technique allows us to design TCM systems that satisfy the two basic design rules noted. 18.2

TCM CODE CONST~UCTiON

There are three basic steps in designing a TCM system: 1, Signal set selection 2. Labeling of the signal set 3, Code selection

A signal set is chosen primarily to satisfy system constraints on spectral efficiency and modulator design. For example, if a spectral efficiency of 17 = k bits/symbol is desired, a signal set with 2k+l points must be selected. Similarly, if, because of nonlinearities in the transmission path (e.g., a traveling wave tube amplifier), a constant-amplitude signaling scheme is required, then a PSK signal set must be chosen. H amplitude modulation can be accommodated, then a rectangular or QAM signal set will give better performance. Several typical signal sets were shown in Figure 18.1. As an example, consider a linear transmission path and a spectral efficiency requirement of 17 = 4 bits/symbol, the specifications for the CCITT V.32 modem standard that can achieve data rates up to 14.4 Kbps over voice-grade telephone lines. In this case, the 32-CROSS signal set was chosen for implementation. The next step in the design process is to assign binary labels. representing encoder output blocks, to the signal points in such a way that the MFSE distance dJ,.ee of the overall TCM system is maximized. These labels are assigned by using a technique called mapping by set partitioning [1 ]. This technique successively partitions the signal set into smaller subsets of equal size, thereby generating a tree structure in which each signal point corresponds to a unique path through the tree. If binary partitioning is used, that is, at every level in the partitioning tree each subset from the previous level is divided into two subsets of equal size, the tree has

TCM Code Construction

Section ·1

k

+ 1 ievels.

each a (k + which can then be To maximize must be done in such a way that the t1.No basic aJ.les for discussed in Section 18.1 are satisfiecL This that the the MSE distance between be n1axirnized at each level p of the n,~·~··~n~h is illustrated vvith tvvo

Consider the

set S shovvn in set S.

tree for the 8-·PSK

Level O of the

unit and is denoted b~ 1; 11 , the MSE distance The notation indicates that this tenn to the MISSD at level O in the Label bit v< 0 > then divides the set S into two Q ( v(Ol) = Q and Q (1), each four lhat the MSSD of both subsets at level 1 to out here two of this JL There is no partition of 8·-PSK into two equal-size subsets that achieves a MSSD. subset Q (1) in the sense that Q (1) can be '.1:o Subset Q (0) is obtained from Q (0) by the points in Q (0) 45°.

I

a

,....o . . .

6

I

l)

o

\

I

j,~ = 0.586

1'

f._)

'
Q(l)

Q(O)

0

b.i =

2.0

v(ll

Q(OO) /0

0

0

Q(10) --- /) '-.. 0

I

I

e,

I

'P

I

-:)---

a

'

I (/)

I

b

',v

0

0

\

I

'I ;J

6

Q(OOO)

/

0

0

\

'I'

I

p

b

I

,v

I ,0

0 '

0

I (/)

\

'/'

I

b

Li~= 4.0

'P

;J

0 /e,

(l

\

(/)

/

Q(ll)

Q(Ol)

p

'·EV

'·EV

Q(lOO)

Q(OlO)

I

I

a

cb I iS)

/e,

0

\

I

'P ;J

cb

. . . -o-Q(llO)

a

,,...e....._

a

0

I

'/'

I

b

. . . . -e-

p

Q(OOl)

I

/

'

/&, 0

d, I

b

',v

0

I

I

'/' ;J

0

0

I (/)

\

'l'

I

b

Q(101)

FIGURE 18.12: Partitioning of 8-PSK.

;J

',v Q(Oll)

/e,

0

I

I

cb I 0

...... -9 ..--

'/' Ll~ ;:!

Q(lll)

=

%

982

Chapter 18

Trellis-Coded Modulation

These two properties of 8-PSK partitioning. namely, maximizing the MSSD and maintaining an isometry among all subsets at the same level, are characteristic of most practical signal set partitionings. The isometry property implies that the MSSDs are the same for all subsets at a given level. Continuing with the example, we see that label bit v< 1l now divides each of the subsets Q (0) and Q (1) at level 1 into two subsets, containing two signal points each, such that t.~ = 4.0 for each subset at level 2. We see again at level 2 that the subset distance has been increased and that the four subsets are isomorphic and thus have the same MSSD. The four subsets are denoted as Q (v< 1lv< 0>) = Q (00), Q (10), Q (01), and Q (11), representing the four possible values of the binary label (vC 1lv< 0l). Finally, label bit vC 2l divides each of the subsets at level 2 into two subsets containing one signal point each at level 3. This is the lowest level in the partitioning tree, and the MSSD t.~ at this level is infinite, since there is only one signal point in each subset. The eight subsets at level 3, Q (v<2!v
Section 1

Branch labels for 4-state. rate R

TCM Code Construction

9113'

= 1/2 coded 8-PSK. 2

both states in the

{Q

and subset Because each of and their MSSD this guarantees that the the same state is 111,aAHH.lU~u, TCM The final labeling of branches 18.13, where the trellis a rate

Q {Q .Q to the othe!· half the level-1 subsets ( Q (0) and Q (l)) contains for a subset of four thus R = 1/2. feedforward encoder.

The o

reniarks relate to

18.6:

The one level-1 subset (Q (0) or Q (1)) to aH the branches and each state that the code bit vCO). which deterrnines the subset chosen at levei 1. must be the same for each set of branches or state. This some restrictions butterflies in the code trellis are to subset Q (0) and the other half to subset Q (1). This ensures that all signal points are used with

o fo

o

to ensure that the diverging that 81'-ee :::: this case. the trellis is connected or thus

in the manner described

His and

;12 -'1 • ·'h [1 us

Hl

and contains 2 A trellis section of any (11.

k.

P)

encoder can be decomposed into a set of

2t,f.

fully connected

subtrellises containing 2f states each. These subtrellises. called butrerf/ies. connect a subset of 2I states at one time to a (in general. different) subset of i: states at the next time. For example. in Figure 18.13. the pair (2i: = 2) of states So and S2 connect to the state pair So and S1, forming one of the 21'-I = 2 butterflies. and the other butterfly is formed by the state pair S1 and S3 connecting to the state pair S2 ands.,.

9184

Chapter 18

o

Trellis-Coded Modulation

remerging distance must equal only ,0,.5. Hence, 2-state trellises (v rute R == 1/2 codes (k == 1) do 11ot yield good TC!v1 designs.

= 1) with

If a rate R = 2/3 code is used in the preceding example, then k

=k

= v,

and the trellis is fully connected. This implies that o}iee is at most equal to ,0,.5 + ,0,.f = 2.586, no matter which code is selected. Thus, for 4-state 8-PSK TCM schemes with 17 = 2 bits/symbol, rate R = 2/3 codes are suboptimal compared with rate R = 1/2 codes with one uncoded bit.

@

®

®

@

In the partitioning of 8-PSK, the two subsets at level 1 are equivalent to QPSK signal sets, and the four subsets at level 2 are equivalent to BPSK signal sets. This isometry between subsets at the same level of the partition tree is characteristic of all PSK signal set partitionings.

For the 8-PSK partition shown in Figure 18.12, mapping by set partitioning results in the natural mapping rule discussed in Section 18.1. If the order of the subsets at any level in the partitioning tree is changed, the resulting mapping is isomorphic to natural mapping. Mapping by set partitioning always results in the distance relation ,0,.5 :S ,6,. ~ :S · · · :S ,0,.Tc, which, along with the proper assignment of subsets to trellis branches, guarantees that the two rules of good TCM system design are satisfied. The separate tasks assigned to coded and uncoded bits by set partitioning, namely, the selection of subset labels for the trellis branches and the selection of a signal point from a subset, respectively, imply that the general TCM encoder and mapper in Figure 18.11 can be redrawn as shown in Figure 18.14.

Signal mapper

(kl lit

(k

+I)...:..._ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [

Select signal from subset

lit

(I-)

lit

-0

e

//)II

"--l>

v;1'1Convolutional encoder v)11--;> rate

FIGURE 18.14:

kl(k + 1)

Select subset

v)111--;>

Set-partitioning TCM encoder diagram and signal mapper.

Yt

Section 1

H k = k, then there are no trellis become labels .

As a second of set S, shown in Figure 18.15. the average energy is

. . ~,ua,c.,

Es= (1/16) {4 [(L',,o/2)

2

TCM Code Construction

~lll:5

and the subset labels on the

we consider the 16-QAM signal set, denoted ,-p,cwpcp,»- the MSSD at level 0, we see that

+ (lio/2)2] + 8 [(3L1o/2) 2 + (Llo/2)2]

+ 4 [(3L1o/2)2 + (31'.1 0 /2) 2 ]} =

(1/16) [ 21i5

+ 2.0li5 + 181'.15] = 51'.15/2.

(18.22)

i'.15 = 2/5 if the average energy = 1. At level 1 of the partitioning tree, we obtain the subsets Q (0) and Q (1), each to an 8-AM/PM constellation, and it is easy to see that L1I = 2L15. down the partitioning tree, we obtain four subsets at level 2, each to 4-QAivL with L'.1~ = 2L1f; eight subsets at level 3, each isomorphic to 2-AM, vvith = 21'.1~; and, finally, the 16 signal points at level 4, each labeled according to the set-partitioning mapping rule. The following observations relate to

18.7:

o

The 16-QAM set can be considered a multidimensional version of 4-AM, that 2 >< 4-AM.

o

fo the 16-QAM case, the MSSD doubles at each level of the partitioning tree; that Llf = 21'.1;_ 1 . i = 1. 2. · · · , k. This is characteristic of most partitionings of rectangular-type signal constellations used in practice.

o

16-QAM is a subset of the two-dimensional integer lattice Z 2 , and the subsets at each level of the are isomorphic.

o

It is not signal sets based on a lattice in such a way that all subsets at a given level are isomorphic. In this case, although the subsets are no longer distance all still have the same MSSD An example of this situation is shown in Section 18.4 for the 32-CROSS constellation.

t,,.7.

o

TCM systems based on 16-QAM modulation can employ code rates R of 3/4 or 2/3 with one uncoded bit, or with two uncoded bits.

We now consider the last step in the design process, that of code selection. Assume that the code is generated by a rate R = k/(k + 1) systematic feedback convolutional encoder with parity-check matrix (18.23)

986

Chapter 18

Trellis-Coded Modulation

0000---:r~

0000::; 0000

N

II

~

('<]]

0

0

0

0000,....(

O /

0000.::::,,

__.-;;ro o o o ~ o o o o ~ 0@00

000@~

0000

ggggg

O

0 © 0 0 0

0

0

0

0

@

0

©

0000~ 0000~

oooos

0000 ~0000 /

0

0000,

o o o o .....,

"

0

0

G

cSi

"

0

0

0

~


.g

"St

1i 0N

II

ci

('~

I

.
\0

,-;

V'.)

0

0

0

0

0

0

0

0

G

I

~

0000,,-...,_

~

oooO ~

o~gggg2, 0 0 0 0 ;:00000 @000,.....(

00@0 0000 /

ooooC:.,

~,...,...-0000 __.-'-....._ 0 0 0 0 ~ 0

0

0

0

0

0

0

0

G)

0

C9 0

0

0

0

0

0

0000 ~0000

0

" " " " ., " " "' "' G

0000-

0 0 0 0 ;:;-, 0000 O

~ggggg 0000 ~ 00000

7

0000;=; 0000-.,

0

000~~0000~ 0000 00000

6666

0

0

s

0

0

0

0

0 0 0 0 .-...,

ooooS

0 ,....(

__.-,r 0

® 0

19

0

0

0

0

0

® 0 {)

0

0

0

0000 0000

0000:::: 0000'-"

/

O

0

oe~

0

0

0

00000 l l ~ 0 0 0 0 ,....(

0

.,.-,

0

o o o

0

0

0 0

8

0

0

G

@

0

Q

0

0 0

0~

oooos

0

gggg 8

c , ~ 0000

© 0

........

~ II

0000

0000

0

0

0

0

0

0

0

0

0

0

0

0

Q

O

O

0

/

0

0~

0

0

0

0

~00©0 O

0

@

0

FIGURE 18.15: Partitioning of 16-QAM.

0

7ooooo 0000-_-

0

0000

0000

~

0000::;"

~o

O

S

0

0

_.---

o eoo

0000"-'

0

c5io@o0 ® 0

s

gggg §

.,.-,,/

~o o o o/ O

ggg~

0

oooos

0 ~ 00000 0

gggg 8

7

0

0~

0

0

0 0 0 0 ,,-.__ 00006 00000 ©000~

00000

0 ~ 0000 O

~

66668

Section 1

where vector

TCM

= h6i) + h;il D + · · · + D''. j 0. L · · · . Tc v is the constrai.nt 0 and li60J = h;, ) = 1.is for a miaimal encoder I"ealization. Then . the = [v
j)

vvhere ,/.iJ

+

j = 0. 1. · · · . k;

7

Y/

E

that selects the subset at time I; the all-zero sequence. The

set-

frmn any information {]) = '162) n1nst be the same for all a state. to guarantee that is the same for all branches state. there must be no connections from an information sequence to _ th a t/ Iv(11 -/(2) 1,, -

-/(kl --0 . '""I' 1,, ·1. 11e~e 0

· · · -

restrictions kad to the feedback convoluti.onal encoder realization shovrn m that is used to search for TCM code for event -error MFSE distance

is to seiect codes that maxnmze the and minimize the ,werage numbe.- of nearest =~···
then select those with the betvveen transitions is 1he MFSE distance between trellis

that the MSE distance 8~,in we can use ( 1 to express

(1

tO

from the

t:-elli.s of the code and the MlSSDs lemma. lLEMMA

in the

]_1.t2 [J.] Let q(e) denote the number of zeros in the error vector e, for example. CJ (e
follov,s from the fact that if ,, and v' signal

two trellis branch labels. then their

Y

EB e

988

Chapter 18

§

:.::, >

Trellis-Coded Modulation

§ §

§

->

->

,..:,:

->

§

§ § 1..:,,:

~

->

->

->

->

4

4 0

0

0

0

0

0 0

0

0

0

0

0

~

D ~

Q

0

0

cl ,... <J)

0 0

§ ~

-;::,

0

0 0

0

0

0

0

0

-~~ -;::,

i::


0

0

§

i5'

-;::,

-;::,

§ /_;..,:

~

-;::,

§ -;::,

FIGURE 18.16: Two systematic feedback convolutional encoder realizations.

labels in the set-partitioning tree will agree in the trailing q (e) positions. This implies that they follow the same path through the tree for the first q(e) levels, and thus t.~(ie) ?: t.~(cr Because this condition holds for all v, 2

t. (e)

= minv t.~(e)?:

Q.E.D.

t.~(cJ·

Using the set-partitioning lemma, we can now write 8.f2.1·eo ?: L

min ~ ,0. 2 c(D)icO(D) L., q(c!)·

I

(18.27)

Section 1

The

comments

o

For the case

TCM Code Construction

to the

= ©, we take

!ll~!ll

lemma.

=

\2 Luq(I[))

= 0.

0

are l\ 2 (1001) > sa tisfiecl with since there are ::f. that achieve the minimum_ value of one that does not include any error vector e1 for

0

the MFSE distance sin.Ce it does not of every error vector. This can be sets or if codes are based on a lattice signal set

0

uuu,-,,-~, ~

search nr,rv•,,,n search [4]. Each

,._,vo,F,,rn based On the -( d). The codes ,vere found information:

o

The 1\/iSSDs

o

The number of coded information bits The in_ octal forn1.

o

= 0. 1, · · · . k.

coefficients rrr/.il =

The IV[FSE distcmce

k.

[1i\P. hUl 1. · · · J}-

(j)l ' = 0· 1· .. · · '-<,

1 · I1 0

An asterisk (,1,) indicates that

J

· J

1

occurs only along

o;,

that is. 81,ee > 1i11 • In Tables 18.6(a) and the ratio the average energy = 1, is the MSSD at level 0, constellation considered, but

with an uncoded modulation The notation denotes the two denotes the of a coded 32-CROSS ·with uncoded 16-QAM. The number of information bits k transmitted per coded symbol, which the 17 in bits/symbol, is also In Tables 18.6(a) and (b ), for several different efficiencies based on constellations chosen from the same lattice. The notation Ye; u denotes the of a coded lattice of infinite size with the uncoded lattice. o

The average number of nearest 1S for the infinite

u·-·-'-•"·"L''V' 0

1 ' " " " "'• .,

fo Tables 18.6(a) and (b), case, that k -> 00.

990

Chapter 18

Trellis-Coded Modulation

TABLE 18.6: List of optimum TCM codes. (a) Codes for one-dimensional AM based on IL~. 0 Si S 2} =

V

2

2 3 4 5 6 7 8

k

!Jill

1,(0)

1 1 1 1 1

2 2 04 04 10 024 126 362 362

5 5 13 23 45 103 235 515 515

I I I 1

zl

Lt. 466. 1666

Y4-AM/2-AM

YS-AM/4-AM

YC/U

Adfree

d?,.ee/ 1;,,f,

k = l

k=2

(k -+ 00)

(k-+ 00)

9.0 9.0 10.0 11.0 13.0 14.0 16.0 16.0* 17.0

2.55 2.55 3.01 3.42 4.l5 4.47 5.05

3.31 3.31 3.77 4.18 4.91 5.23 5.81 5.8[

3.52 3.52 3.97 4.39 5.11 5.44 6.02 6.02

4 4 4 8 12 36 66 2

-

-

5.30

-

(b) Codes for two-dimensional AM/PM based on

V

I

2 3 4 5 6 7 8 9

k I

1,!21 -

04 2 16 2 10 2 2 064 2 042 2 304 2 0510

11111

;.IOI

z2

Yl6-QAM/8-l'SK

Y32-CI!/ 16-QAM

Y64-QAM/32-CDl

YC/U

dT,,e/ 11,fi

k=3

k=4

k=5

(k _. 00)

4.0* 5.0 6.0 6.0 7.0 8.0 8.0 8.0*

4.36 5.33 6.12 6.12 6.79 7.37 7.37 7.37

3.01 3.98

2.80 3.77 4.56 4 ..'i6 5.23 5.81 5.81 5.81

3.01 3.98 4.77 4.77 5.44 6.02 6.02 6.02

2 5 11 02 04 23 06 41 016 IOI 014 203 056 401 0346 1001

4.77 4.77 5.44 6.02 6.02 6.02

;\f~ji·ee

ik

-+ 00)

4 16 56 16 56 344 44 4

II

(c) Codes for 8-PSK {Lr OS i S 2) = 4sin 2 1Jr/8). 2. 4 Y8-PS1K/4-PSK V

k

2 3 4 5 6 7 8

1 2 2 2 2 2 2

1,i21 -

04 16 34 066 122 130

11111

1,101

'1·ee/ /;,,~

k=2

Ad free

2 02 04 16 030 054 072

5 11 23 45 103 277 435

4.000* 4.586 5.[72 5.758 6.343 6.586 7.515

3.01 3.60 4.13 4.59 5.01 5.17 5.75

1 2 "'2.3 4 "'5.3 "'0.5 ccl.5

(d) Codes for 16-PSK

Yl6-i'SK/8-PSK

" 2

3 4 5 6 7 8

k l l 1 I l 1 2

Adapted from [4].

1,121

-

--

374

hill

1,111)

2 04 04 IO 024 024 176

5 [3 23 45 103 203 427

'1·e/il.~

k=3

U24 1.476 l.628 1.910 2.000' 2.000* 2.085

3.54 4.01 4.44 5.13 5.33 5.33 5.51

Ad

free

4 4 8 8 2 2 "'8.0

-

Section 1

TCM Code Construction

!ll!lli

The codes listed for one-dimensional AM are based on the one-dimensional integer lattice Z 1 , and the codes listed for tvvo-dimensional AM/PM are based on the two-dimensional integer lattice Z2 , where these lattices are infinite extensions of the one- and two-dimensional constellations shown in Figures 18.l(a) the same maximum dJ,-eel and 18.l(b ). In these cases the same codes independent of the size of the constellation chosen from the lattice, although the minimum number of nearest can vary to the effect of the signal constellation boundaries. In Tables and (b ), to negate the effect of Ac1/iee assuming a constellation we list the average signal constellation of infinite size. Because set partitioning of an infinite lattice results in a regular the values of in Tables 18.6(a) and (b) are all integers. In Table 18.6(a), we see that for codes based on Z 1 , the asymptotic k; that is, the largest coding coding gain y increases with the two 256-state codes are gains are achieved in the limit as k --+ oo. listed. The first code, whose parallel transitions, is optimum when the number of information k :::,: 2; that when the trellis contains parallel transitions. The second code, which achieves a larger dJ,-ee' is optimum only when k = 1, that is, when the trellis does not contain transitions. In Table 18.6(b) we note the relatively large of coded 16-QAM compared with uncoded 8-PSK. This difference is due to the restriction that PSK signals must an have the same energy. The coding gains of 16-QAM compared with uncoded rectangular constellations are not as as shown in Problem 18.15. In contrast with the lattice-based codes, in Tables 18.6(c) and (d) we see that different codes are optimum for 8-PSK and 16-PSK and that the nonregular mapping Adfiee. can result in values of the average When a trellis contains parallel transitions, care must be taken in computing branch may contribute to a minimum-distance the value of Ac1fiee' since each For example, in Table Ac1fiee = 4 for the 4-state coded integer lattice to the error trellis in for code 2, which is equivalent to the 4-state code in Table 18.6(a), we note that the error trellis for the coded lattice set of parallel transitions. Z 1 is formed by replacing each branch with an fo this case the trellis branches labeled e = (00) will now contain the set of parallel transitions representing all error vectors ie = (· · · eC3le(2)00), the trellis branches labeled ie = (10) will now contain the set of parallel transitions representing all error vectors ce = (- .. e(3li 2 llO), and so on. The MSE distance 9t.5 is achieved by a path that diverges from state So along the branch labeled e: = (10) to state S1 (a squared distance of 4t.5 ), continues to state S2 along the branch labeled <e = (01) (a squared distance of t.5), and remerges with state So along the branch labeled e: = (10) (a squared distance of 4t.5). Now, note that for a given parallel transition on the branch labeled e: = (00) leaving state So, say ie = (· · · eC 4 le(3lOOO), there are two parallel transitions, e: = (· .. eC 4leC 3 lllO) and ie = (· .. eC 4le(3)010), with squared distance 4t.5 on the diverging branch labeled e = (10). The same situation holds when the minimum-weight remerges with state So along the branch labeled e = (10). For the middle branch on the minimum-weight path, there is only one parallel transition, e: = (- .. eC 4 l i 3 l001 ). with squared distance along the branch labeled e = (01). Because there are four possible combinations of minimum-weight

~5

~5

992

Chapter 18

Trellis-Coded Modulation

paths, in this case, Adfiee = 4. Another example of computing Ac1/iee for a trellis with parallel transitions is given in Problem 18.16. Finally. we recall that when of the coded system exceeds Amin of the uncoded system, the real coding gain at practical BERs of around 10~ 5 is somewhat reduced compared with the asymptotic coding gain y. It is interesting to note that many of the optimum codes listed in Table 18.6 contain one or more uncoded bits. This is because, particularly for short constraint lengths, the parallel transition distance o;zin = 6.i+i is already larger than the free distance o}ree between trellis paths, and thus using a higher rate code cannot improve the overall free distance dii-ee· For longer constraint lengths, however. the free distance o}ree between trellis paths increases, and then more coded bits, that is, a larger k, must be used to increase the parallel transition distance o;,in and consequently the overall free distance dT,·ee· 18.3

TCM PEIRFORMANCIE ANALYS!S

The average weight enumerating function (A WEF) Aav(X) and the average input output weight enumerating function (AIOWEF) Aav(W, X) of a TCM system can be computed by labeling the branches of the binary error trellis with their corresponding AEWEs, augmented by the input weight enumerators when computing Aav(W, X), and then forming the modified state diagram and using the transfer function approach developed in Chapter 11. Once A"v(X) and Aav(W, X) have been evaluated, the event-error probability P(E) and the bit-error probability Pb(E) can be estimated using the union bounding techniques developed in Chapter 12. For an unquantizedoutput A WON channel whose inputs are drawn from the TCM signal set, this process gives the expressions P(E) :S f(d}.eeEs/4No)Aav(X)IX=e-t,,/ 4 N0

(18.28a)

and (18.28b) where f (x) = ex Q( v'2x), and dj,·ee is computed under the assumption of a unit average energy signal set. The reader should note the similarity between the expressions in (18.28) and those derived for binary convolutional codes in Chapter 12. In fact. they are identical except that the WEFs are replaced by average WEFs, and the Hamming distance, dH,fi-ee, in Chapter 12 is replaced by dT,·ee/4, where dkee is SE distance, in the preceding expressions. This reflects the fact that, for unit energy binary signals, dj,·ee = 4dH,jiee, as noted in (18.8). The bounds in (18.28) are valid for any TCM system without parallel transitions, that is, the case for which each error event represents a path through the trellis at least two branches in length. In the case of parallel transitions, that is, one-branch error events, the bounds are modified as follows: P(E) ::=: f(o;zinE.,/4No)Afv(X)IX=e-t.,/4No

+ f (o?i·ee E., / 4No )A~,v (X) IX =e-'' 14 N0

(18.29a)

Section 18.3

TCM Performance

and

X)/aw\

v_

./1..--e

.

-fo .1 '/,jNO

X)/ow\ X=e!- .

+

"'-1

, \'V-

1~1 i-l-,1\\1.

H'=l

.

represent the A-WEF and AIOV/EF fo, the the AVVEF and AIO'WEF and A~IV (h should be noted that Afv(X) and A!:v(W. X) of the subsets at the last that ievel k + t in The use of WEFs to evaluate the ·,er,~,·,~ . ; anc!' 'JI/ ..7~ehav, v olf 'l2d] . , an d an in [25]. We now illustrate the

VVe consider the 4-state, rate R = 1/2 feedforward encoder shown for code 2 in Figure 18.4(a) along with 4-AM. The encoder W3S shown in 18.4(b ), and the AEWEs of 'Nere listed in Table 18.4(b). In Figure 18.17(a) we show the modified state labeled with the AEWEs. We now the A WEF using the standard transfer

C

C

+

b=

b= c=

. _ v!l.S C -- ./t.

cl= e=

+

7.-:l.

)(O.S

f=X" g = x~2 (a) Modified state diagram

d=

e _- Ay(I.S f'=WX" g=

x-'2

(b) Augmented modified state diagram

F!GURE 18.17: Modified state diagrams for naturally mapped, 4-state, rate R trellis-coded 4-AM.

=

1/2

994

Chapter 18

Trellis-Coded 1\/lodulation

function approach as follows: _ xos) + x3·2co.sxos + o.sx1.2)2 x3.2. (1 - xo.s)(l - xos xo) - (O.sxo.s + O.SX7.2)2 xo

x32 xo.s x3·2(1

Aav(X) =

=

x 72 - 0.7sx 80 + o.sx 14.4 + o.2sx 208 1 - 2xos + 0.7sxt6 - o.sxs.o - o.2sx14.4 x 72 + 1.2sx80 + 1.75x88 + 2.062sx 9·6 + ....

(18.30a)

Equation (18.30a) implies that for an arbitrary transmitted sequence y, there is an average of 1 error path y' with MFSE distance d};.ee = d°;,(y, y') = 7.2, an average of 1.25 error paths y' with SE distance d1(Y, y') = 8.0, an average of 1.75 error paths y' with SE distance d}(y, y') = 8.8, and so on. In Figure 18.17(b) we show the modified state diagram augmented by the input weight enumerators. In this case, following the same procedure as before, we find that the AIOWEF is given by Aav(W. X)

=

2

WX 7 ·

+ 1.25W 2X80 + 1.7SW 3 X 88 + 2.0625W 4X9·6 + · · ·.

(18.30b)

Here we see that the error paths at a distance of 7.2 from the correct path are always associated with 1 information bit error, those at a distance of 8.0 are always associated with 2 information bit errors, those at a distance of 8.8 with 3 information bit errors, and so on. Finally, the expressions of (18.30) can be used in (18.28) to evaluate P(E) and Pb(E) as functions of the channel SNR E.,/ No. The bounds are sketched in Figure 18.18 along with uncoded BPSK, which has the same spectral efficiency of 17 = 1 bit/dimension. The following observations relate to Example 18.8: @

®

@

The codeword multiplicities are averages because TCM systems are nonlinear, and the number of codewords at a particular distance from the correct sequence depends on the transmitted path. The codeword multiplicities are fractional because the signal constellation is finite, and the mapping is nonregular. Thus, the multiplicity of 1.25 associated with incorrect paths at distance 8.0 means that, depending on the correct path, there may be either 1 or 2 incorrect paths at distance 8.0.

= 1/2 code was used, along with an infinite number of uncoded information bits, to code the one-dimensional integer lattice Z 1, the average multiplicities would be integers, since a regular signal mapping can then be achieved with set partitioning, as noted in the previous section. In this case the average number of nearest neighbors is Ac11;,,,, = 4, since the parallel transition subsets in both the first and last branches of the shortest error event contain exactly two signal points distance 3.2 away from any given reference point (see Problem 18.15).

If the same rate R

Section 1

TCi\11 Performance I BPSK ··--P(E) ~~~, P1,(E) ··········

0.1

0.01

;§ ..0

"'

..0 0

~-

0.00[

'-"

l -•

0

rl:l'" 0.0001

le-05

le-06~-~-i---~~~~~~~~--~---~~~~~~~~-~~,~~~-~

2

4

6

8

10

12

E/N11 (clB)

FIGURE '18: Error treHi.s-coded 4-AM[.

bounds for

4-state. rate R

=

1/2

structure of the encoder in this a determinexists between codeword distance from the correct path and the SE disthat d7,; = 6.4 + 0.8w I, vvhere d7,; tance from the co:crect and w I the information For all codewords distance 14.4 from the correct have 10. information

istic

0

From the bound on P1,(E) in gain at a BER of 10- 5 of this TCM 2.1 dB. This coding

Ul.18. we see that the real coding with uncoded BPSK

rate R = 1/2 binary feedback encoder shown for code 2 in with one uncoded information bit and naturally mapped 8-PSK error trellis of this encoder was shown in Figure 18.9(b), transition connecting each of states. and the AEWEs 8-PSK modulation were listed in Table 18.5. fo Figure 18.19(a) we show the modified state in which each branch is labeled with the sum of the AEVVEs for the corresponding parallel transition branch labels in the binary the A WEF for the trellis paths A~v (X) using the trellis. V\/e can now

Now, consider the

996

Chapter 18

Trellis-Coded Modulation C

n

f 2

a= (W+ W )X 2

2

a= 2X b = xo.oK6 + xHt4 C

d =

e

b =

+ X.1414 xo.,86 + x.1414 xll.586 + x.1.414 1 + x4

c

= X0.,86

=

f=

hw +

w2)(X11,K6 + x.1414) + wx.1.414 ~(W + w2)(X11,86 + x.1414) xll.586 + wx.1.414

= xo.w,

d = e=

f= 1 +WX 4 g= (W+ W 2 )X 2

g = 2x 2

(b) Augmented modified state diagram

(a) Modified state diagram

Modified state diagrams for naturally mapped, 4-state, rate R trellis-coded 8-PSK.

FIGURE 18.19:

= 1/2

standard transfer function approach as follows: 2 x2cxoss6 A;,v(X)

+ x3.414) 2 X2( 1 _ xoss6 _ 2 x2cxoss6 + x3Al4)2 2 x2

x3.414)

+

= --------------------~ [1 _ X0.586 _ x3.414][1 -- (X0586 _ x3414)(Xo [X0586 + x3.414]2[xO + X4] 4X4586

+ X4)] _

+ 4 x7.414

1 _ 2 xoss6 _ 2 x3.414 _ x4s86 _ x7.414

= 4X4586 + 8X5172 + 16xs 758 + 32 x6344 + ....

(18.31a)

Equation (18.31a) implies that for an arbitrary transmitted sequence y, there is an average of 4 error paths y' with a MFSE distance between trellis paths of 81,-ee = d}(y, y') = 4.586, an average of 8 error paths y' with free SE distance d}(y. y') = 5.172, and so on. Because this TCM system includes parallel transitions. we must also compute the parallel transition A WEF AKv(X). From the 8-PSK set-partitioning tree, we see that there are only two signal points in each parallel transition, and that (18.31b) A;~,,(X) = X 40 . Equation (18.31b) implies that the MSE distance between parallel transitions is o?iiin = 4.0, and hence the MFSE distance of the TCM system is (18.31c) as noted earlier in Example 18.4.

Section ·133

TCM Performance

In 18.19(b) we show the modified state enumerators. In this case, the same that the AIOWEF is by (see Problem (W, X)

2

= ( W + w2 /

+

we find

+ [ ( w + w2 ) + 0.25 ( w + w2 ) + w4) x4ss6 +

+ l.5VV 6 +

+

4 ]

x 5 172 + ...

+

+

Here we see that the error at a distance of 4.586 from the correct are associated vvith 2, 3, or 4 information bit errors, those at a distance of 5.172 are associated with between 2 and 8 information bit errors, and so on. In the coefficients of the W terms denote the relative likelihood that a certain number of information bit errors will of a For the terms 1 indicate that error 5.172 from the correct are five times more to have 4 information bit errors than 8. the parallel transition AKOWEF is

\
2

4

6

8

10

12

14

EJN11 (dB)

FIGURE 18.20: Error trellis-coded 8-PSK

bounds for naturally

4-state, rate R =

998

Chapter 18

Trellis-Coded Modulation

Now, we can use the expressions of (18.31) in (18.29) to evaluate P(E) and P1,(E) as functions of the channel SNR Es/No. The bounds are sketched in Figure 18.20 along with uncoded QPSK, which has the same spectral efficiency of 17 = 2 bits/symbol. The following remarks apply to Example 18.9: ®

The WEFs for each possible parallel transition, that is, for each subset at level 2 of the partition tree, are identical, because the four BPSK subsets at level 2 are isomorphic. fo general, however, this may not be the case, and Afv(X) is computed by taking the average of the WEFs of each subset at level k + 1 in the set-partitioning tree.

e

The MFSE distance path is a parallel transition. This implies that at high SNRs, Afv(X) and Afv(W, X) are the dominant terms in the error probability bounds, thus allowing approximate bounds on P(E) and Pb(E) to be obtained very simply.

o

The possible codeword weights represented in (18.3 la) and (18.31 b) are separated by the value t-.~in = 0.586, the MSE distance between signal points. This is characteristic of any TCM system; that is, codeword weights increase by multiples of t,.~in.

®

From the bound on P1,(E) plotted in Figure 18.20, we see that the real coding gain at a BER of 10- 5 of this TCM system compared with uncoded QPSK is approximately 2.6 dB. This coding gain is achieved without bandwidth expansion.

As a final comment before leaving this section, we note that whereas the asymptotic coding gain of a TCM system can be obtained by computing dJ,.ee' as shown in Section 18.1, the real coding gain at a particular BER must be obtained either from computer simulations or estimated from bounds such as the ones presented in this section. 18.4

ROTATiONAU. Y iNVAmANT T(M

The typical signal set used in a coded modulation system has several phase symmetries; that is, phase rotations by certain angles replicate the signal set. For example, 8-PSK has eight phase symmetries, spaced 45° apart, and any QAM constellation has four phase symmetries spaced by 90°. In general, when a receiver locks onto a particular phase, a trial-and-error procedure must be initiated to determine whether it is in the correct phase. U a particular system is prone to frequent loss of carrier synchronization, reacquiring sync can be a time-consuming exercise. Thus, it is desirable in many applications that a coded modulation system be invariant to phase rotations. In other words, if the receiver locks onto the wrong phase, the system should still be able to operate properly. Hence, in case of temporary loss of synchronization, the receiver must lock onto only one of the possible symmetries and need not initiate a procedure to reacquire the correct phase. The basic requirement for a coded morh1fation system to be ir1.variant to a particular phase rotation is that when the symbols of each code sequence are

lnvai-iant TCi\/l

9J9J:91

the

in the rotated set, it is still a valid code the decoder would still decode a proper code sequence, albeit an incorrect code sequence; hmvever, differential of the information sequence and differential of the decoder sequence can be to ensure that the correct sequence is decoded. :nvari2J1t code cc1r1bined ',Nith differential "0,.,,.,," reliable communication even when the :;:eceiver is locked is incurred in BER are doubled differential that if a code is not rotated code sequences may be used to detect an, 0

To illustrate the idea of we start with the case of a code with BPSK modulation in which the encoder bits are The of the signal set which has the effect of inverting the sign of every -1 -+ +1 and -1-1 -> --1. every codeword is For any linear the of a codeword is a codeword if and if the all-one sequence is a codeword. the simple condition for 180° rotational invariance for any linear code with BPSK modulation is that the cill-one sequence is a codeword. ¥Ve now consider the case of QPSK modulation and 90° symmetries, with an_,,.-.,.~,·~·

Consider a rate R = 1/2 convolutional code with

matrix (18.32a)

matrix (18.32b) vvhose two

into QPSK, as shown m (PCE) is given (18.33)

where 'Y(D) = [Y< 1l(D). yCOl(J.J)] a codeword. l"'Tow, note that after a 90° rotation of the signal set, the rotated code sequences become (18.34) where Jl(D) ,-p,·wp,,pn we obtain

the all-one sequence. Using the rotated code sequences in

(18.35)

1000

Chapter 18

Trellis-Coded Modulation

which must equal O(D) for all V,(D) = [v?\D), v~0 \D)] for the code to be invariant to 90° rotations. From (18.33) we see that for any information sequence ~(D), V(D) = [v(ll(D), vC0l(D)] = [l!ll(D)hc 01(D), u(D)hC 1l(D)] is a valid code sequence. Substituting this V(D) into (18.35) we have V,(D)HT (D)

=

u(D) {[ h(l) (D)

J2 EB [h.< \D) J2} EB lbJ.CO\D)l(D). 0

(18.36)

Examining (18.36) closely we see that the first term equals llll(D) times a nonzero binary polynomial of degree at most 2v, where vis the constraint length. Considering l(D) to extend infinitely in both directions, we see that we can write the second term as lht0 (D)1(D) = h_(O) (l)(D), which equals either O(D) or l(D) depending on whether the Hamming weight of JlnCOl (D) is even or odd, respectively. For example, code 1 in Example 18.2 has 1nW1(D) = 1 + D 2 and lht
=

D 2 u(D) EB O(D)

=

D 2 l1l(D).

(18.37)

Clearly, (18.37) is not equal to O(D) for any nonzero lill(D), and thus the coded modulation system is not invariant to 90° rotations. In the case of a 180° phase rotation, we can see directly from Figure 18.3(a) that both rotated code sequences are complements of their respective correct code sequences. This situation is exactly analogous to the BPSK case, and thus rate R = 1/2 convolutional codes with Gray-mapped QPSK are invariant to 180° phase rotations if and only if the all-one sequence l(D) is a codeword. Now. consider the same example with natural mapping. fo this case the 90° rotated code sequences become (18.38) Again, substituting in (18.33) and using V(D) obtain V, (D)HT (D)

= l!ll(D)

{ [ i1i0\D)

=

[u(D)lh/Ol(D), l.Il(D)lhi<11(D)], we

J2} EB h(Ol (D)l(D).

(18.39)

which for code 1 in Example 18.2 becomes V,(D)HT (D)

= (1 + D + D 2

)2 lill(D) EB O(D) = (1 + D

2

+ D 4)

l1l(D).

(18.40)

As in the case of Gray-mapped QPSK, we see that for naturally mapped QPSK (18.40) does not equal O(D) for any nonzero M(D), and thus the coded modulation system is not invariant to 90° rotations. 1 Qno F or<> .._,,..1..u._1._v'J

nhace yu.,

0

l(D) and v~ 1(D) 0

,,([)I") \V

rr.t~t;r.n ma caa J'rr.m y:;o;~,,•·6 1 Q '2/J.. \
= v< 0 l(D); that is,

~a.,(])

-v

f

rn

ffi

\V}w

1

v~ \D) is the complement of the correct

sequence, and v; l (D) equals the correct sequence. In this case, V, (D)JHlT (D) = 0 v}1\D)h( 1\D) EB v;. l (D)ttii.C 0 \D) = h(l\D)l(D). Thus rate R = 1/2 convolutional codes with naturally mapped QPSK are invariant to 180° phase rotations if and only if h(l) (D) has even weight.

Invariant TCM

Section 18A

The

1(0)1Jii

18.10:

renaark:s relate to

o Rotational invariance a of the code, not the encoder, so the results lKlO also hold for the feedback encoder

=

J.

[i:nU> (D) /lln(Ol ( D) 1

to start at time 0, then terms < v) before This vvould affect rotational invariance sense to this transient condition in both directions Problem 18.19). c

to show that any rate R = k/(k + 1) PCE V(D)lI-IT 7 (D) = (D)lh/kl(D) EB (D)!ln(OJ (D) = l(D(D) that maps into a tvvo-dimensional can at best be invariant to 180°

This can convolutional code ;,,vith a linear · .. ffi,/lJ (D)lln(l)

set vvith 90° rotatioEs o

As wiH be seen next,

rate R = k / (k + 1) convolutional codes can achieve ''"'"'"""' use of a nonlinear PCE.

To see how nonlinear PCEs can achieve a greater degree of rotationai invari· ance than linear lPCEs. we consider the case of a rate R = 1/2 convolutional code ·with modulation. As noted in 18.10, linear 90° rotational invariance in this case. Before PCEs are we vvrite the two encoded sequences in integer form as follows: vvher1; the coefficients of Y(D) are elements in the ring of :;:E4. the remainder of this we will use the + to denote addition in a EB to denote addition as 7,(D)

vve can yvrite the tvvo

=

+

(18.42)

4).

polynomials in integer form as

consider the PCE (mod 4)] 1 =

l(D(D),

vvhere the notation [i:1 (D)] 1 means that from the binary representation of every 1 1 element a 1 = ! + aJl l E ::Z4 in vi (D) the most significant bit is chosen; that is, [(ot(D)]1 = a/1). represented as [01(D)]1 in (18.44) causes this PCE to be both and nonlinear.) For the PCE represented by (18.44) to be invariant

ot?)

1002

Chapter 18

Trellis-Coded Modulation

to 90° rotations, it must still be satisfied when requires that

Vr

(D) is substituted for v(D), which

+ Jt(D)] (mod 4) = h(D)v(D) + h(D)l(D) (mod 4) = h(D)v(D) + h(l)(D) (mod 4).

h(D)v(D) = h(D)vr(D) = h(D) [v(D)

Because h(l)(D) is a constant sequence with h(l) only if h(l) (mod 4) = tP\1) + 21n< 0l(l)

E

(18.45)

Z4, (18.45) is satisfied if and (mod 4)

= 0.

(18.46)

Note, for example, that if HJt
= [lhi<1\D)v< 0\D) + 21!n(l) (D)'>'(l\D) 1

+ 2fu/Ol (D)v(O) (D) + 4l!n(O) (D)v(l) (D) (mod 4) J = {h(l) (D)v(O) (D) + 2 [11/1) (D)v
+ h(O) (D)v(O) (D) J

(mod 4)}

1

= O(D),

(18.47)

where we have simplified (18.47) by noting that the term 4B:ii< 0l(D)v(l\D)(mod 4) = O(D). Following the restrictions of Figure 18.16(b) for good TCM code design leads us to search for codes with parity-check polynomials (18.48a) and (18.48b) that satisfy (18.46), thus guaranteeing 90° rotational invariance. We must then substitute these polynomials into (18.47) to specify the binary nonlinear PCE. We now consider an example in which we choose lhl(l) (D) to have only two delay terms and proceed to derive a general binary nonlinear PCE that guarantees 90° rotational invariance for rate R = 1/2 codes with naturally mapped QPSK. EXAM Pl IE 18.11 Mapped QPSK

Rotarl:ionaily im,ariant Rate R

= 1/2 Codes

for Naturnlly

Consider the choice (18.49)

Invariant TCM

Section 18.4

1(l)IOJ3

where v > b > o > 0. Note that in this case, since !hiCil(l) = 2 bave an odd numbe_t of nonzero terms to (18.4-6). into vve obtain the PCE 4)

the AND of the sequences in (18.50) and recalling that 4m(D) we obtain the PCE

J11 =

O(D).

and

=

1Jl(D) for

+ 2 [Dhv(O\D) o D",1( 0\D) EB (D 1' EB Du)Y( 1\D)EB lln(O)

{D)]}l

= 1Jl(D). where (1 has been the fact that the term (Db EB D")v< 0l(D) Iws no effect on the most bits of the sequence in braces. (18.52) a binary nonlinear PCE that 90° rotational invariance for defined and any Iln(O) with an odd rate R = 1/2 codes with nnOl is the n.umber of nonzero tenns, where the Al\TD Problem 18.21 it is shown that the ··-·-· 0<,ch satisfied when the rotated sequences in (18.38) for natural are substituted into (1 and is assumed to have an odd number of nonzero 0

The addition of the nonlinear term Dh,/Ol(D) o D{[y(O\D) makes code. Given this it is not different from a linear PCE for a rate R = dear if in result in an encoder realization with only 2v states. the specific example lHI(D)

=

[w

2

+ D)/(D 3 + D + 1)

1].

we show in Figure 18.21(a) an 8-state encoder realization in vvhich the nonlinear term D 2 v(OJ (D) o D,y(O) (D) can be obtained directly from the feedback shift register that forms v(Ol(D); that no additional states are needed in this case. (The separate 2-state differential encoder for the information sequence is also shown in Figure 18.21(a).) This nonlinear encoder results in a 90° rotationally invariant code, since Iln{OJ (D) = + D + 1 has an odd number of terms, and thus the rotated code sequences (18.52) (see Problem 18.22).

1004 u(D)

Chapter 18

Trellis-Coded Modulation

-0>--------oal-------<>--------------1> vlll(D)

L[J~J (a)

(b)

Realization of a rotationally invariant, 8-state, rate R = 1/2 QPSK encoder (a) with separate differential encoding and (b) with embedded differential encoding. FIGURE 18.21:

The following comments apply to Example 18.11: 0

The nonlinear PCE represented by (18.52) can always be realized using v' = max [v. 2(b - a)] delay elements [26]. In this example, v' = v = 3 (see Problem 18.23 for an example that includes the case v < 2(b - a)).

e More classes of rotationally invariant codes can be found by dropping the condition of (18.49) and merely requiring that (18.46) be satisfied; however, in this more general case, it is not as easy to determine the conditions under which additional states are not required in the encoder realization. o

Natural mapping is assumed and (18.44) is used as the PCE because these choices result in simple conditions on the parity-check polynomials to guarantee rotational invariance and because if JmUl (D) is chosen as in (18.49), the PCE contains only one nonlinear term. Additional classes of 90° rotationally invariant codes can be found if other mappings are assumed or different PCEs are used, but the code specification and realization is, in general, more complex.

For a given v, a large family of 90° rotationally invariant rate R = 1/2 nonlinear codes for naturally mapped QPSK are defined by (18.52). A computer search can be used to select the parameters a and b and the coefficients of fuiCO) (D) that satisfy the conditions of (18.46) and (18.49), maximize cl},ee' and minimize Ac11;,,· (The search technique cannot use the method of Euclidean weights to find cl},.ee in this case, since the code is nonlinear: that is, all pairs of trellis p::iths mnst he rnmpared.) A list of the best 90° rotationally invariant rate R = 1/2 nonlinear codes for naturally

Invariant TCIVI

Section 18.4

i((l)i[!5

y {iilllil,}

3.98 0.5 1.0 1.875 0.25 0.312 0.75

1

20 20

2.c! 2~

found for constraint coefficients in octal form.

1

20

0, /

1

=

in Table 18.7. 0. Lare

and Adfiec for the best 180° and 360° are also listed for along y of the best nonlinear code. (The codes for 360° free distance codes found in Table 12.1 used with as the value of helve been normalized to unit energy. The for v = 6 means ihat the best 360° invariant code is identical to the best 180° invariant The vcJlue of y is with reference to an uncoded vvith the same that has Amin = 1. Note in than the best linear so th_ere ils soine to achieve full rotational invari,mce. On the other the vc1lues of ,1c1/icc for the linear of the nonlinear codes at moderate BERs. which lessens the values of Ac11;"" are, in fractional in the nonlinear case. since not all codewords have a nearest at distance For the best 8-state, 90° invariant nonlinear code code in 18.11) has y = 3.98 which is 0.79 dB less than the best 8-state, 180° im1rn-iant linear the nonlinear code has one--fourth the number of nearest of the linear code. In 18.22 we the simulated BER of the nonlinear code is the nonlinear 90° invariant codes described in this section guarantees that rotated code sequences are still valid through the trellis. For the a small loss in decoded BER), even if the receiver has lost and locked onto the wrong differential encoding and the through the one time unit circuit + 1) to convolutional and the decoded sequence 1fo(D) must be processed For some invariant it is the inverse circuit (D + 1) after 3S

in Table 18.6. The values of

11 2 11 2

4.77 5A4 6.02 6.02 6.53 6.99

1006

Chapter 18

Trellis-Coded Modulation

Linear. 180° Nonlinear, 90°

0.1

''

0.01

'

\

"'

"" \

G?

~

'

\ \

'

0.001

\ \ \ \

'

\ \

\ \ \ \

0.0001

\ \ \ \ \

\ \ \ \ \

le-05

le-06'--~_..c.._~~~-~~~~~-'-~~-'--~~ 2

3

4

5

6

EJN11 (dB)

FIGURE 18.22:

Performance curves for two 8-state, rate R

= 1/2 QPSK codes.

possible to embed the differential encoding within the convolutional encoder, thus eliminating the need for separate differential encoding and decoding circuits. For the 90° rotationally invariant code of Example 18.11, an encoder realization that includes embedded differential encoding is shown in Figure 18.21 (b ). Any QAM signal constellation has exactly the same four rotational symmetries as the QPSK signal set. Thus, the same approach used for rate R = 1/2 coded QPSK can be used to construct 90° rotationally invariant codes for rate R = k/(k + 1) coded 2k+ 1-ary Qil,J\1 constellations, as long as the tvvo least significant label bits are assigned using natural mapping. In this case only the first information sequence vC 1\D) and the parity sequence vC 0 \D) are affected by a rotation, so the check polynomials lhP\D), ... , n-i.
Section 18.4

cocled

v=.rith

Rotationally Invariant TCl\11

iirNQl7

k coded inforn11ation bits as (D)v 12 l(D) EB

EB· · EB

EB

(D).

TCM code

+

(j)

-J

,,here v > bi 2:: oi > 0 fm 2 :'S j :S k. Further, b'

b).

== n1ax

0

1

= max

vve can ;tealize the ncnJine8r PCl~ - o + 1/ - c/)

using v'

im1awazi11·~ ~zi'\t,:2 iZf.

whl:ih Two Co
= 2)

=

matrix for a rate R = 2/3 convolutional code:

Consider the = [ D / (D

3

+ D + l)

+ D)/(D 3 + D + 1)

vihich t!l([J and TI1\0J (D) are chosen as in Example 18.11 to ensure 90° rotational inv8ric1nce. In this case v = 3. h = 2. o = 1 c1nd b2 = a2 = 1 so b1 = 2, a' = 1 we can use this code with one uncoded bit invariant 8-state encoder realization of a rate R = The encoder realization differential -,,--,.. ,, .. ~

flGURE 18.2.3: Realization of a encoder.

invariant. 8-state. rate R

= 3/4 1

1008

Chapter 18

Trellis-Coded Modulation

TABLE 18.8:

= k/(k + 1) QAM codes. = 2, Amin = 4('.Z::2)

Rotationally invariant rate R rJ

= k bits/symbol, a';:nin

90° lhrnvaria1rnce 180° invaria1rnce 360° JiIDI.Vll!B.'Ralrn
1t!l2)

11/1)

h(O)

V

k

3 4 5 6 7 8

2 02 06 13 2 04 12 23 2 02 14 45 2 020 014 103 2 100 060 205 2 100 210 417

d2

'free

5 5 6 7 7 8

Adjree

16 8 16 80 16 60

d2

'free

5 6 6 7 8 8

Adjree

16 56 8 44 204 28

d2

Adji-ee

'free -

7 8 -

-

40 172 -

y

(dB)

3.98 3.98 4.77 5.44 5.44 6.02

Adapted from [26].

Because (18.54) defines a large family of 90° rotationally invariant rate R = nonlinear codes for naturally mapped QAM signal constellations, a computer search can be used to select the parameters a and b and the coefficients of 11<0) (D) and Jln( 2l (D), · · . , JlnU~l (D) that satisfy the conditions of (18.46) and (18.49), maximize dj,·ee' and minimize Adfiee· A list of the best 90° rotationally invariant rate R = k/(k + 1) nonlinear codes for naturally mapped QAM found for constraint lengths up to v = 8 is presented in Table 18.8, where the uncoded reference system is the (scaled) infinite two-dimensional integer lattice 2 2 with 1in = 2 and Amin = 4. The values of dj,·ee and Adfiee for the best 180° and 360° rotationally invariant rate R = k/(k + 1) linear codes are also listed for comparison, along with the asymptotic coding gain y of the best nonlinear code. In all cases the best nonlinear codes found had only two coded bits, that is, k = 2, and the same codes resulted in the same maximum dj,·ee independent of the size of the signal constellation chosen from the lattice, that is, independent of the number of uncoded bits and the spectral efficiency 17 = k bits/symbol. Because boundary effects cause the (in general, fractional) values of Adfiee to differ depending on k and the size of the signal constellation, only the (integer) values of Adfiec· corresponding to an infinite-size constellation are listed in the table. Note that the best nonlinear codes have smaller values of dj,-ee than the best linear codes in only two cases, namely, v = 4 and v = 7, and that in the other cases the only penalty to be paid for full rotational invariance is a somewhat larger value of Adfiec. In fact, the best 8-state 90° invariant nonlinear code (the code in Example 18.12) has exactly the same parameters as the best 8-state 180° invariant linear code. In Figure 18.24 we plot the simulated BER performance of these two codes with one uncoded bii and 16-QAM, where we see that the 90° invariant nonlinear code is actually slightly better than the 180° invariant linear code (owing to the effect of higherorder terms in the distance spectrum of the two codes). Finally, we note that for larger values of v, more than k = 2 coded bits will be needed to achieve the maximum dT,·ee· k/(k

+ 1)

d;

Section 18.4

Linear. 180° ~~~ Nonlinear. 90° - - - - -

0.1

0.01

0.0001

le-05

le-06 ___L___J_~1- ~ - ~ - ~ - 5 5.5 6 6.5 7 7.5 8

8.5

9

£ 1,/N0 (dB)

rn::iURE 18.24: Performance curves for two

rate R

= 3/4 16-QAM[ codes.

of 45° invariant rate R = 2/3 codes for naturally 8-PSK modulation by following the same approach used for the 90° invariant rate R = 1/2 QPSK case. We begin by the PCE 8)] 2

= il])(D),

(18.58)

where = l!n< 2) (D) + 21ln(l) (D) + 4Jru

0/?1.

lhi(l)

(18.59)

1010

Trellis-Coded Modulation

Chapter 18

so that, for example, if Jm( 2l (D) has two nonzero terms, and Jhi(l) (D) has one nonzero term, Jln(O) (D) must have an odd number of nonzero ter::ns t0 satisfy (18.59) (see Problem 18.24). Now, choosing 1m<2\D) =DC+ Db,

Da,

(18.60b)

+ h(O) Dv-l + · · · + ,/0l D 2 + h(O) D + 1 v-1 2 1 '

(18.60c)

lhi(l) (D)

and Jm 0 (D) = DV

=

(18.60a)

where v > c > b > a > 0, and substituting (18.60a) and (18.60b) into (18.58) we obtain the rate R = 2/3 binary nonlinear PCE (18.61a) where f(D) = Dc"l(ll(D) o [Dhw(ll(D) EB D"w(O\D) EB Dcw<0\D)

EB Dbw(l) (D)

o [ Daw(Ol (D)

EB Dc,,/Ol (D)

0

o d'w(Ol (D)

D"w< 0 \D)]

J (18.61b)

Note that in this case, since JhiC 2l(l) = 2 (mod 8), and Jln<1l(l) = 1 (mod 8), 1m<0l(D) must have an odd number of nonzero terms to satisfy (18.59). Equation (18.61) represents a binary nonlinear PCE that guarantees 45° rotational invariance for rate R = 2/3 codes with !in< 2l(D) and !in< 1\D) defined by (18.60a) and (18.60b), respectively, and any !in<0\D) with an odd number of nonzero terms, where f (D) represents the nonlinear portion of the PCE. (In Problems 18.25 and 18.26 it is shown that the preceding nonlinear PCE is satisfied when the 45° rotated binary code sequences for naturally mapped 8-PSK are substituted into (18.61), and !hJCO) (D) is assumed to have an odd number of nonzero terms.) As in the case of the rate R = 1/2 binary nonlinear PCE used to guarantee 90° rotational invariance for QAM constellations, certain conditions must be satisfied by the rate R = 2/3 binary nonlinear PCE of (18.61) for the encoder to be realized 0 with only v delay elements [26]. Let 11\ l be the lowest-order nonzero coefficient in Jh/Ol (D), that is, lhl(O)(D) = DV + · · · + 0)D 1 + 1. (18.62)

h\

where 1 s t _s v - 1. Then, the following four conditions are required to realize (18.61) with v delay elements: (i) t

=::_

c - b,

(ii) b = 2a.

(iii) c _s 3a.

(iv) v =::_ 2(c - a).

IEXAMPLIE 18. B IR<.otail:o©>i1aliy im1adai1t IR<.ail:e R Mappetdl 3-PSK wuil:h lw©> (©>ded 18011:s (k = 2)

= 2/3

(18.63)

Codes for Nat1.11rn!!y

Consider the following parity-check matrix for a rate R = 2/3 convolutional code: lHI(D)

=

[(D

3

+ D 2 )/(D 4 + D + 1)

D/(D

4

+ D + 1)

1],

(18.64)

and Il1,<0) (D) has an (D) and n.P l in which odd r11u1r1ber of nonzero tern1s lo ensnr~, t.~,5° Iotation8Jl iI1variance. In this case v = 4, c = 3. b the conditions of are all encoder ca11 be i·ealized elements. F;·om (18.61) the PCE is EB

+D

-i-

l "(OJcD) = j

(01] ~v I

J PCE cap_

invariant 16-state encoder !Problem 18.27 we see how f (D)

found for reference and

FIGURE 18.25: Realization of a

encoder.

nonlinear codes for search can be used to select that the conditions .A 8.. PSK the

rate R

=

2/3 8-PSK

WU

Chapter 18

Trellis-Coded Modulation

TABLE 18.9:

;7

Rotationally invariant rate R

= 2 bits/symbol, d,;,in = 2, A 111 ;11

= 2/3 8-PSK codes. = 2 ( QPSK)

45° l!nvall'i!llnce 180° foV!llll'RailllCC 360° lllliVilllrim:J1.ce

ni 2 l

V

k

3 4 5 6 7 8

1

2 2 2 2 2

-

h(l)

06

14 02 14 02 060 004 014 002 120 004

h(O)

d2

'free

13 4.0 23 4.586 43 4.586 127 5.172 23515172 57

Adfi'ee

LO l.O

0.25 0.469 0.012 0.016

d2

'free

4.586 5.172 5.172 6.343 6.343 7.515

Adfi-ee

2.0 4.0 0.25 3.25 0.125 3.375

d2

'free

Adfi'ee

-

-

5.172 5.757 -

6.586 7.515

2.25 2.0 -

0.5 1.5

y (dB)

3.01 3.60 3.60 4.13 4.13 4.59

Adapted from [26].

nonlinear code. For v = 3, the best nonlinear code has k = 1 and one uncoded bit, but for all v ~ 4, the best nonlinear code has k = 2 and no uncoded bits. The nonlinear codes have smaller values of dJee than the best linear codes, indicating that a penalty must be paid for full 45° rotational invariance; however, the nonlinear codes generally have smaller values of Adfi,·e than the best linear codes. For example, the best 16-state 45° invariant nonlinear code (the code in Example 18.13) loses 0.52 dB in asymptotic coding gain compared with the best 16-state 180° invariant linear code. In Figure 18.26 we plot the simulated BER performance of these two codes with naturally mapped 8-PSK modulation, and we see that the 45° invariant nonlinear code is only about 0.15 dB worse than the 360° invariant linear code at a BER of 10- 5 ( owing mostly to the fact that the nonlinear code has a factor of 4 fewer nearest neighbors than the linear code). The following comments apply to Example 18.13: (I

ei

A similar approach to that used for 8-PSK, but using modulo-16 arithmetic over the ring of integers Z16, can be used to find 22.5° rotationally invariant codes for naturally mapped 16-PSK. In this case the best codes up to v = 7 use the rate R = 1/2 invariant PCE with two uncoded bits, but for larger values of v higher-rate PCEs are better [26]. As noted in Section 18.2, this is because for short constraint lengths, the parallel transition distance (8~ 1;,, = 2 in this case) is already larger than the free distance oi;·ee between trellis paths, and thus using a higher-rate PCE cannot improve the overall free distance dT,-ee· As in the QPSK case, additional classes of fully rotationally invariant codes can be found for PSK and QAM constellations if alternative mappings are assumed, different PCEs are used., or other restrictions are placed on the parity-check polynomials, but the code specification and realization is, in general, more complex.

We close this section with an example of the 8-state, rate R = 2/3, nonlinear, 90° invariant code designed by Wei [27, 28] and chosen for the V.32 and V.33 highspeed modem standards. The V.32 standard uses 2 uncoded bits and a 32-CROSS signal constellation for a spectral efficiency of 17 = 4.0 bits/symbol. In the V.33

Invariant TOI!

Section 1

WE:

Linear_ 360° -- --··~ Nonlinear. 45° ~ - - - ~ ·

0.1

0.01

0.001

0.0001 \:-

le-05 -

rlGURE 1

Performance curves for two 16-state. rate R

=

8-PSK codes.

4 uncoded bits and a 128-CROSS constellation are used to achieve 17

= 6.0

of the 8-state . rate R = 2/3, nonlinear, 90° invariant encoder and A block the 32-CROSS constellation used in the V.32 standard is shown in 18.27. that the 32-CROSS constellation is not naturally since a 90° rotation does not alter the two least significant label bits in the same way The encoder has four information bits. uUJ. and u( 4 J = v( 4 J are uncoded and Bits u 0 ) and are first rate R = 2/3 feedback nonlinear convolution21l encoder. the three output bits v(l) and v( 2 l and v(OJ The five encoded bits v
~

1014

Chapter 18

Trellis-Coded Modulation

M(4) - - - - - - - - - - - - - - - - - - - - - - V ( 4 )

iPJ - - - - - - - - - - - - - - - - - - - - - - v l 3 1

uit21-

t--'<~-----------------vi2l

Differential encoder

32-CROSS

1
y

_ signal mapper

Nonlinear convolutional encoder

"

"

0

®

"

©

"

00000 10010 01000 11111 ©

"

@

0

"

00111 11001 01111 10101 00101 11000

.,

" 11110 00010 10011 01010 10000 01001

"

"

"' 01110 10111 00110 11010 01101 10100 0

"

©

"

©

"' 11100 00001 10001 01011 11101 00011 <J)

©

"

0

"

©

"

"

"

11011 01100 10110 00100

Signal constellation

FiGURE 18.27:

The V.32 TCM system encoder and signal constellation.

to the mapping shown in Figure 18.27. Because one 32-CROSS signal is transmitted for every four information bits entering the encoder, the spectral efficiency of the code is 17 = 4.0 bits/symbol. (The V.33 standard uses the same code along with four uncoded information bits and a 128-CROSS constellation to achieve a spectral efficiency of 17 = 6.0 bits/symbol.) At the receiver, soft-decision Viterbi decoding, using an 8-state trellis with 4-fold (16-fold in the V.33 case) parallel transitions, is performed based on the noisy received symbols at the demodulator output. After Viterbi decoding, the decoded output bits i/ll and u< 2) are differentially decoded. The 8-state, rate R = 2/3 nonlinear encoder used in the V.32 and V.33 standards was designed completely by hand. without benefit of a systematic code search [27, 28]. His invariant to 90° phase rotations of the 32-CROSS constellation

Section 18.5 S

=

W'TIS

32-CROSS

00000000

o o O O O O o o -----o

1\/lultidimensional TCM

z"]_ lattice

O O O O O O o

o O O O O O O o o O O O O O O o o O O O O O O o 00000000

v'"I

=

0 Q(l)

Q(O)

000000 ,}OCOOOA~-

000000~1-

V(IJ =

000000

000000

000000

000000

oooooc

000000

.~~

: : : : : :

=

000000

000000

000000

000000

000000

000000

000000000000

000000

000000

000000

000000

000000

000000

000000000000

0

000000

000000

000000

000000

000000

000000000000

000000'

000000

000000

000000

000000

000000

000000000000

000000

(000)

(JOO)

(010)

(110)

(001)

FIGURE 18.28:

Three-level

of the

(101)

(Oll)

0

0

0

0

0

1~ =

(111)

32-CROSS

constellation. has free distance dfi.ee = S and average number of nearest and achieves a real of 3.6 dB at a BEJR (n is due to effects of the m 18.12 and sketched in was using a code search. ¥Vhen this encoder is used with the naturally mapped 32-CROSS constellation (see it one AND gate and one encoded information and differential encoding can be embedded ·within the encoder. (Note, as mentioned in Section that level 3 in the partition tree is an of a case in which not all subsets at the same level are isomorphic.) For two-dimensional signal since 1t 1s to achieve 90° invariance with linear codes (the best that can be done is 180° invariance), nonlinear codes are needed for full rotational invariance. This was the crucial nrnde by Wei [28] in the design of the V.32 code.

to this point in our discussion of TCM we have considered the case in which the (k + 1) convolutional encoder output bits at each time unit are into one

1016

Chapter 18

Trel Iis-Coded IVlodu lation

signal point drawn from a 21c+ 1-ary constellation described in one-dimensional (1-D) or tvvo-dimensiona! (2 .. D) Euclidean space. ln this section vie consider the more general case of designing TCM systems for use with multidimensional (multi-D) signal sets, pioneered by Wei [29]. Although it is possible to construct multi-D signal sets directly in multidimensional Euclidean space, we consider here only the more practical case of constructing L-dimensional or 2L-dimensional signal sets as the L-fold Cartesian product of a 1-D or 2-D signal set. First, let S = {so. s1. · · · , sM-d be a 1-D or 2-D signal set with M = 21 signal points. Then, the L-fold Cartesian product 5L

=

S x S x · · · x S (L times)

(18.66)

represents the L-dimensional or 2L-dimensional signal set consisting of all possible combinations of L signal points drawn from S, and the multi-D signal set sL contains a total of 2 1 L signal points. Transmitting a signal from the multi-D signal set sL is then equivalent to transmitting L signals from the constituent 1-D or 2-D signal set S, and the multi-D signal Y1 transmitted at time unit l is represented as the L-tuple Y1 = (yn, Yl2· · · ·, YlL), where Yli, 1 :Si :SL, represents a signal from the constituent signal set S. For example, 3 x 8-PSK denotes the six-dimensional (6-D) signal set consisting of all 83 = 512 possible combinations of three 8-PSK signals, and the 3-tuple Y1 = (Y11, Y/2, Y/3) represents the 3 x 8-PSK signal transmitted at time unit l. In a TCM system design using a multi-D signal set, the (k + 1) convolutional encoder output bits at each time unit are mapped into a signal point drawn from the multi-D 2 1 L-ary signal constellation sL, so that k + 1 = IL. Because each time unit now corresponds to the transmission of L signals from the elementary signal set S, each branch of the trellis representing the encoder/modulator is labeled with L signals from S. In decoding, the soft Euclidean metrics for each of the L signal labels on a branch are added to form the branch metrics. Multi-D TCM systems have several advantages compared with 1-D and 2-D TCM systems: L Fractional values of spectral efficiency can be realized. 2o Full rotational invariance can be achieved with linear codes. :t Additional povver savings can be realized using a technique called shaping. 4o The signal sets have a smaller peak-to-average power ratio. 5o Higher-speed decoding can be achieved.

The first two advantages are explained in detail later in this section, and advantages 3 and 4 are illustrated in an example discussing the V.34 high-speed modem standard at the end of the section. The higher decoding speed stems from the fact that the metric calculations on a trellis branch are normally done with quantized lookup tables. Thus, for multi-D TCM with L signals on each branch, as opposed to 1-D or 2-D TCM with only one signal per branch, performing the branch metric calculations in parallel results in a decoding speed advantage equal to a factor of L. To better exphin thP h,:isics nf nrnlti-n Tl'M rlPsign, we !!OW intrnrlnrP an example.

Multidirnensional TCM

10'17

linear convolutional encoder In this we consider an with 3 uncoded infon:rratim, bits. that a rate R = k/(k + 1) = encoder set. The multi-D into a four-dirnensional encodes 5 information bits per unit time. Because two 8-PSK. signals are trans1nitted per unit tirr1e, the JS 17

= set set Because this tree will the MSSDs

the

8-PSK

S >< S, 'Nhere S denotes H1e coEstituent

k+l=6 level p in the tree. is illustrated in

(0 )( 0 U 1 )(

contains all the 2-D subset all ~~acnca.HHHf',

from the 2-D subset Q(O) and all from x 00) the 4-D subset Q(OO) >< Q(OO) from the 2-D subset Q(OO)

Q2 (00

tree are also indicated vve down the set partitioning. To see how each 4-D signal set; for example. ')'{ = (0. 0) to two uses of the 8-PSK labeled 0. A.t level p = 0 we see that there are signal pairs such as Y = 0) and 1{' = (0. 1) for which = 0 + 0.586 = 0.586, so that = 0.586. A.t level p = 1 we see that there are pairs such as 1{ = (0. 0) and Y' = (1, 7), both belonging to subset Q2 (0 >< 0 U 1 >< 1), for which d10Y. Y') == 0.586 + 0.586 = so that L\I = 1.172. At level p = 2 we such as 1{ = (0, 0) and 1!1 = (0, both belonging to see that there are subset Q2 (0 x 0), for which = 0 + 2.0 = so that = 2.0. At partition level p = 3 we see that there are signal pairs such as Y = (0. 0) and Y' = (2. 6), both belonging to subset Q2 (00 x 00 U 10 x for which (Y Y') = 2.0 + 2.0 = 4.0, so that ,!\~ = and so on. From the set-partitioning encoder shown in Figure 18.14 we see that at each time the three output bits of a rate R = encoder select one of the level p = 3, and then the three uncoded bits select one of the 4-D these 6 bits determine the two 8-PSK signals to be transmitted in a given time unit. The from these 6 bits to a of 8-PSK signals that the distances

di

i'\5

nr. ';'{')

4

rn·rn

Chapter 18

Trellis-Coded IVlodulation S = 8-PSK 2

3.,

'

/

I

'G

I

\

\

I

4

\

\

0

I

\

I

\

5 ",

,,/ 7 '

6

S2 =

00 01 02 OJ 04 05 06 07 10 11 12 l.l 14 15 16 17 20 2 i 22 n 24 25 26 n 30 3 l 32 33 34 35 36 :,7 40 41424.1-1445 4647 50 5 l 52 53 54 55 56 57

S XS

flf =

o.s86

11f

l.172

60 616263 6t.J 65 (16 67

711 71 72 73 74 75 76 77 v(OJ~

~

00 02 04 06 1.1151711

0 I O.l 05 07 14161012

n

26 20 24 3\JJ_1,)J7

Q2(0 XO U J X l)

2121 2:1

0- 2(0

.44 '16 40 42 5751)355 62 (14 66 60

X

l

uJ

X

0)

~

00 02 04 06

Q2(0

X

26 20 12 24 44 46 40 42

0)

Q'(l X 1)

o/ \t (12 64 66 60

v1'1 =

OU 04 26 22 4440 62 66 C-___A_______

v(l)

o/

=

yi4) =

o

I

X

\1

01 11.10507 27212325 4'.i 47414.1 (iJ

Q2 (1 X 0)

65 67 61

o/ \

1.1 17

01 0.\

03 117

JI 35 57 )3 7.\71

nn

21 25

45 41 6.167

47 4.1 6561

fl~= 2.0

74 76 70 72

o/ \1 12 16 .111 .14

14 Ill 32 36

56 52

)0 54 7672

7470

fl~= 4.0

~~~~

Q'(Ol X 01 U 11 X 11) Q"(OO X 01 U 10 X 11) Q 2(01 X 10 U 11 X 00)

X

00)

~1 fl~= 8.0

04 40

000 U LOO X 100)

1111

12 14 16 10 30 32 .l4 .l6 56 50 52 54

26 22 62 66

011 44

Q2(000

Q 2 (0 X 1)

~l

00 04 44 40

Q2(00

111'.i J7 JJ 55 5 l 7.177

~

Q'(OO X 00 U 10 X 10)

11 1.1 15 17 .17 .11 3.l .1.\ 55 57 5 ! 53 7.1757771

o/

02 Iii, 20 24 4642 64 611

=

)() 52 54 56 63656761 7h 711 72 74

75777173

yil)=1,,/

2s

.l2 .l4 .16 .lll 4547-114.1

44

~

04

40

Q2(ooo x oooJ FIGURE 18.29: Partitioning tree for 2 x 8-PSK.

Section '18.5

1\/lultidimensional TCM

W'u9i

the subset

(0 >< 1 U 1 >< 0) at (0 >< 0 U 1 X 1) caHed a coset generator, to each member of the subset (0 >< 0 U 1 x the coset as 1:¥0 = (0, 1) and integer notation for the within each subset vve can the coset (0

LJ

X

X

(0 >< 0 U 1

0) =

X

(0 )( 0 U l ><

+ 1/;0 1) + (0, 1)

1)

8),

·01heTe the addition is modul.o-8. At level p = 2, we form the subsets (1 x 1), (1 x 0) as cosets of the ~ubset Q2 >< 0) by ~u,J~'""'" 2 second ccset gJ E (0 >< 0 U 1 x 1). 1:¥1 ¢:. (0 x 0) ). Fm the choice 1:¥l = the subsets at level 2 as

+ 1:¥1 = (0 >< 0) + (L 1) (0 )( 0) + 1) (0 )( 0) + 1/;0 = 2 (0 >< 0) + ~1 + 11;0 = Q (0 x 0) + (L 2)

(.1><1)==

(0 )( 0)

(0 )( 1) =

(1

X

0)

=

8)

(18.69a)

8) 8).

level p = 3 . we choose a coset generator 1:¥2 (1:¥2 E Q 2 (0 >< 0) . ,~2 y". to the subsets. For the choice 1:¥2 = the subsets at level 3 are formed as ><10U10><

X

(00 )( 00 u )( 01 U 11 ><

(OJ

X

)=

(00

JJ U J ><

(00

X

01 U 10

X

X

00 U

X

00 LJ

(00 >< 00 U

(00

X

00 U

(00 )( 00 u

J1) =

(00 >< 00 2

Q (00

X

2

X

00 lJ

=

Q 2 (00

X

00 U

00) =

(00

X

00 lJ

X

(00 x 00

(01

X

u

11 lJ 10 >< 01) = Q (00

(01 >< 10 lJ 11

QQ lJ 11 >< JO)

= =

u

(00 X 00 U 2

+ 1:¥2 10 )( 10) + (0. 2) 8) 10 >< 10) + \\l;l 8) 10 >< 10) + (L 1) 10 X 10) + 11;2 + 1/;l 8) 10 X 10) + (i. 3) 10 >< 10) + 1:¥0 8) 10 >< 10) + (0.1) 10 >< 10) + ,!l;2 + ,!l;O 8) 10 >< 10) + (0. 3) 10 X 10) + ,!l;l + ,!l;O 10 x 10) + (1. 2) (mod 8) 10 X 10) + ,!l;2 + 1i;l + 11:0

00 LJ 10 )( 10)

Q (00 x 00 U 10 >< 10) + (1, 4) (mod 8).

(18.

(18.70c)

(18.70e)

(18.70f)

(18.70g)

The subsets al the three !e-vels of the partition tree can be formed in a similar fashion. For the choices of coset generators 1/;3 = (2. 2), 1:¥4 = (0 4), and

11()20

Chapter 18

Trellis-Coded Modulation

gs= (4, 4), the mapping from the binary code vector v = (v( 5lvC 4lv(3lv( 2lvCllv(O)) to the 8-ary signal vector Y == (Yi, y2) can be given as follovvs:

L

Y = (y1, Y2) =

v
0:Sp:SS

= v< 51( 4, 4) + v< 4\0, 4) + v(3) (2, 2) + vC2\0, 2) + v< 11(1, 1) + vC0\0, 1) (mod 8),

(18.71)

where v< 5l, vC 4l, and v< 3l represent the three uncoded bits, and v< 2l, vC 1l, and vCO) represent the three encoder output bits. For example, the code vector v =(111010) generates the signal vector Y = (4, 4) + (0, 4) + (2, 2) + (1, 1) = (7, 3). If the coset generators are given in binary form as 3-bit column vectors

g,~u ngi~u n n g,~u n ng,~u n ~~u

(18.72)

~~u

then the mapping from the binary code vector v to the binary signal vector 1{

J

= (w.11,.12 w ) = [ (/2l/lly
E

(18.73)

v
0:Sp:SS

and for v = (111010), we see that Y = [(lll)T, (Olll], the binary equivalent of Y = (7, 3). The mapping function represented by (18.73) is shown in Figure 18.30. Next we must choose the 8-state, rate R = 2/3 encoder to maximize the overall MFSE distance dj,·ee· From (18.18) we see that dj,·ee is the minimum of "1,·ee' thP minim11m rlidcinrP hetween trp]]i~ paths ,:in.I A;llll' the miniffillffi rlishnrP between parallel transitions. The value of 1in can be determined directly from the partitioning tree as the MSSD at partition level p = 3, so that o;,in = ~~ = 4.0. Thus, the encoder must be chosen to make oJ,.ee as large as possible. The best 8-state, rate R = 2/3 encoder has parity-check polynomials (in octal form) 1ii.<2l = (0 4), !Ji(I) = (0 6), and h_(O) = (11), and oJ,·ee = 7..97.9, so that dJ,.ee = min{?979, 4.0} = 2.929. To determine the asymptotic coding gain y in this case, we must compare

0;

Section 18.5

(2)

YI

Multidimensional TCM

(I)

(])

Y2

YI

FIGURE 18.30: A 2 >< 8-PSK

Ul21

(0)

Y2

set mapper.

with an uncoded system ·with the same efficiency as the coded system, that is, 17 = 2.5 For this purpose we consider the 2 x 8-PSK subset Q2 (0 x O U 1 >< 1) of 32 which has 17 = 2.5 bits/symbol and d,~,in = 1.172. Thus, y = l0log10(dj,ee/d1~,in) = 10log10 (2.929/1.172) = 3.98 dB. The overall encoder block for this 4-D TCM is shown in Figure 18.31. Note that the encoder in this case are for a 2T-second time interval, that is, the time to transmit two 8-PSK ~"h'""'V·

J ,] o

Unlike 2-D signal sets, multi-D spectral efficiencies.

o

The choices encoder output bits to signal itself.

o

The overall MFSE distance can be increased in two ways, either by using more encoder states, thereby increasing 31,-ee' and/or by encoding more bits, that using a larger k, increasing 3~,in (see Problem 18.28).

determine the mapping from do not affect the partitioning

We now generalize the preceding example into a design procedure for multi-D TCM. The major steps in the design procedure are as follows:

1022

Chapter 18

Trellis-Coded Modulation ,P1(D) v<-llrn\

I

u<-11(D)

-----------------+-----'-~-';----,

ulol(D)

v i(D) ------------------a---0------1--1--.

13

v121(D) u(2l(D) - - - - - - - < ; ; - - - - - - - - - - - - - + - - + - - - - - - 1 - - 1 - - 1 - - - ,

FIGURE 18.31: Encoder block diagram for 8-state, rate R

2/3 trellis-coded

2 x 8-PSK. lo Partition the multi-D signal space sL such that the MSSD each partition level p = 1, 2, · · · , k.

1',.t is maximized at

k/(k + 1), 1 S k S k, convolutional encoder to maximize o}ee' the minimum distance between trellis paths, and minimize Ac11,ee' the number of nearest-neighbor codewords, for some given number of states.

2o Select a rate

The partitioning of the multi-D signal set sL uses the multilevel coding technique first introduced by Imai and Hirakawa [30] and developed further in [31]. We begin by representing each of the 21 signal points in the constituent 2-D signal set S by a binary vector y = (yU-lJ, ... , y< 1l, y< 0l), where yUl determines the subset selected at level j + 1 in the complete binary partitioning tree for S, and by letting s · · · s 07 = oo. (At the MSSDs in the partitioning tree be represented by 05 s level I in the complete tree, there is only one signal point in each subset, so 87 = oo.) Then, we can represent the 21 L multi-D signal points Y in sL as an I x L binary matrix

or

Y=(Y1,Y2,

i

~l where each column

y;

=

(1-1)

Y1

(1-1)

Y2

(1)

Y1

(1)

Y2

(0)

Y1

···.yL)

(0)

Y2

(y,U-1). · · · . y,
representation of one of the

21

(l-1)

YL

(1)

h h

(0)

l

J

(18.74)

1 ::: i ::: L, of Y is the binary

signal points in S. (Alternatively, we can represent

Multidim~11sio11al TCM

Section rn.5

each

~(Q):B

column

}i

Yi= Each rnw of -i is a (L. L. 1) block code Co. 11, dimension k, and mininrnm multi-D set as the I -level block code Ao=

*Co*···*

codewords dravvn where Ao :frorn Co, m,d ·Ir a code in the chain to the bottom row of determines of codes tha! constitute l\.o the L bits that select the level-1 subsets in the L trees for S. vvherens to the cletern,ines the bits a sequr:nce of linear subcodes of

To dPpn,Pd

C CL-i C · · · C

C

and the dimension of subcode Ci is ) =

Tl)

L -- i. 0 Si SL.

. 0. co) cod,:; CL contains the alJ.-ze:c"O codeword.) H di the minimum distance of subcode , ihen (18.

where do = 1, and ch = o,J. YNe will see at each level p in the mu!ti-D subcode to maximize its mmimum To divide the multilevel code !\o set ) into 21 L subsets. each a single tree 1Nith l L levels. we use the code sequence tCo. !C1. · · · . to form a sequence of multilevel subcodes of i\o, denoted i\o(O). A 1(0). · · · . i\n (0). such that i\n (0) C i\IL-1 (0) C .. · C i\ 1(0) C Ao(O).

and the dimension of multilevel subcode dirn [i\i(O)]

(0) is

= IL -

by

i. 0 Si SIL.

(18.80)

where Ao(O) = J\o and An (0) contains l of the all--zero codeword. Our goal in designing this IL-level tree is to maximize the MSSDs at each level. Before with the design of a multi-D tree, we note that the SE distance between two rnulti-D signal the sum of the SE distances between the 2-D signal

'\' ,2 L.., CE l:<:i:<:L

. :/).

(18.81)

1024

Chapter 18

Trellis-Coded Modulation

~6 05,

For the entire multi-D signal set sL, the MSE distance is given by = the MSE distance of the constituent 2-D signai set S. This follows from the fa<.:t that tv;o distinct multi-D signals, each represented by a vector of L signal points, may differ in only one component, in which case (18.81) contains only one nonzero term. Because the MSE distance in the constituent 2-D signal set is 85, we obtain = 85. In terms of the multilevel code representation Ao(O), we note that two distinct multilevel codewords, each represented by a set of I binary codewords of length L from the code Co, may differ in only one codeword. If the two multilevel codewords differ only in the jth codeword, then the MSE distance is do87. Minimizing over all j, 0 :s j :s I - 1, we can represent the minimum distance between two multi-D signal points as '(d2 ) 2 (18.82) ~o2 = mm 080 , do8 12, · · · , d 082 1 _ 1 = 80 ,

~6

since do= 1, and 85 :Sor :S · · · :S 8;_ 1 . Now, to partition Ao(O), of size 21 L, into two subsets of size 21L-l, namely, the subcode A1 (0) and its coset A 1(1), we must replace one of the codes Co of size 2L in the /-level code Co* Co*···* Co with the code C1 of size 2L-l. Further, we should do this in such a way that the MSSD at level 1 in the multi-D partitioning tree is maximized. If the code C1 is inserted at the jth level, the multilevel subcode A1 (0) is given by Co * · · · * Co * C1 * Co * · · · * Co, and an argument similar to that used to obtain (18.82) gives

~I

~I= min ( do85. · · · , do87_ 1, d187- do8]+l • · · · • do8;_ 1);

(18.83)

that is, if two multilevel codewords in A1 (0) differ only in the jth codeword, then the MSSD is d187- Assuming that d1 >do= 1, and > 85, we see from (18.83) that = 85' unless C1 is inserted at level j = 0, in which case = min(d185' or) > 85. In other words, C1 must partition the initial level of the multilevel code Ao(O), corresponding to the initial level of partitioning of the constituent 2-D signal sets, to maximize The two subsets at level I of the partition tree are now given by the subcode A1 (0) = C1 *Co* ···*Co and its coset

or

~I

~I

~I-

(18.84) where the coset generator go is a multilevel codeword not belonging to the subcode A1 (0); that is, go E Ao(O), go rfc A1 (0). Here we note that (18.84) corresponds to (18.68) in Example 18.15, with the suhcode A1 (0) corresponding to the subset Q2(0 x Ou 1 x 1) and the coset A 1(1) corresponding to the coset Q 2(0 x 1 u 1 x 0). (The particular choice of go does not affect the formation of the coset A1 (1).) Further, since the lowest-order bit v< 0l in the binary code vector w selects the first branch in ihe pariiiion iree, we can represeni rhe two subsets at level 1 as (18.85) Next, to partition the subcode A 1 (0) and its coset A1 (1) each of size 2 1L-l, into four subsets of size 21 L- 2 , namely, the subcode A2(0) and its cosets A2(l), A2(2), and !t 2(3),

v.;e,

n1ust replace one of the codes C 1 of size

zL-i

A1 (0) = C1 *Co*···* Co with the code C;+1 of size

in the multilevel subcodc

2L-i-l _ Again,

we should do

Section 18.5

this in such a way that tbe MSSD maximized. In this case, tvvo choices

*

1\/lultidimensional TCM

11Dl25i

at level 2 in the multi-D the subcode (0) are u'v00,.v,,c,.

* ... *Co,

and (0) =

* ... *Co,:,

*

*Co* ...

* Co.

To exa:n1h1e the effect of these tvvo choices for the subcode lo,N1:;r bound on the MSSDs level p. at level p is ated

(0) on

YNe

state a

Assume that the rrmHilevel subcode Ao(O) at = C 17o * Cp I * · · · * Cu• /-1 and that the assoc1distances of each code are level p is

Ao(O)

(18.87) The

to Theorem 18.1: of the the bound holds with

a lower bound on 11y;. sets of interest.

0

can be shown Problem that the level p equals the sum of the redundancies of the I linear block codes that define the subcocle AP that (18.88) p= Pi·

o It

0"':cj"':ci-1

o

From we see that to maxirnize the MSSDs p., each subcode should be chosen to maximize distance d;. VI/ e can now use

at each

to determine for the two choices for the subcode For choice (a) we see that = min(d205. Of). For choice = min(d105. Of, c/10J2) = Ci is inserted at level j, we obtain . ) 2 L in which case 11~ = rnm(d105. 81 ) = liI, unless C1 is inserted at level j = rnin(d105, o~) ::::: 111for choice it is best min(d185, d18r, to insert C1 at j = 1. that to choose i\.2(0) = C1 * C1 * Co * · · · * Co, to maxirnize \J\/hether choice (a) or choice (b) gives the largest value of ~~ values of c/1. d2. and Choice implies that the first t,No the multi-D signal set involve at level 1 of the constituent 1-D or 2-D set, whereas choice (b) implies that the second partition in the multi-D signal set involves a at level 2 of the constituent signal set.

05. of.

or

1026

Chapter 18

Trellis-Coded Modulation

Once the subcode A2(0) has been chosen, the four subsets at level 2 of the hy thP ~111"hr~rlP j\ 2 (Q) 3nrl lt~ rn~Pt(;

p~rtltlrun rh~ln 01rP glVPP

A2(2) A2(l) A2(3)

= A2(0) + 1,';1, = A2(0) + go, = A2(0) + g1 + go,

(18.89a) (18.89b) (18.89c)

where the coset generator g1 is a multilevel codeword belonging to the subcode A1(0) but not belonging to the subcode A2(0); that is, go E A1(0), go r/. A2(0). Here, as before, we note the analogy between the general level-2 coset generation of (18.89) and the specific, for 2 x 8-PSK, level-2 coset generation of (18.69) in Example 18.15. (Again, the particular choice of g1 does not affect the formation of the cosets of A2 (0).) Further, since the two lowest-order bits v
87,

Ap(2(p-l)v(p-l)

+ · · · + 2v(l) + v< 01 ) = Ap(O) + v
where (v
+ ... + 2v(l) + v
+ vUL-l)gIL-1 + ... + v(l)g1 + v
(18.92)

where the encoder output vector v = (vUL-1) .... , v<[). v< 0 )) = (v
E

1/Plgp,

(18.93)

O<:p<:J L-1

( Tf the coset gener::itors (g\~-IJ' · · ·,

g;Y. g;~\

1

gr = (gr 1 , gp2 · · · , grd. 0 :S p :S

s

i

:s

TI. - 1 . where

gpi

L, are expressed in integer form, that is, !l;p

Multidimensional TCM

Section 185

using modulo-2 1 addition we see that (18.93), and its form for generating multi-D JL,A
(gpl, gp2, 'gpL), gp; = y = (y1. Y2 • , , , . YL) integer form noted signals 0

0

rn21

0

For the 2 >< 8-PSK set. the },ASSDs of the constituent 8-PSK signal set are = 0,586 :S = 2,0 :s 8~ = 4,0. we choose the sequence of linear codes
of

05

o

Co is the

2. 1) code that consists of the codewords (00), (01), (10), and (11),

o
is the (2, 1, 2) code that consists of the codev,rords (00) and (11).

0

is the (2. 0. oo) code that consists

of the codeword (00).

t

The choices of multilevel subcodes that result in the best !'l at each partition level p. along with the coset and the values of calculated are listed in Table 18.10, Note that in this case is maximized at using each level p if the first two levels of the multi-D partition involve partitions at level 1 of the constituent 8-PSK set, the next two levels of the multi-D involve at level 2 of the 8-PSK set, and the last two levels of the multi-D at level 3 of the 8-PSK signal set. The to the muHi-D binary signal vector

t1t

t1t

(18.94)

TABLE

lP'm11'1lifo»Illl Ilewell

18,10: The 2 >< 8-PSK

set partition.

M[§§[J)

§1rnlb>l\:1Dirrlle

(Cm,etl 11;ellllern1lm

(all)

p

!l;p

0

lCo *Co* Co

rnin(4, 2, 0.586)

= 0.586

(0, 1)

1

Co* Co* lC1

2, 1.172)

= 1.172

(1, 1)

Co*
2. oo)

= 2.0

(0, 2)

* IC1 * + Co* C2 * C2 + C1 * C2 * C2 + C2 *
min(4, 4, oo)

= 4.0

(2, 2)

oo. oo)

= 4.0

(0, 4)

00,

= 8.0

(4, 4)

+ ,f

2 3 4

5

6 Adapted from [33].

+


oo, oo)

=

oo

1028

Chapter 18

Trellis-Coded Modulation

or Y = (y1, y2) in integer form, corresponding to a particular binary encoder output vector w = (v< 5lv( 4lv(3lv< 2lv< 1lv(OJ), was illustrated in the previous example. For larger multi-D signal constellations, there may be several sequences of L subcodes of Co that can be used to partition the multilevel code Ao, none of which maximizes ~ at every partition level. Which partition is best in a particular case will depend on the rate R = k/(k + 1) of the convolutional encoder that determines the partition level and thus 1in· An example for the 3 x 8-PSK multi-D signal set illustrates this point.

t

0;

EXAMPLE 18.17

85

or

As for the 2 X 8-PSK signal set, for 3 X 8-PSK we again have = 0.586 :::'. = 2.0 ::: 8~ = 4.0. In this case we consider two options for the sequence of linear codes, C3 c q c Cf c Co or C3 c C~ c c Co, defined as follows:

ct

e Co ®

q

is the (3, 3, 1) code consisting of all eight binary 3-tuples. is the (3, 2, 2) code consisting of the codewords (000), (011). (101). and

(110). ®

ct is the (3. 2, 1) code consisting of the codewords (000), (011), (111), and (100).

®

q

®

C~ is the (3, 1, 3) code consisting of the codewords (000) and (111).

®

C3 is the (3, 0, oo) code consisting only of the codeword (000).

is the (3, 1, 2) code consisting of the codewords (000) and (011).

We see that code sequence C3 C C2 C C1 C Co achieves a better minimum distance at the first partition level but that the sequence C3 c C~ c c Co achieves a better distance at the second level. Either of these sequences alone, or combinations of the two sequences, can be used to partition the 3 x 8-PSK signal set. Three partitions of 3 x 8-PSK, corresponding to three distinct combinations of the foregoing two code sequences, along with the corresponding coset generators (in integer form) and values of calculated using (18.87). are listed in Table 18.11. Partition I uses only the code sequence C3 c q c q c Co for partitioning, whereas partitions H and HI use combinations of the two code sequences. Note that partition I is best at partition level p = 1, n and HI are best at p = 2. I and n are best at p = 4, and HI is best at p = 6. Thus, for example, if the code rate is R = 3/4. either partition I or partition II should be selected, whereas partition HI is the best choice for a code rate of R = 5/6. (It should be noted that although different code sequences can be used to partition different levels of a constituent signal set, it is not allowable to "jump back and forth" between two code sequences in partitioning a given level of the constituent signal set; that is, only a proper subcode sequence can be used to partition a given level.) Finally, using the coset generators from Table 18.11 in (18.93), we can determine the signal set mappings corresponding to the three partitions of 3 x 8-PSK (see Problem 18.30).

ct

~t

Section '18 5

Multidimensional TCl\ll

'rn,2s3

1: Three 3 >< 8-PSK signal set partitions.

TABLE 1

~,1]Jlll]<[email protected]

M§§ICD

Ap(®)

= 0.586

0 l

Co* Co*

2

2. L

=

1.172

7, l. i

=

1.172

3

*

2, 00) = 2.0

4


,too)=Li,O

1J)

= 4.0

5

6

00,

co) ~-= 4.0

= 8.0 8

= 8.0

00, C'O) 00. 0())

=

00

}Y

= 0.586

(O,OJ)

2.

= 0.586

(OJ

2. L

= 1.757

0 1 2

Co* :Co*

= 2.0

3

2.

4

4. oo) = 4.0

5

4,

6

* Cj * C3 * C3

Co* C3

co.

00)

00)

= 4.0

(0,2,2)

4.0

(4.4A)

= 8.0

(4,4,0)

00) =

f

7

min(tl, oo. oo)

t 0

0

(i' j

"~2

f

9

*

00, 00)

00, 00)

=

8.0

=

00

overleaf')

1030

Chapter 18

Trellis-Coded Modulation TABLE 18.11: ( continued)

Pmr1lilftioll11llllll lPairtiition !even

M§SD

p

Sad])!Coirlle Ap((li)

0

Co* Co* Co

min(4, 2, 0.586) = 0.586

(0,0,1)

Co* Co* cf

min(4, 2, 0.586) = 0.586

(O,U)

min(4, 2, 1.757) = 1.757

(1,1,1)

Co* Co* C3

min(4, 2, oo) = 2.0

(0,0,2)

4

Co* cf* C3

min(4. 2, oo) = 2.0

(0,2,2)

5

c~ * C3 -I 2 ct* c * C3

min(4. 6, oo) = 4.0

( 4,4,4)

min(8, 6, o~) = 6.0

(2,2,2)

C} * C3 * C3

min(8, oo, oo) = 8.0

(4,4,0)

min(8. oo. oo) = 8.0

(0,4,4)

1

2 3

6

Cose1l gelll.erntor

azp

+ +

Co*Co*q

+

+ +

Co*

gp

+2

7

t

8

C~ * C3 * C3 -1,

9

C3 * C3 * C3

min(oo. oo, oo) = oo

-

Adapted from [33].

We now consider the partitioning of the 4 x 8-PSK signal set.

!EX.AMPLE UUB

of

8i

For the 4 x 8-PSK signal set we still have 05 = 0.586 :::c = 2.0 :::c = 4.0. In this case only one sequence of iinear codes results in a good partition. We choose C4 C C3 c C2 c C1 c Co as follows: e

Co is the (4, 4, 1) code consisting of all 16 binary 4-tuples.

@

C1 is the (4. 3, 2) code consisting of the 8 binary 4-tuples of even weight.

®

C2 is the (4, 2, 2) code consisting of the codewords (0000), (1010), (0101), and (1111).

e

C3 is the (4, 1. 4) code consisting of the codewords (0000) and (1111).

@

C4 is the (4, 0, oo) code consisting only of the codeword (0000).

t.t

The choices of multilevel subcodes that result in the best at each partition level p, along with the coset generators (in integer form) and the values of calculated using (18.87), are listed in Table 18.12. Using the coset generators from Table 18.12 in (18.93), we can determine the signal set mapping corresponding to the partition of 4 x 8-PSK.

t.t

Section 18.5

Multidimensional TCl\ll

i!JJ3l1

The 4 >< 8-PSK M§§!O)

0

= 0.586

1

= 1.172

2

=

2. 2.343)

1.172

= 2.0

= 2.343 .r::)

4.

-

"1.0

6

L! •

--

4.0

8.

= 4.0

8. oo)

= 8.0

8 9

= 8.0

CY).

from [33].

The

and 18.18: choice for

0

a multilevel code that n1a:,imizes

at every

o A summary of L x 8· ;PSK developed in these three v"'~"'"J,,-u appears in Table 18.13 for L = 2. 3, and 4. o

the same sets of code sequences used to multi-D 8-PSK multi-D 16-PSK and QAM sets. sets can be used to a wrnmary of the L x 16-PSK and L >< 16-QAM partitions in 'Tables and for L = 2, 3. and 4, where = 0.586 :::: = 2.0 :::: = 4.0 for the constituent 16-PSK = 1.0 :S of = 2.0 :::: = 4.0 :::: = 8.0 for the constituent set. In both cases the coset to form (18.93) requires addition modulo-16.

8i

Given a

cn,,~,HHF,

8i

8~

Li;,

of the multi-D signal space sL such that the MSSD is level p. a number of coded information bits 1 :::: k :::: k. the desired MSSD L',~ . Then. a rate R = k/(k + 1)

t

k+l

-· 0

IJJ l'J

n

::,

TABLE 18.13: Summary of L x 8-PSK partitions, L

PmrtitfoJra MSSlD

level

L = 3(1)

L =2
MS§D

GeIDJ.erntor

Generntoir

ro

= 2, 3, 4.

L = 3(U:)

MSSlD

OJ "O r>

L = 3(U:Jf)

M§SD

-·

-s

Gelillerntoir

00

L = 41

MSSD

Gelflerauor --j

l'1 p 2

p

I

Ap 2

gp

Ii p 2

gp

Ap 2

'/i',p

gp

10

-

-

-

-

-

-

-

-

11

-

-

-

-

-

-

-

-

0.586 1.172 1.172 2.0 2.343 4.0 4.0 4.0 8.0 8.0 8.0 16.0

5

4

0 1 2 3 4 5 6 7 8

0.586 1.172 2.0 4.0 4.0 8.0

(OJ) (1,1) (0,2) (2,2) (0,4) (4,4)

-

-

-

-

-

(1,1,1) (1,1,0) (0,1,1) (2,2,2) (2,2,0) (0,2,2) (4,4,4) (4,4,0) (0,4,4)

0.586 1.172 1.172 2.0 4.0 4.0 4.0 8.0 8.0

(0,0,1) (0,1,l) (1,1,l) (2,2,2) (2,2,0) (0,2,2) (4,4,4) (4,4,0) (0,4,4)

0.586 0.586 1.757 2.0 4.0 4.0 4.0 8.0 8.0

(0,0,1) (OJ,1) (1,1,1) (0,0,2) (0,2,2) (4,4,4) (2,2,2) (4,4,0) (0,4,4)

0.586 0.586 1.757 2.0 2.0 4.0 6.0 8.0 8.0

9

no

Ap 2

rr1

n2

Acapted from [33].

1 3

5

0

3

6

2

3

6

2

6

gp

(0,0,0,1) (0,0,1,1) (OJ,0,1) (0,0,0,2) (1,1,1,l) (0,0,2,2) (0,2,0,2) (0,0,0,4) (2,2,2,2) (0,0,4,4) (0,4,0,4) (4,4,4,4) 8

11

ro

;:;;·

h 0

o_ ro o_

s:

0

Q. C OJ

r>

o·

:J

L

of L >c 16-PSK and L >< 16-QAM

TABLE 18.14:

= 2, 3, 4.

L x 16-PSK

lF'mrilntliliD>rrn Ile Yell

JL M§§]lJ)

=2

IL

= 2l(TI)

Gerrneirmfoir / M§§]lJ)

Geimernill[J)n·

JL M§§ill)

= 2l(TITI) Gerrneirmil(!]1Ir

lL

= 2l(TITITI)

fyj]§§]l])

L

= jJ.

CGenniemillD>Ir

M§§]l])

CGenneirIDil([l)n·

(O,OJ)

0.152 0.304 0.304 0.586 0.609 1.172 1.172 2.0 2.343 4.0 LLO 4.0 8.0 8.0 8.0 16.0

(0,0,0,1)

p

0 1 2 3 4 5 6 7 8 9

1.172 2.0 4.0 4.0 8.0 -

(0,8) (8,8)

-

10 11 12

0.457 0.586 J .172 1.172 2.0 4.0 4.0 4.0 8.0 8.0

(8,8,8) (0o~o, 0 011 (0,8,8) -

-

-

14 15 n1

(8,8,0) (0,8,8)

-

13

no

0.3040.304 0.586 1.172 1.172 2.0 L'.!-.0 4.0 4.0 8.0 8.0

n2

n3

-

1 3

5

0

3

6

9

2

3

6

(0,2,2) (2.2.2)

(8,8,8) (8,8,0) (0,8,8)

-

-

7

0.152 0.457 0.586 0.586 1.757 2.0 4.0 4.0 4.0 8.0 8.0

9

-

-

2

5

6

9

;

4 8

'fl 11)

r,

F>

o·

:::;

~

~

(0,0,0,8)

C

a::

Q.

(0,0,8,8) (0,8,0,8)

~j"

ro

:::;

"'a·

:::;

w

12

15

(continued overleaf)

-I

n

s;: d>

0

w w

_. 0

w

.j::,,

TAB LE 18. 14: ( continued)

('"',

::, Ill

(b) L x 16-QAM

L = 3(1)

L =2

.....

L = 3(H)

Partition Bevel

MS§D

Gelrlerafor

MSSD

Generator

MSSD

Grel!llrerator

p

/.i 2

gp

/.i 2

p

gp

/.i 2

gp

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

1 2 2 4 4 8 8 16

(0,1) (1,1) (0,2) (0,4) (2,2) (4,4) (0.8) (8,8)

1 2 2 2 4 4

(1,1,1) (1,1.0) (O,U) (2.2,2) (2,2,0) (0,2,2) (4,4,4) (4,4,0) (0,4,4) (8,8.8) (8,8,0) (0.8,8)

1 1 2 3 4 4 4 8 8 8 16 16

(0,0,1) (0,1,1) (2.2,2) (U,1) (2,2,0) (0,2,2) (4,4,4) (4,4,0) (0,4,4) (8.8.8) (8,8.0) (0.8.8)

p

p

.::i I'll ....,

L

= 3(lln) Gellllrerator MSSD

....

L=4 MSSD

Geneirat([})r

00

gp

/i2 p

gp

-I ....,

(0,0,1) (0,1,1) (0.0,2) (0,2,2) (1,1,1) (4,4,4) (2,2.2) ( 4,4,0) (0,4,4) (8,8,8) (8,8,0) (0.8.8) -

1 2 2 2 4 4 4 4 8 8 8 8 16 16 16 32

(0,0,0,1) (0,0,1,1) (0,1.0,1) (0.0.0.2) (1,1,1,1) (0,0.2,2) (0,2,0,2) (0.0.0,4) (2,2,2,2) (0.0,4,4) (0,4,0,4) (0,0,0,8) (4,4,4,4) (0.0,8,8) (0,8,0,8) (8.8,8,8)

112 p

I'll

no

4

1 1 2 2 3 4

6 8 8 8 16 16

-

-

8 8 8 16 16

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

n1

Adapted from [33. 34].

1

3

0

3

3

2

-

-

4

6

4

8 --

.:;;·

A

0

Q_

I'll

Q_

:?:: 0

Q_

C Ill

.....

o·

:::l

Section 1

8~·/!e~

convolutional encoder mlust b,e selected to JLnaxin1ize the rriilnirmusrt distanc:e ar1td to rnini1nize . the nun1ber of code-bet\Neen trellis for the desired numbe;· of st2tes. the k + 1 = 1l ..... vlO v
of the n1uHHeveJ.

used to select a subset of A 1 (0) instead of 17 =

8.nd tlrn.t different codes will be

~1nd vc~lues of q.

When q > 0, the (V(JL-q-\)

••• 0

1/'cJJ":cf L--1

'§pare

or when the coset

1Y =

(r1 . .\'2., · ·

m

fornL '?{ ls

• .YLl =--=

convolutional encode:r the 2 >< 8-l?SK instead of the entire 18.15 to

X O lJ l >< } ) ) of function ( lKTl)

becorn.es ·y

=

cr1 . .\'2)

=

:z=

8)

l'cp'c5

(18.96)

8).

·where v( 4 ) and 1} 3 ) rP,n,·,,cF•n1 encoder and the MSSDs are

= 00 The best codes for multi-D 8-PSK, 16-PSK and QAM sets with L = 2, 3. and 4 are listed in Tables 18. For the QAM sets the codes are based on the 4-D. 6-D. and 8-D lattices zl. 2°, and ::Z8 , The codes search [33, 34]. Each lable were found

1CJ36

Trellis-Coded Modulation

Chapter 18

TABLE 18.15: Optimum codes for multi-D 8-PSK 16-PSK and QAM signal '--'Vi1;,tcllaL~vu~. (a) Codesfor8-PSK

TRELLIS-CODED 2 x 8-PSK 17 = 2.5 bits/symbol, q = 0, d;,in = 1.172, Amin = 4 (2x8-PSK)

k

V

1 1 2 2 2 2 2 3

1 2 3 4 5 6 7

IIJm

11(3) -

-

-

-

04 16 10 004 044 044

-

-

110

11.m

Jhi(O)

1 2 06 12 06 030 016 016

3 5

Knnaria111ice

90° 90° 45° 45° 45° 45° 90° 90°

11 23 41 113 107 317

d2

free

Aafree

d,~ext

Adnexl

1.757 2.0 2.929 3.515 3.515 4.0 4.0 4.0

8 4 16 56 16 6 6 2

2.0 2.929

4 32

-

-

-

-

-

-

4.101 4.101 4.101

80 48 25

y (dB) 1.76 2.32 3.98 4.77 4.77 5.33 5.33 5.33

TRELLIS-CODED 2 x 8-PSK 17 = 2.0 bits/symbol, q = 1, d1~ 1;11 = 2.0. A 111 ;11 = 2 (1 xQPSK) V

k

1 2 3 4 5 6 7

1 1 2 3 3 3 3

11P>

Ili(2)

-

-

-

-

-

04 14 14 050 044

04 24 012 110

nn
nn
!m'm"ia!l1iCe

1 2 02 02 06 004 016

3 5 11 21 43 125 317

45° 45° 180° 90° 90° 90° 90°

d2

free

Adr,·ee

d,~ext

Adnext

y (dlR)

3.172 4.0 4.0 5.172 6.0 6.343 7.515

8.0 6.0 2.0 8.0 6.0 5.5 25.0

4.0 5.172 5.172

6 32 16

2.00 3.01 3.01 4.13 4.77 5.01 5.75

-

-

-

-

-

-

-

-

TRELLIS-CODED 3 x 8-PSK 17 = 2.67 bits/symbol, q = 0, V

1 2 3 4 5 6 7

k 1 1 2 3 3 3 3 3 3

lhi(3)

lhi(2)

-

-

-

-

-

14 lO 30 050

04 04 04 14 022

ncr...

v~•v

112

100

050

1u' 1) 1 2 02 02 02 02 006 004 022

Ili(O)

3

5 11 21 21 53 103 225 255

d,;,;

11

= 1.172,

A

111 ; 11

= 12 (3x8-PSK. I)

~

Kmivariam:e

djree

A 1~i·ee

45° 45° 45° 90° 180° 90° 90° 9Qn 180°

1.172 1.757 2.0 2.343 2.343 2.929 3.172

4 16 (i

d;e.ri

Ad,u.'xt

-

-

-

-

2.343

16

i2 8 48 12

-

..J •.JJ.J

,, r, r

84

-

-

3.515

76

-

-

-

-

-

-

-

-

-

y (
Signnail set

0.00 1.76 2.32 3.01 3.01 3.98 4.33 4.77 4.77

II II I I I I I i I

Section '185

1(!})317

1\/lultidimensional TOil

T.A13LE 1 15: TRELLlS-CODJED 3 q = L

2.33

I) =

u,/~) ;,111

fu:('1})

l

3

1 02 02

7

V

~~

2 ?:,

1 2 2

06

2

Qi.

4 5 6

~1(.3)

3

3 2

lO

3 3

22 010 02,1

Lj.

060

04 M 16

Oc:J-6 014

06

02 06 060 00"2

Cllil

dl~lill =

·'""'-'"...

11 11 21 27 41

45°

180°

=

6 6 16 24 12 15 7 .1

8 (3><8-PSK, Ad11ex1

' 1/i-ee

2.0 2.586 3.515 3.757 3.157 4.0 4.0 4.0 -1.0

45°

105 101

1.757.

'j,-ee

90° 90° 90° 180° 45° 90°

8-PSK

)(

2.343

16

L).343

24

4.686

8

2

1[lllR/ §IlEn11_21~ 5ett 0.56 1.68

II

3.01

II II

3.30 3.30 3.57 3.57 3.57 3.57

m II III III III

TRELUS-CODFD 3 x 8-PSI(

ll11 \ t~ }

V

I)=

2.00

u1 Uil

Iln17)

= 2.0. q = 2. d2. /}1[/1 il1l{O)

1I1

1

2 :,

2

4

2

r

3

.)

1,!.

~

16 030 o,i4

.)

3 :/

6

0" 12 24

01-1

22

1

I 0-12 024

2 02 04 02 0(5 014 006

innvmtTI21rrn(C·£

180° 180° 45° 45° 180° 360° 180° 180°

3 5 ll

27 41

53 103

103

'Ji-ee

A,d/iw

3.757 4.0 4.0 4.0 5.757 5.757 6.0 6.0

24.0 15.0 7.0 3.0 17.5 17.0 11.0 4.0

/1 \"

>< QPSK)

d~exi

A\d11ex1

5.757

144

5.757

32

y{,,TIIF,)

5figlrll22 §<et

2.74 3.01 3.01 3.01 4.59 4.59 4.77 4.77

II III III III III Ill II

11

TRELUS-CODED 4 >< 8-PSK )) = V

Jnt4)

!~

q = 0. c12. = J}llJl

2.75 lli1( 2 }

n~131

~11 ( 1)

nnWI

]fo1v2111.·ft31nn~~

''.Ji-ee

3 5 ll 21 41 101

45° 45°

un

45°

2.0

1 2

3 4 5 6

2 2 3 3 4

030

10 30 020

2 04 04

l4 052

1.172.

1 02 02 02 014

45°

45° 45°

Amin = 24 (4

0

1.757

64

812.341 -

64

2..929 -

288

"

l.757 2.343 2.343 2.929

y(1uIBs)

d/i-ee

0.00 1.76 2.32 3.01 3.01 3.98

48

40 8 136

·---

TRELLIS-CODED 4 >< 8-PSK 17 =

2.50 bits/symbol. q = 1. d1~ 1i11 = 1.172, Amin= 4 (2x8-PSK) ---

V

k

1

4

l 2 2 3

6

3 3

2

3

s

Il11Ili

!l-tl(3)

-

14 24 014

-

2 04

o,!.

14 024

1 l 02 02 02 0Ll2

KnnY2:tii·Il21n11cc

3

5 11 21 4l 103

45° 45° 45° 45° 45° 45°

'fi-ee

2.0 2.343 2.343 3.172 3.515 4.0

A\d/i-ee

8 40

8

16 64 28

'11ex1

Aduext

2.343 -

64 -

3.l72 -

4.686

32 -

1088

)I

{«JIJB,)

2.32 3.01 3.01 4.33 4.77 5.33

( continued overleaf)

W38

Chapter 18

Treiiis-Coded Modulation TABLE 18.15: (continued)

TRELLIS-CODED 4 x 8-PSK 17 V

1 2 3 4 5

k

= 2.25 bits/symbol, q = 2, i;,,in = 2.0, 11<3)

1n(4)

1 2 2 2 3 4

1IiPl

-

-

-

-

-

3 06 02 06 06

-

-

-

-

-

04 04

10

110)

Hl(ll)

1 1

3 5

02 06 12 22

11

A 111 ;11

= 8 (4 x 8-PSK)

linv:uiam:e

d2 free

Adfree

d~ext

45° 45° 45° 90° 45° 45°

2.343 3.172 4.0 4.0 4.0 4.0

8 16 28 28 12 4

3.172

11 21 41

y (dB)

Adnexl

32

-

0.69 2.00 3.01 3.01 3.01 3.0i

-

4.343 4.686 4.686 4.686

64 64 32 16

TRELLIS-CODED 4 x 8-PSK 1)

= 2.00 bits/symbol,

q = 3, d,~zin

--

V

1 2 3 4 5 6

l 2 3 4 4 4 4

--

-

--

-

-

2 04 02 04 016 014

1n(2)

1n(3)

02 04 10 044 024

--

10 02 034 044

= 2.0,

Amin = 2 (1 x QPSK) ~

k

Hl(4)

h(I)

lhi(O)

1 3 03 03 22 036 016

dz

l!narian1ee 90° 45° 45° 45° 45° 45° 90°

3 5 11 21 41 107 103

.free

Adfi-ee

d,~ext

Adnext

y (dlIB)

4.0 4.0 4.0 4.686 6.343 6.686 7.029

28 12 4 8 16 6 24

4.686 4.686 4.686

64 32 16

3.01 3.01 3.01 3.70 5.01 5.24 5.46

-

-

-

-

-

-

-

--

(/,) Codes for 16-PSK

TRELLIS-CODED 2 x l6-PSK

= 3.5 bits/symbol,

11

I

V

1 2 3 4 5 6 7

11<2)

k 1 1 2 2 2 2 2 2

Hl(l)

1 2 06 12 06 030 016 132

-

04 16 10 004 044 074

CJ= 0, d 1;,i11

!n
Jlnnarianee

3 5 11 23 41 113 107 217

45° 45° 22.5° 22.5° 22.5° 22.5° 45° 22.5°

= 0.304, dz

Amin= 4 (2 x 16-PSK)

free

Adfi-ee

0.457 0.586 0.761 0.913 0.913 1.066 l.066 1.172

8 4 16 56 16 80 48 4

d,;ext -

y (lillIB)

Adnext -

0.761

32

-

-

-

-

-

-

-

-

I

-

-

l.218

228

I

1.76 2.84 3.98 4.77 4.77 5.44 5.44 5.85

TRELLIS-CODED 2 x 16-PSK 17 = 3.0 bits/symbol, q V

k

1P'

h(2)

h(I)

n!Ol

= l,

d1~,in

= 0.586,

fovariancc -----

1 2 3 4 5 6 7

1 1 2 2 2 2 3

-

-

-

-

-

110

04 14 30 044 044

1 2 02 06 16 016 016

3 5 11 23 41 107 317

22.5° 22.5° 90° 45° 45° 45° 45°

Amin

dz .fi'ee

Ad/i-ee

0.89 l.172 1.476 1.757 1.781 2.0 2.085

8 4 16 8 16 4 25

= 2 (1 x 8-PSK) d,~e.,·t -

1.476

Adnext

48

1.82 3.01 4.01 4.77 4.83 S.33

-

'i.'il

-

32

-

-

-

-

-

-

2.085 -

y (li!R)

J

Multidimensional TCM

Section 1 TABLE

8.1

TRELLIS-CODED 17

k 1 2 3 4 5 6 7

1 1 2 3 3 3 3 3 3

= 3 67 bits/symbol

q

'Hll3l!ll

=0

·; iin

ilni3)

><

16-PSK

= 0 304

A 171111

= 12 (3 x 16-PSK I)

'kee

14 10 30 050 056 100

04 04 04 14 022 112 050

1 2 02 02 02 02 006 004 022

22.5° 22.5° 22.5° 45° 90° 45° 45°

3 5 11 21 21 53 103 225 255

90°

0.304 0.457 0.586 0.609 0.609 0.761 0.890 0.913 0.913

4

16 6 12

0.609

16

0

0

48 12 84 76

II II I I I I I I I

0.00 1.76 2.84 3.01 3.01 3.98 4.66 4.77 4.77

TRELUS-CODED 3 >< 16-PSK 17 V

1 2

3 4 5 6 7

k 1 2 2 2 3 2 3 3 3

3

= 3.33 bits/symbol,

1m(3)

n,12) -

-

3 06 04 04 14 16 046 102 072

-

10 -

34 032 014 006

c1 L

= 1. c12Jtll11.. = 0.457,

n,m

ft-u(O)

TIIT11v21rrll21m1~e

1 1 02 02 06 02 06 006 044 062

3 7 11 li 21 27 41 103 203 223

45° 45° 45° 90° 22.5° 45° 22.5° 22.5° 22.5° 45°

= 8 (3 x 16-PSK, U)

'/i-ee

Jl~free

d,;exl

0.586 0.738 0.913 1.043 1.043 1.172 1.172 1.218 l.370 1.476

6 6 16 24 12 12 4 8 32 8

0.609

Adnext

y iirlliE)

§igmnll §e\t

16

1.08 2.08 3.01 3.58 3.58 4.09 4.09 4.26 4.77 5.09

II II II II III II Ill III III III

-

-

-

--

-

-

-

-

1.195

24

-

-

-

-

-

-

-

-

I

TRELLIS-CODED 3 >< 16-FSK 17 V

1 2 3 4 5 6

7

k 1 1 2 2 2 2 2 3 3 3 3

ilnl3)

= 3.00 bits/symbol, Illl(2)

11nm

Il,Wl

-

-

1

3

-

-

-

04 12 14 22 054 004 030 106 110

2 02 04 02 14 020 012 026 050 052

5 11 27 41 43 115 101 101 213 203

-

020 050 060 016

q

= 2,

JIIT11vaR'Il£1i!T1<ee

90° 90° 22.5° 22.5° 22.5° 45° 22.5° 45° 90°

,_,

j'70

90°

= 0.586,

Amin

'.free

A,~i-ee

d~ex1

1.043 1.172 1.172 1.628 1.628 1.757 1.757

24 12 4 32 16 16 8

c/1~ 1; 11

'U 2.0 2.0 2.0

6 6 6

-

1.628

= 2 (1 x 8-PSK) Aduext -

144

-

-

-

-

-

-

-

-

2.085 2.085 2.085 2.214 2.343

48 72 60 56 64

y (iilliE)

§ignual §e\t

2.50 3.01 3.01 4.44 4.44 4.77 4.77 5.33 5.33 5.33 5.33

II II III III III III III II II III III

(continued overleaf)

1040

Chapter 18

Trellis-Coded Modulation TAB LE 18.15: ( continued)

TRELLIS-CODED 4 x 16-PSI(

= 3.75 bits/symbol,

1J V

k

1 2 3 4 ' 5 6

1 2 2 3 3 4

ni3l

1:ni4J

11P>

-

-

-

-

-

-

-

-

10 30 020

2 04 04 14 052

-

030

CJ= 0, d1~iin

= 0.304,

1Ji(1)

lli(OJ

Inuvari:mee

1 1 02 02 02 014

3 5 11 21 41 101

22.5° 22.5° 22.5° 22.5° 22.5° 22.5°

Amin= 24 (4 x 16-PSK)

d2

ji-ee

Adji-ee

d;;ext

0.304 0.457 0.586 0.609 0.609 0.761

8 48 8 40 8 136

0.457

Adnext 64

-

-

0.609

64

-

0.761

-

288

-

-

y (
0 1.76 2.84 3.01 3.01 3.98

TRELLIS-CODED 4 x 16-PSK 17 V

k

= 3.50 bits/symbol, 1Ji(2)

11Pl

1

1

-

2 3 4 5 6

2 2 3 3 3

-

-

2 04 04 14 024

-

14 24 014

CJ= 1. d1~ 1in

= 0.304,

Amin

= 4 (2 x 16-PSK)

1nm

1m
Km•!i!ri!mee

d};.ee

Adji-ee

d,~exl

1 1 02 02 02 042

3 5 11 21 41 103

22.5° 22.5° 22.5° 22.5° 22.5° 22.5°

0.586 0.609 0.609 0.890 0.913 1.172

8 40 8 16 64 24

0.609 -

Adnext

64 -

0.890 -

32 -

-

-

1.218

1088

y («i!IIB) 2.84 3.01 3.01 4.66 4.77 5.85

TRELLIS-CODED 4 x 16-PSK 1J V

1 2 3 4 5 6

k 1 2 2 2 3 4 4

= 3.25 bits/symbol, lli(3)

1n<4J

1Ji(2)

-

-

-

-

-

-

-

-

-

3 06 02 06 06 024

04 04 030

-

10 050

q

= 2.

c1;,in

= 0.586,

Amin= 8 (4 >< 16-PSK)

1nm

1n
Knnariallllee

d2 free

Adji-ee

d;;e.,·1

1 1 02 06 12 22 016

3 5 11 11 21 41 101

22.5° 22.5° 22.5° 45° 22.5° 22.5° 22.5°

0.609 0.890 1.172 1.172 1.172 1.218 1.499

8 16 24 24 8 16 72

0.890

Adnext

y (
32

0.17 1.82 3.01 3.01 3.01 3.18 4.08

-

-

1.195 1.218 1.218

64 64 32

--

-

-

-

--

TRELLIS-CODED 4 x 16-PSK 1J V

1 2 3 4 5 6

k 1 2 3 3 3 3 3

= 3.00 bits/symbol,

li(3)

lil(2)

-

-

-

2 04 10 16 14 024

02 04 22 24 050

IJi(l)

1 3 03 06 06 02 006

CJ

= 3,

c/;

; 11 11

11<01

Kllllvariam:e

3 5 11 21 41 43 103

45° 22.5° 22.5° 22.5° 22.5° 45° 22.5°

= 0.586, dz

Amin

= 2 (1 x 8-PSK) y 1
:Ji-ee

A,l;,·ee

d;;e.,·/

Aduext

1.172 1.172 1.218 1.781 1.804 1.827 2.0

24 8 16 48 24 64 8

1.218 1.218

64 32

3.01

-

3.18 4.83 4.88 4.94 5.33

---

-

-

-

-

-

2.343

64

3.01

Seci:ion 8.5

Multidirnensiona! TCfv1

'l!l},i.1

TABLE H.U5: (c) Codes for QAM

TRELLIS-CODED 1) =

k

rrn( 3 )

lin(i!l)

Amin = 24 ('.2::4 /2)

]-1-

02 10

6

10

34 020

030

052

I

_)

11

06 06 06 004

dfl·~ Ari/i-ee

180° 90° 90° 90° 90° 180°

~

3

2 3 4

5

Ilnnvaitfimarrc

:',

l 2 2

2 3 4

n,!Ol

mil'!)

;,12)

23 41 ]01

I ___ ~-ly-i,mi

l 1:: -1.8

-

53

8

s

g

l36

-

64 512 288 ~-

--

o.oo I l.76 3.01 3.01 3.01 3.98 -

TRELLIS-CODED :z: 4 /2

q=L

I)= J

i/2. nun

=2- Amin = 4-

----

n1{2)

k 2

3

2 _,' 3 3 3

4 5 6 7

04 04

14 24 024

1,1 014 03
l),U

i!i['1l

n-,10)

_,"

90°

2 02 02 06 042 106

5 ll 21 43 103 203

90° 90° 180° 180° 180° 180°

Aduext

4. 4 4 6 6

88 24 8 176 48 l 112 216

,, 0

8

~

0'

6

--

=----r'

y (irl!113) .

3.01 3 01 3.01 4.77 4.77 I 6.02 -=--L6.02 512 320

TRELLIS-CODED ;;z6 1)

V

k

n,1")

= J -1- 2/3

nn13)

n,rn

1 l l

-

-

-

-

--

-

2 2 2 3 3

-

-

-

-

-

-

2 3 4

I 5 6

3 4

12 -24 024 044 I 024 -

02 14 16 04 14 042 i 072

fi,1 i I

nu WI

Hnnvnw]21nnirie

1 2 06 02 04 02 02 010 022

3 5 11 21 23 21 41 105

90° 90° 90° 180° 360° 180° 180° 180° 90°

103

'/i-ee

2 2 2 3 3 4 4

I

4 4

Ad/i-ee

'11exl

Adnest

3 3

256

28 12 4

M 32

3

48

-

-

40 316 124

-

-

-

-

--

-

60

~512

2~--

-

y\1TI!B)

§Ilgm1~n~ ~i~t1"

0.00 0.00 0.00 1.76 l.76 3.01 3.01 3.01 3.01

l Il III II

11 J I I

lll

(co11ti1111ed overleaf)

1042

Trellis-Coded Modulation

Chapter 18

TABLE 18.15: (continued) 1 KELuS-cuutu 11.,6

= J + 1/3 bits/symbol,

ry 1u(4)

k

V

lll(3)

1 1 1 2 1 2 3 2 3 4 3 5 4 10 6 4 044 4 030

lll(2)

-

-

III(l)

III(Oi

1 1 2 3 1 02 06 04 03 04 12 22 04 022 010 014 042

3 3 5 5 11

-

-

--

06 14 12 006 046

q

= 1,

d;,in

Iravaria11.ce d2 free

11

23 41 111 101

= 2,

= 12 ( Z 6/4, II)

Amin d~ext

Adfree

2 3 3 4 4 4 4 5 6 6

90° 360° 360° 180° 180° 90° 90° 90° 90° 180°

/2

4 56 32 348 60 28 12

Adnext

y (!l!JB)

Sig11.aH set

0.00 1.76 1.76 3.01 3.01 3.01 3.01 3.98 4.77 4.77

III II II I II III III III III III

y (!l!JB)

Sig11.aD set

3.01 3.01 3.98 4.77 4.77 5.44 6.02 6.02

II III Ill III III II II II

64

4 -

-

5

288

-

-

6

6400

-

-

5

160

96

-

1012 856

-

-

-

-

-

TRELLIS-CODED 71.,6 /4

=

17 V

k

1u(4)

1u(3)

1 2 3 4 5

1 2 3 3 3 4 20 6 4 024 4 044

J bits/symbol, q h(l)

III(2l

04 04 30 10 010 024

= 2,

= 4 (Z 2 /2)

Amin

Ac'l'ree

d~exl

Adnext

60 12 64 176 32 240 700 316

6 5

1024 128

4 4 5 6 6 7 8 8

180° 90° 90° 90° 90° 180° 180° 360°

-

-

d;,in

K!nariam::e d2 free

1uW1

3 1 3 2 5 03 11 02 12 21 06 16 14 41 04 02 41 004 042 101 014 002 103

-

= 2,

-

-

-

-

7

288

-

-

-

-

-

-

TRELLIS-CODED z8 ry V

k

1 2 3 4 5 6

1 2 3 3 3 3

=

J

h(3)

+ 3 /4 bits/symbol,

lht(ZJ

-

-

2 02 04 14 032

-

04 10 24 050

q

= 0,

= 2,

d;,i 11

h(l)

h(O)

li11.variimce

dz ji-ee

1 1 01 02 02 004

3 5 11 21 41 103

90° 90° 90° 90° 90° 90°

2

Amin

= 112 (Z8/2)

Adfree

d,~exl

48 16 224 1264 496 240

3 3

2 3 4 4 4

Adnext

512 384

-

-

-

-

-

5

5120

y (!l!JB)

0.00 0.00 1.76 3.01 3.01 3.01

TRELLIS-CODED z8 /2 17 V

k

1 2 3 4 5

1 2 2 2 3 3

6

4

lht(4)

= J + 1/2 bits/symbol, q = 1, III(31

-

-

-

-

h(2) -

-

-

-

-

2 04 12 34

-

14

-

0~

1~

006

056

014

lhi(l)

!n(ij)

1 1 02 04 06 22 022

3 5

c1;,in

llnnari:mce

= 2, dz

Amin= 24 (Z 4 /2)

Ji-ee

Adfree

23 41

90° 90° 90° 90° 90°

2 4 4 4 4

16 1264 496 240 112

43

180°

4

112

103

90°

~

48

11

i;;ext

4 -

Adnext

1024 -

-

-

y (dB)

0.00 3.01 3.01 3.01 3.01

6 6

20480 11264

L V

1rv,,1n _LVL.'"'TV

~ _,,V.l

3584

3.01

6

"'

Multidimensional TCI\Jl

Section 'l8.5

'U0Jl1::,i

TABLE 18.15:

TRELUS-CODED "£8 /4

k

V

I 21 3

4 5

= 16 (Z: 8 /8)

+

17=

u,m I nnW>

t,Pl

Tinn-v21rrli21n11ce

I

I

1

-

-

-

1

1 2 2

-

--

-

-

-

-

-

3

-

.'..j

lO

1.0 04

06 02 14 06

2 02 06 06 22

3 5 11 11

90° 90° 90° 180° 90° 90°

21 41

Pldnext

"free

lidfi'ee

4 4 4 4 4

496 240 112 112 1.J.g

6

Lj.

16

6

-

yiiilllIB)

3.01 3.01 3.01 3.01 3.01 3.01

-

6 6 6

4096 4096 2048 512 512

TRELUS-CODED z:8 /8 1) =

k

V

hnu-:.,

t,(3)

q

.J llii(2)

lln(3 1

r.~lll)

l 3 03 03 06 006

3 5

= 3.

2. d 111111

AilllvZJrcii21ll1lee

dj,.ee

180° 90° 90° 90° 90° 90°

4 4 4

I-·

l 2 3 4

5

1

-

-

2 3 4

-

-

t.!. Lj.

6

-

10 20 0,12

-

04 04 10 020

2 02 02 02 010

11 21 41

101

= 2'

0 0

8 8

= 4 (z 2 /2)

A due.YI

Adfm

112

8 8

48

0

16 6896 3056 1264

0

16384 16384 11264

-

-

-

-

-

-

)l

\~1113) 3.01 3.01 3.01 6.02 6.02 6.02

Adapted from [33, 34].

For the

0

sets, 17 is given set S has

The value of q that determines the multilevel subcode at which

n

The M1SE distance and number of nearest set wii:h the san1e

n

The number of coded bits I(, lhf"l ... · . ]ln(lJ. JmiOl in octal minimum rotational MFSE distance dJ,.ee' the average number of nearest Ac11,cc, and the gain y for the best code of each constraint length v.

o

When occurs and the average number of next-nearest neighbors Ac1

SE distance are also

d,~ext

o

In the L

o

Sometimes more than one code is listed for a given constraint to indicate the best code for different of rotational invariance. In these cases, the code with the worst rotational invariance has a larger free distance or a smaller average nurnber of nearest or next-nearest neighbors.

=

3 cases, the

11

n,

that results in the best code is also

1044

Chapter 18

Tre!lis-Coded Modulation

The following comments apply to Table 18.15: n

®

For a gi,.1en multi-D signa! conste!!ation, each value of q results iG a diffe;ent spectral efficiency. For example, for 4 x 8-PSK, the values of q = 0, l, 2, and 3 result in spectral efficiencies 17 = 2.75. 2.5, 2.25, and 2.0 bits/symbol, respectively. In general. different values of q result in different optimum codes for a given multi-D signal constellation.

"' The uncoded signal sets that are chosen for comparison are uncoded subsets of a multi-Dor 2-D signal set with the same spectral efficiency. For the QAM signal sets, these are uncoded subsets of an integer lattice. For example, Z 4 /2 denotes the best binary partitioning of the integer lattice Z 4 . In the 6-D case, the partition (I, II, or IH from Example 18.17) that determines the uncoded subset is indicated. e

0

Unlike the case with 2-D signal sets, multi-D signal sets are capable of achieving full rotational invariance with linear PCEs, as first noted by Wei [35]. In fact, many of the best codes listed in Table 18.15 have full rotational invariance. Tables of optimum codes for multi-D QPSK signal sets are also given in [33]. Again, many of the best codes achieve full rotational invariance with linear PCEs.

o Tables of optimum geometrically uniform codes for L x M -PSK signal sets are given in [36]. 0

An estimate of the real coding gain at a BER of 10-5 for the codes in Tables 18.6-18.9 and Table 18.15 can be obtained by adding the adjustment factor -0.2 log 2(Ad/in.I Amin) dB to the asymptotic coding gain y given in the table. For example, from Table 18.15(a), we see that the estimated real coding gain of the 8-state, 2 x 8-PSK code of Example 18.15 with spectral efficiency 17 = 2.5 bits/symbol is given by Yadf ~ y - 0.2 log2 (Ac11;ccl Amin) = 3.98 - 0.40 = 3.58 dB.

As noted previously, multi-D signal sets are capable of achieving full rotational invariance with linear PCEs. We now illustrate this fact with an example.

lEXAM PllE

nu 9

Rotatiol!llai !m,aii-iam:e for Mu!ti-D T(M

Consider the 8-state, rate R = 2/3, 2 x 8-PSK TCM system with spectral efficiency = 2.5 bits/symbol of Example 18.15. The parity-check matrix is given by

17

(18.97) and the binary parity-check equation can be written as V(D)HT (D)

= v( 21 (D) [ D 2 /(D 3 EB 1) J EB vOl (D) = O(D)

J

[ (D 2 EB D)/(D 3 EB 1) EB v< 0 \D)

(18.98a)

Section 18.5

1\l'iultidimensional TCf\/1

UD&15

or

EB (D 3 EB l)w(Ol (D)

=

(18.98b)

constituent 8-PSK set shown in y results in the rotated signal also Problem

For

v,C 0l

v,
= v(O) = v(ll

v,m

= v( 2 ) EB v(OJ

EB

L

a

(18.99a)

EB v
To determine the effect of a 4-5° rotation of the 2 >< 8-PSK vector c;y = (.r 1 • on the code vector v = (vC 5),/+lv< 3l v< 1lv(Ol), we express the ""'"·~~·,,,- fonction of (18.71) as 8).

(18.100) the Yi.r = Yi -1--

and we can write

1

of the rotated 8),

as

+ 2v( 3l + v< 1l)(L 1) + (4v( 4 l + 2v< 2 l + v(Ol) 1) + (L 1) 8) (4v< 5l + 2v< 3 ) + v(l) + l)(L 1) + (4vC 4l + 2v< 2 ) + v(O))(O, 1)

= (vi.,,

=

= (4v(S)

8).

(18.102) (18.100) and (18.102). we see that bits vC 5l, v(3l. and v(ll are affected rotation in the same way as bits vC 2 J. vUl, and by the multi-D 2 >< 8-PSK 45° v (0) are affected when the constituent 8-PSK set is rotated by 45°, whereas v( 2l, and vCOJ are not affected a, all. bits (0) __

Vr

vCO)

(18.103)

To check for rotational invariance of the linear PCE of we use (18.103) to form the rotated encoder output sequences v;.0\D) = ~/Ol(D). w;1\D) = v( 1l(D) EB 2 Jl(D), and v;. l (D) = v< 2l (D). these rotated sequences into (18.98b )., v,;e obtain (D)]PJT (D) = D 2w; 2\D) EB (f/ EB D) 11;1\D) EB

EB 1)w;0\D)

(D) EB (D 2 EB D)[w( 11(D) EB li(D)] EB

EB D)li(D) = @(D):

EB l)'Y(O\D) (18.104)

'W46

Chapter 18

Trellis-Coded iv1oduiation

that is, the rotated sequences satisfy the PCE. Thus, this 2 x 8-PSK TCM system with a linear PCE is invariant to 45° phase rotations. Because only vf 5l. vf 3l. and vm are affected by the phase rotation, these are the only bits that must be differentially encoded. (In this example, only v(l) is an encoder input bit, whereas v( 3 l and v( 5l are uncoded input bits.) We now conclude our discussion of rotational invariance for multi-D TCM with a few remarks. ®

In general, for full (360/ M) rotational invariance of M-PSK signal sets, I = log 2 M bits are affected by the phase rotation and must be differentially encoded. (For QAM signal sets, full 90° rotational invariance requires that 2 bits be differentially encoded.) In the case q = 0, that is, when partitioning begins with the entire multi-D signal set, the levels in the partition chain corresponding to the I affected bits are denoted by no, n1. · · · , n 1_1. These affected bits are listed in Tables 18.13 and 18.14 summarizing multi-D partitions. (Note that the bits to be differentially encoded depend only on the signal set and the partition, and not on the particular code.) 0

o

In the case q > O. that is. when partitioning begins with a subset of the entire multi-D signal set, the I affected bits correspond to levels no - q, n1 - q. . . . • n 1 -1 - q in the partition chain. (If n.i - q is less than zero for some j, full rotational invariance cannot be achieved. See [33] for more details on determining the bits to be differentially encoded when full rotational invariance is not possible.)

o

If v(O), the parity bit, is one of the bits affected by a phase rotation, it must be fed back to the differential encoder from the encoder output. since it is not an encoder input bit. The reader is again referred to [33] for details.

0

0

The possibility of achieving full rotational invariance with linear PCEs and multi-D signal sets depends on the multi-D mapping functions of (18.93) and (18.95). (As shown in Section 18.4. the simple mapping function used for 2-D signal sets does not allow full rotational invariance to be achieved with linear PCEs.) For a given multi-D signal set, partition, and linear PCE, the degree of rotational invariance can be determined by checking whether the rotated encoder output sequences satisfy the PCE (see Problem 18.32). In general, it is possible to achieve full rotational invariance with linear encoders of rates R = 1/2, 2/3, and 3/4 and multi-D signal sets, but higher encoder rates still require nonlinear encoders similar to those employed in Section 18.4. Uncor1Pr1 hit~ 51ffertPil hy

~

ph~(;:p rot~tinn rln. nnt in-Anenre thP rntatlnn~l

invariance of the code; however. they must still be differentially encoded. o

More general classes of both 2-D and multi-D TCM systems can be obtained by defining the code alphabet over a ring or a group rather than using conventional finite-field algebra. These more general formulations often make

Multidimensional TCM

Hli!7

it easier to obtain

'UvuL!",Huare summarized in [37, 38], uniforn1 codes over groups for L x !VI-PSI(

and tables of

\!Ve novv use an Section 18.3 can be

of

Consider the The encoder

feedback encoder used in one uncoded information Xn this of The first two levels of

the 2 >< 8-PSK,

partitioning are shovvn in In this case, q = l, the t,, 0 = 1.172. t,,, 1 = and branch of the binary error )

a

the MSSDs are parallel transitions on each function (see (18.95a)) from the

)

00 02 04 06

13151711 26 20 22 24

31 33 35 37

:l~

x OU] X l) 44 46 40 42

Q'(O

=

1.172

57 51 53 55 62 64 66 60 757'/7173

11 13 15 17 37 31 33 35

00 02 04 06 26 20 22 24 44 46 40 42 62 64 66 60

0206 2024 46 42 64 60

00 04 26 22 44 40 62 66

;_)\_~

Q'(l

X

1) 55 57 5153 73 75 77 71

1115 3733

1317 3135 57 53 75 71

55 51

73 77

:lf =

2.0

t.~ =

4.0

~ ~)\____\ ~~ 0

Q2(oo x 00 U 10 x 10) Q (00 x 10 U 10 x 00) Q 2(01 X 01 U 11 x 11) Q (0l x 11 U 11 x 01) FIGURE 18.32:

0

of the

4-D signal set Q2 (0 x O U 1 x 1).

10413

Chapter 13

Trellis-Coded Modulation

binary code vector v = (v( 4lv(3lv(2Jv< 1lv< 0J) to the binary signal vector Y is given by

un un un un un

Y = (y1, Y2) = v( 4lgs EB 1P!g4 EB v(2)g3 EB v< 1lg2 EB v(Olg1

~ v'4) v'"

$ v(3)

$

$ v"'

v{O)

$

(18.105)

(If the coset generators g; are expressed in integer form, then (18.95b) using modulo-8 addition gives Y = (_v1, _v2) in integer form.) This TCM system is listed in Table 18.15(a), where it is seen that d},ee = 4.0, Ad/ice = 6, d?ext = 5.172, and Ac111e.r1 = 32. Before we can calculate the A WEFs for the trellis paths A;1" (X) and the parallel transitions Af:v(X), we must first determine the AEWEs of the 32 possible binary error vectors e = (e< 4 le< 3le< 2le0le(Ol). We do this by using the AEWEs for naturally mapped 8-PSK listed in Table 18.5. First, we use a binary error vector e in (18.105) to compute the 4-D error signal IE= (e1, e2). Next, we use e 1 and e2 to determine the AEWEs of each component of IE from Table 18.5. We then multiply these two AEWEs together to form the AEWE corresponding to the error vector e. For example, the error vector e = (01011) results in the error signal lE = [ (001/, (111) 7 ], or E = (1, 7) in integer form. From Table 18.5 we see that the corresponding AEWEs are given by ,6,f001 /X) = x 0586 , and ,6,f111 J(X) = 0.5x 0586 + 0.5X 3414 . Thus, the AEWE for the error vector e = (01011) is given by ,6,To1011/X)

= ,6,~01/X) ,6,Zlll /X) = (Xo.ss6)(0. 5 Xoss6 + 0 _5 X3414) = o.5x

1172

(18.106)

+ o.5x 40 ;

that is, for branches in the binary error trellis labeled with the error vector e = (01011), half the branches have squared Euclidean weight 1.172, and the other half have weight 4.0. The complete list of AEWEs corresponding to the 32 possible error vectors in this example is given in Table 18.16. The modified state diagram, similar to the modified state diagram of Figure 18.19(a) for Example 18.9, is shown in Figure 18.33, where in this case each branch is labeled with the sum of the eight AEWEs corresponding to the eight parallel transitions on that branch. For example, the branch labeled c represents the self-loop around the state S3, caused by the input bit Oand resulting in the output bits 01, for the rate R = 1/2 encoder. Thus, the eight possible parallel transition erro1 vectors corresponding to this branch are e = (00001), (00101), (01001), (01101), (10001), (10101), (11001), and (11101), and from Table 18.16 the branch label is given by c

= x 1172 + 2x 40 + 4(0.25x 1172 + o.5x 40 + o.25x 6·828 ) + x 6828 = 2x1112 + 4x40 + 2 x6828_ (18.107a)

Section ·1 a.5

4-D

Ti\BLE 1

JI

o~oc

10000 10001 10010 10011 10100 10101 10110 10111 11000 11001 11010 11011 J 1100 11101 11110 11111

vl.172

00001 00010 00011 00100

_/_~

l

2

OCH)} 00110 0011

I

2

01000 01001 01010 010Jj OllOO

l

2

01101 01110 I o- 111

lL'!-

Multiditnensionai TCM

i

-~--

2

172

+!

set

(0

X

O LJ 1

l

2 l/yl.172 -+ '

+ J2

+

v6

A

l

::r

l

2

x-+ _L I

l

2

C

a= 4)(-~ + 4){h h = 21Y 1· 17 =+ 41Y-++ 2_.)(h.:,c~ c= 2 11·+ ~x-1 + 2 682s d = 2.;,,::: 1. ii::+ 4x-+ + 2./Y h.s~s

x,

e

f g FIGURE 1

T!+ 4-X-++ 2X 68 ] 8 X + 6X .\ + X" = 1 + 6X .\ + X s 4)(:: + 4)( 6

= 21Y = =

x

1 1 ·

11

Ivfodified state

for Example 18.20.

X

11J""'·S'

1)

1050

Chapter 18

Treilis-Coded Modulation

We determine the remaining branch labels in a similar way, yielding a= 4X20 + 4X6o b

= 2x1.1n +

d

= 2x1.1n

e

= 2x1.1n +

4X4.0 + 2x6s2s

+ 4X40 + 2X6.828 (18.107b)

4X40 + 2 x6828

f = Xo + 6X4.0 + xs o = 1 + 6X4 o + xs.o g

= 4X2.o +

4X6.o_

In this case, we note from (18.107) that a = g, and b compute the A WEF for the trellis paths as follows: 1

Aa/X) =

aeg(l - c) + abdg (1 - c)(l - ef) - bdf

=

=

e. Now, we can

a2b , 1 - b(l + f)

(18.108)

c

=

d

where the simplified expression follows from the foregoing identities. Carrying through the algebra associated with (18.108), we obtain the following result:

1 Aav(X)

=

32X 5·172 + 64X 8·0 + 64X 9172 + 32X 10828 + 128X 120 +32X13172 + 64Xl4828 + 64X16.0 + 32X18.828 ------------------1 - 4X 1172 - 8X 40 -12X 5·172 - 4X 6·828 - 24X 8·0 _ 2 x91n _ 12 x10.s2s _ 4 x120 _ 2 x14.s2s

= 32X 5 172 +

128X 6 344 + 64X 8 o + 576X 9172 + · · · .

(18.109)

Thus, for a given path through the trellis, there are an average of 32 error paths with a free SE distance of 5 .172, an average of 128 error paths with a distance of 6.344, and so on. The A WEF for the parallel transitions is given by the set of intrasubset distances within the subsets at the final partition level p = k + 1 = 2. From the subset Q 2 (00 x 00 U 10 x 10) in Figure 18.32 we see that (18.110) (Note that Agv (X) is just the set of distances corresponding to the parallel transitions in the self-loop around the all-zero state So. This set of distances is represented by the branch label f in the modified state diagram (with the weight-0 term, which corresponds to comparing a branch to itself, removed), since the branch label f gives the AEWE for the self-loop around the state So.) Equation (18.110) implies that there are six parallel transition error events with a distance of 4.0 and one with a distance of 8.0. Summarizing, we see from (18.109) and (18.110) that the minimum distance between parallel transitions is 1in = 4.0, the minimum distance between trellis paths is oJ,·ee = 5.172, the overall MFSE distance is

o;,

d};.ee = min {87,-ee•

o;,,in} =

4.0,

(18.111)

Section rn.5 =

Multidimensional TCNl

'W'S'i

6. and the number of next-nearest we can

for and Agv stand:u-d transfer function bound on event-error for multi-D TOA with a

Vie now conclude our discussion of

o

}Branches in the error trelllis vvith the sa1ne set of labels must have the smne labels in the modified state that o == g ~ b == d ~ and C

o

== e H1

18.20.

Ti]e transition A'WEF can be found from the AEWE of the around the all-zero state So the set of distances in the subsets at the final level of

.,,._,1.,_-1vv

·we can use a modified state

0

A'.,v Problem 18.33). We can then use these in (18.29b) to evaluate the standard transfer function bound on bit-error P1, method of vvith linear encoders. Nonlinear in Section 18.4. also can be used TCI\IJ[ but the same

to multi-D TCM vUvV'U<•>

invariant multi-D

values of such constellations are used. For ""cc"""~'""· CROSS constellation shown in Figure 18.34. combined with called ccm be used to achieve a The 4-D signal set con·wins a total of more than 7.0 bits/symbol log 2 36864 = 15.17, 1 parity bit and more than 14 information bits however. if subset of 2 ts = 32768 15

7.0

two categories of called inner points and outer in the constituent 2-D signal set, as illustrated in Figure 18.34. ln this case, the set of inner the 128 points. and the set of outer the 64 highest-energy signal Hall combinations of from the two constituent constellations are is not a valid multi-D signal point, 64 2 = 4096 points are excluded, exactly 2 15 = 32768 signal eliminating all combinations of two outer points. vve are the multi-D signal with the energies, thus achieving a power of about 0.3 dB with using the full signal

W52

Chapter 18

Trellis-Coded Modulation

0

0

0

147 146

"144 0

150 0

149 0

0

0

0

145

0

80 0

0

81 0

179 178

115 114

Q

0

0

0

112 113

0

0

©

0

183 182

119 118

G>

0

0

180 181 0

134 0

133

0

116 117 Q

87 0

84 0

136 137 168 169 ~ Inner points

0

0

0

0

0

"74

0

0

0

0

0

56 0

0

85 0

57 0

51 48 0

50

0

49

0

63 0

53 0

62 0

60 0

61

191 190

127 126

0

0

0

188 189

0

124 125 0

0

"24

0

19

25

0

0

0

0

28 0

95 0

92 0

0

0

0

0

0

94 0

93 0

0

156 157

0

0

12 79 0

76 0

39 36 0

14 0

0

32

0

5

15

35

©

6

4 0

40

Q

1

7

29 0

2

47 0

13 0

78 G

77 0

44 0

131 "130

0

105 42 0

41 0

34 0

33 0

38 0

37 0

46 0

45 0

0

128 129

"67 0

64 Q

99 0

96

~ 5

0

97

())

0

G

167 166 0

0

0

0

71 0

0

70 0

68 0

69

135 0

132

0

175 174

0

0

160 161

164 165

0

108 109

0

163 162

100 101

111 110 0

148

0

103 102 0

151 Q

"98 e

0

0

0

0

143 142 171 0

0

0

43

@

0

0

30

9

Q

0

"104 0

3

21

187 186 159 158 0

"8 0

0

107 106

0

10 0

22 0

73 0

11

17

"20 31

72

18

16 23

0

C>

26

0

0

0

89

0

75 0

0

27

0

54 0

52

88

0

0

0

55

86

58

90 0

0

0

0

0

91 0

120 121 59

0

0

0

123 122

0

176 177

0

0

0

82

0

184 185 152 153

0

0

83

0

----- Outer points 0 139 138

"1S5 15""

0

172 173

0

170

0

140 141

FIGURE 18.34: A 192-point generalized CROSS constellation.

set (see Problem 18.35). This power-saving approach to multi-D constellation design is called shaping. Ultimately, for very large signal dimensionalities and very large constituent 2-D signal sets, an approximately spherical constellation (known to be optimal in high-dimensional signal spaces) is approached, and a shaping gain of as rr1uch as 1.53 dB can be achieved. Now, we illustrate how a rate R = 3/4 encoder, combined with 11 uncoded bits, can be used to select a signal point from this multi-D shaped constellation. First, consider that all combinations of two inner signal points form a 4-D subset of 128 2 = 214 points. This 4-D subset can be partitioned into 23 = 8 subsets of 211 = 2048 points each. For a particular value of the parity bit, say v
185

Vv e also note that the of

Multidimensional TCM

11())53

results in a constellation 4-D TCM achieves

This is less than the 100% CER

of the n13xirriurrJL energy of any in the sets with the same set with the smaller CER will also have the smaller PAR of the rate l:i. == 90°

modern standard.

Jnode:n1 standard uses a :rate

;_fl..

==

allows the mapping of a standard also includes two other cost of increa:;ecl One with the 16-state code but is four time,; n1ore nonlinem· code that ::i.n additiornil 0.15 dB but is 3notheT factor of 4 times more In both of these cases more bits are encoded the convolutiona!I so fewer uncoded bits are needed to achie,1e a of the rnte is shown in CROSS conconvolutional encoder has and one uncoded (u< 3l). In bits and r:C 3 ) are are affected rotations. Full 90° rotational invariance can be achieved with a linear code in this case, since the constellation is multidimensional. The rate R = 2/3 feedback linear convolutional encoder has matrix IHI(D) = [(D

+ 1)/(D--I + 1)

+- 1)

1]

(18.112)

and v
Chapter '18

Trellis-Coded Modulation

ul91 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ~ I

ui81

r--1

Shell mapper

u(-11 - .~ - - - - - ~

1-----------------------~ yl-'1

--!>

Signal mapper

Differential encoder

!8lJ

'fl!

222 0

!85

!73

1(,4

162

17()

1g1

0

0

0

0

0

0

0

l'I

125

137

154

179

0

0

0

92

" 117

©

[I)()

1411

172

193

!6)

1-l-b

!33

!23

0

0

©

0

0

0

1)6

131

1111

%

87

.),)

llJ7

2117

0

0

0

0

0

0

0

0

0

0

0

0

1(,11

126

98

79

64

58

)4

62

71

911

! 12

141

" 77"

0

0

0

0

0

0

0

0

55

41

35

.11

37

65

91

118

0

0

I Ti

!.1)

0

0

!58

119

1()2

84

611

"

48

0

0

0

0

0

0

0

0

0

.W

24

17

15

20

.10

49

7)

1111

0

0

0

0

0

0

0

0

0

0

0

0

194

148

108

75

)0

28

13

6

4

8

11

38

(1:l,

9.1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

186

!42

10.,

69

+3

22

9

I

()

:i

16

32

85

0

0

0

0

45

25

II

" " Ill " 23

56

0

0

0

80

" " :i~

.1.1

[l)

12

0

0

0

0

0

94

67

37

27

G>

0

0

0

190

144

1116

7.1

0

0

0

0

!lJlJ

L'i2

11.1

" 2111 "

0

0

Jf,/

12X

0

"

47

" "

0

0

0

18

36

)9

88

0

0

0

0

0

14

26

42

66

97

0

0

0

0

0

29

40

)7

81

2

Ill

0

0

0

0

0

0

0

0

0

0

0

11'13

l'tl)

11:.::;

89

6(')

)3

46

,H

)!

61

78

l)l)

132

0

0

0

0

0

0

0

0

0

0

214

17.~

1.19

"

0

82

74

70

76

0

116

l))

0

0

@

0

0

0

20.'i

176

l'iO

1,11

114

1117

0

"

86

0

•

J()l)

120

104 t:N

l'i7

0

0

0

161

191 0

0

0

0

0

0

0

0

0

0

2 l)

IK4

!(19

1)3

14:i

14.1

151

])l)

17K

2U2

0

0

0

0

@

0

0

0

0

0

2l l

!lJ:2

IK8

!96

204

0

0

0

0

0

10:'i

0

[Bil 221 0

0

!55 198 0

0

!38 182 0

0

1"7 171 0

0

in !63 21.1

"

0

0

"

0

124 !66 134 174 ID

0

147 187 0

168 0

136

.,

208

0

2{ll) 0

19) 0

Signal constellation

FiGURE i8.35: The V.34 TCM system encoder and signal constellation.

y

Section ·i s.s

0f these encoder The constituer1t 2-·D

~ilultidimensional TCM

W55

with the and v< 2 l enter the multi-D constellation is divided into 4 and thus there 2 U>J-o""'W• The four bits v
l.

CH1

of

:m +0

ll

The size of the constituent

2-[)1

CJEJR. r::on_steHation s}ze of L!.l.'.!-8 vvith 17 g_o '.JO% due to

gain. In this with an uncoded of 75% constellation w•v•-•uc,~,,., was included in the standard 1--'"'''°''"'"" up an additional 0.8 dB than = 1.75

linear 4-D code used m the V.34 standard has and achieves a BER of 10- 5

witb uncoded 256-QAM (17 = 8.0). (Note that the smaller free distance with the 8--state 'V.32 code does not translate into less ::inr:e the uncodecl in and tis i:ion to modern in [44]. A more thorough discussion of the of the V.34 modem can be found in [45. 0

r.~

00

6

'~'"

n

,. __ -----·-------.. ~ - - ~ - - - - -

..

-

------- - ~ ~ - --

------~-'""""'~.,,.,....==-"------- -- - - - - -

cocks quickly dispelled the belief that poweiwith a decrease in bandwidth •-H.>v,cv>.", l our choice of modulation to BPSK. This was a welcmne ,Nho had been frustrated in their attempts to go Since the International Telecommunications Union's ITU-T

1056

Chapter 18

Trellis-Coded Modulation

V.29 modem standard was adopted in 1976 little progress was made in increasing the speed and quality of data transmission over voice-grade telephone lines until the appearance of the V.32 and V.33 standards in 1986 (see Example 18.14). The V.29 standard used uncoded 16-QAM and a 2400 symbols/second signaling rate to achieve a spectral efficiency of 17 = 4.0 bits/symbol and a transmission speed of 9600 bps in a half-duplex ( one-way) mode. Owing to the bandwidth constraints of the channel, signaling rates higher than 2400 symbols/second were not considered feasible. Thus, the only avenue to increased data rates was to expand the size of the signal constellation; however, because of the SNR constraints of the channel, this meant that signals had to be packed closer together, resulting in degraded performance. Thus. a clear need developed for a scheme that could allow constellation expansion at the same signaling rate, thus achieving higher data rates, and yet provide a coding gain to at least recover the noise margin lost by the closer packing of signals. TCM proved to be just such a scheme and, combined with some sophisticated signalprocessing techniques, has resulted in a series of improvements that have pushed modem speeds to 56 Kbps.

PROIBllEMS l!U Prove equation (18.8). 18.2 Find, as functions of the parameter d, the AEWEs L',~(X) and the MEWEs 8~(X) for the two signal set mappings shown in Figure P-18.2, and determine if they are uniform. Assume each constellation has unit average energy. 18.3 Determine if an isometry exists between the subsets Q(O) and Q(l) for the two signal set mappings in Problem 18.2. 18,4 Use Lemma 18.1 to prove that for uniform mappings, Aav(X) can be computed by labeling the error trellis with the AEWEs and finding the transfer function of the modified state diagram. 18.5 Construct a counterexample to show that Lemma 18.1 does not necessarily hold for rate R = k/(k + 2) codes. State a rate R = k/(k + 2) code lemma, similar to Lemma 18.1, specify the conditions for uniformity, and prove the lemma.

2d

010 I

I

fJ

/

001

I

"

/

I I

" ,.,_ 111 \

d

000 y - ".

\

I I

\

I I

\

I I I

I

2d

I

,

011

I

\

I

\

100 "\,

\

//)' 101

(a)

\

Id

][;., , I/,,;; \

I

010 "',_ -)'011

(b)

FiGURE P-18.2

/ 2d

/ I

Problems

11JJ5'?

011

010 /// /

--0-' , 001

a

0

''

I I

I

\

\

I

\

I

I

110 ~

'/ 000 I I

I I I

100

111

101

FIGURE P-1

l~tf Determine the AE\VEs and the MEWEs 8~(X) for and 4-AM and show that they are both uniform mappings. Jlllio Consider mapping a rate R = 2/3 convolutional code into 8-AM using natural mapping. filL Determine the AEVi/Es Li~(X) and the MEWEs 8~(X) for this mapping. li:Do Determine if the mapping is uniform. CCo Find the gain ( or loss) y for the three 4-state. rate R = 2/3 convolutionai codes of Example 18.4 with uncoded QPSK. rnl Can you find a rate R = convolutional code with a better coding when used with naturally 8-AM? JUU3 Show that the mapping of the 8-PSK signal set shown in Figure P-18.8 is not uniform. JLfliotlll 18.4, finding the MFSE distances and gains for three rate R = 2/3 trellis-coded 8-PSK if natural is replaced the uniform mapping of the results with natural mappmg. ]J:lJLt!D 18.5 finding a to the rate R = k/(k + 1) code lemma for the nonuniform signal set mapping in Problem 18.2(a). JL~LllJl Repeat finding the hAFSE distances and for three rate R = 2/3 trellis-coded 8-PSK if natural the nonuniform Gray-mapped 8-PSK signal set in Problem 18.8. this case, since the rate R = k / (k + 1) code lemma is not satisfied. the distances between all possible path pairs must be considered.) Compare the results with natural mapping. UU_Z Show that set partitioning of the infinite two-dimensional integer lattice 7!} results in a regular mapping. iUtB Apply mapping by set partitioning to the 32-CROSS signal constellation and determine the error vectors(!; for which (18.26) is not satisfied with ll[iUAl Construct an example in which (18.25) and (18.27) do not the same result. Utll§ Apply mapping set partitioning to the 8-AM signal constellation and determine the MSSDs t:,}, i = 0, 1. 2. Find the asymptotic coding gain JI and the average number of nearest neighbors when the 4-state code of Table 18.6(a) is applied to 8-AM. Repeat for the one-dimensional integer lattice Z 1 . ll[U<6 Compute, as functions of the parameter cl, the asymptotic coding gains of the 16QAM codes in Table 18.6(b) compared with the following uncoded constellations:

1Cl58

Chapter 18

Trellis-Coded Modulation

r

r; r;~

I

0

y~

0

t

0

1 &7

·;

O0/

L

<

0

Q'~

(i

"'

0

/

(b)

(a) FIGURE P-18.16

18.17 llU.8

UU9 18.20

18.21

18.22

18.23

(i) the 8-CROSS constellation shown in Figure P-18.16(a) and (ii) the 8-QAM constellation shown in Figure P-18.16(b ). Assume each constellation has unit average energy. Calculate A~1v(W, X) for Example 18.9. Apply mapping by set partitioning to the 8-QAM signal constellation shown in i = 0, 1, 2. Find A~v(W, X) and Problem 18.16 and determine the MSSDs Afv(W, X) for the code of Example 18.9 using this constellation. Let l(D) = 1 + D + D 2 + D 3 + · · · in Example 18.10 and recalculate (18.37) and (18.40). Are the conditions for rotational invariance affected? Derive general conditions on the number of terms in !Jn(l) (D) and Jln(O) (D) to satisfy (18.46). Show that (18.52) is still satisfied when the rotated binary sequences for naturally mapped QPSK given in (18.38) are substituted into the equation, and Jln(OJ(D) has an odd number of nonzero terms. Verify that (18.52) is satisfied for the encoder of (18.53) when the rotated binary sequences for naturally mapped QPSK given in (18.38) are substituted into the equation. Find minimal encoder realizations for the 90° rotationally invariant v = 4 and v = 5 nonlinear rate R = 1/2 codes based on the parity-check matrices

6.7,

IHI(D)

=

[(D

3

+ D)/(D 4 --1- D + 1)

1]

+ D)/(D 5 + D 2 + 1)

1],

and IHI(D)

=

[(D 4

respectively. Show that the v = 5 case cannot be realized with 32 states. 18.24! Derive general conditions on the number of nonzero terms in fu!( 2l(D), !hi.( 1)(D), and Jln(O) (D) to satisfy (18.59). 18.25 Show that the 45° rotated binary code sequences for naturally mapped 8-PSK are given by v?\D) = vC 2l(D) EB v(ll(D) o v< 0l(D), v?J(D) = vCl\D) EB v< 0 >(D), and v[0\D) = v< 0 l(D) EB l(D). 18.26 Show that (18.61) is still satisfied when the rotated binary sequences for naturally mapped 8-PS!( given in Problem 18.25 are substituted into the equation, 2.nd !bJ.COl (D) has an odd number of nonzero terms.

Jlft27 Show how

can be revn-ictten to

n1

realization sh'.J-;_,vn in JU3,2g Use the method of Euclidean

to the encoder

to show that

= (1 5).

R = 2/3, trellis-coded 8-PSK Prove that the JW3!fD Draw the ~n.Yc~nn x 8-PSK Jl~t31 Draw the

1 JlD< l

rate

= (12), and

1_§,,29

18.30, for the three 3

rn. (@o. @1. Tlli2. · · ·)

= (1011001. 0001110. llOHlO. ···).and

sequence fill = encoder starts in the

all-zero state. 17 =

18'19 to dete,:mine the rotationai invariance of the 2.33 encoders listed in Table 18.15(a). and find the AIOWEFs in ~'''""'-"v and uncoded and estimate

and the 8-st2,te r:ocle listed in Table with q = 0. r",~'.H"'''"'·"' a distance of d bet"vveen signal calculate the average of the 18.34 and of its 4-D version as functions of d, and compute the ilg,31(i Assuming a distance of d between signal with and the PA Rs of the 1-D signal sets and 8-AM as functions of d. the rate R = 2/3 matrix of Jl.337 Draw the encoder to the encoder in Figure 18.35. (18.1

"IEEE Trans.

3, G. I:

,t

''Trellis-Coded Modulation with Redundant Signal Sets. Part "IEEE Commun. , 2.5: 5-11, 1987.

, "Trellis-Coded Ivfodulation with Redundant Signal Sets. Part U: State of the

"IEEE Commun.

, 25:

1987.

5, J. L. 1974 Zurich Seminar on March 1974. (Di,

J. B. Anderson and D. P. Codes," IEEE Trans.

"A Bandwidth-Efficient Class of Signal Space IT-24: November 1978.

1060

Chapter 18

Trellis-Coded Modulation

7. G. D. Forney, Jr., R. G. Gallager, G. R. Lang, F. M. Longstaff, and S. U. Qureshi, "Efficient Modulation for Band-Limited Channels," IEEE .r. Select. Areas Commun., SAC-2: 632-47, September 1984. 8. A. R. Calderbank and J.E. Mazo, "A New Description of Trellis Codes," IEEE Trans. Inform. Theory, IT-30: 784-91, November 1984. 9. A. R. Calderbank and N. J. A. Sloane, "New Trellis Codes Based on Lattices and Cosets," IEEE Trans. Inform. Theory, IT-33: 177-95, March 1987. 10. G.D. Forney, Jr., "Coset Codes-Part I: Introduction and Geometrical Classification,'' IEEE Trans. Inform. Theory, IT-34: 1123-51, September 1988. U.

, "Coset Codes-Part H: Binary Lattices and Related Codes," IEEE Trans. Inform. Theory, IT-34: 1152-86, September 1988.

12. D. Divsalar and M. K. Simon, "The Design of Trellis Coded MPSK for Fading Channels: Performance Criteria," IEEE Trans. Commun., COM-36: 1004-12, September 1988.

B. - - - - , "The Design of Trellis Coded MPSK for Fading Channels: Set Partitioning for Optimum Code Design," IEEE Trans. Commun., COM-36: 1013-21, September 1988. 14.

, "Multiple Trellis Coded Modulation (MTCM)," IEEE Trans. Commun., COM-36: 410-19, April 1988.

15. C. Schlegel and D. J. Costello, Jr., "Bandwidth Efficient Coding for Fading Channels," IEEE J. Select. Areas Commun., JSAC-7: 1356-68, December 1989. 16. R.H. Deng and D. J. Costello, Jr., "High Rate Concatenated Coding Systems Using Bandwidth Efficient Trellis Inner Codes," IEEE Trans. Commun., COM37: 420-27, May 1989. 17.

, "High Rate Concatenated Coding Systems Using Multi-Dimensional Bandwidth Efficient Trellis Inner Codes," IEEE Trans. Commun., COM-37: 1091--96, October 1989_

18. S. L. Goff, A. Glavieux, and C. Berrou, "Turbo-Codes and High Spectral Efficiency Modulation," in Proc. IEEE Int. Conf Commun. (ICC 1994), pp. 645-49, New Orleans, La., May 1994. J19. S. Benedetto, D. Divsalar, G. Montorsi, and F. Pollara, "Bandwidth Efficient Parallel Concatenated Coding Schemes." IEE Electron. Lett., 31: 2067-69. November 1995. 20. P. Robertson and T. Worz, "Coded Modulation Scheme Employing Turbo Codes," IEE Electron. Lett., 31: 1546-47, August 1995. 21. E. Biglieri, D. Divsalar, P. J. McLane. and M. K. Simon, Introduction to TrellisCoded Modulation with Applications, MacMillan, New York, 1991.

Linea,· Octobe;.·

"'Modulation and IT-44: 1998. 1

Uniform 1991.

IEJEE

~

JE]EE

1

rans.

',N,JlL '·"On the Performance Evaluation of Trellis IT32: 196-202. March 1987.

the Distance lEEJE ]. Select A\.reas Cor:unun.~ JS/\(:,-7:

November 1994.

I'T-40:

Invadant Convolution.al Channel '\Nith Pcirt Jf: 180°:' IEEE J. Select. Areas Commun., SAC-2:

Convolutional

Channel '' IEEE

with

Select. Areas

1984. l_PE'JS

""Trellis-Coded f,;fodulation with Multidimensional IT-33: 1987. ""A. Nevi Multilevel

Tmns.

Method ErrorIT-23: 371-77, May 1977.

'·"I
3!t L. F.

Invariant Trellis-Coded Modulations with MultidimenSelect. Are[ls Commun., SAC-7: 1281-95. December

1989. lVI. form Partitions of L rel="nofollow">< MPSI( Constellations and Related Binary Trellis IEEE Trans. IT-39: 1773-98, November 1993.

Hl62

Chapter 18

Trellis-Coded Modulation

37, M. D. Trott, S. Benedetto, R. Garello, and M. Mondin, "Rotational Invariance of Trellis Codes. Part I: Encoders and Precoders," IEEE Trans. Inform. Theory. IT-42: 751-65, May 1996. 38. S. Benedetto, R. Garello, M. Mondin, and M. D. Trott, "Rotational Invariance of Trellis Codes. Part II: Group Codes and Decoders," IEEE Trans. Inform. Theory, IT-42: 766-78, May 1996. 39, S. Benedetto, R. Garello, M. Mondin, and G. Montorsi, "Geometrically Uniform TCM Codes over Groups Based on L x MPSK Constellations," IEEE Trans. Inform. Theory, IT-40: 137-52, January 1994. 44). G.D. Forney, Jr., and L. F. Wei, "Multidimensional Constellations. Part I: Introduction, Figure of Merit and Generalized Cross Constellations," IEEE J. Select. Areas Commun., SAC-7: 877-91, August 1989.

41. G.D. Forney, Jr., "Multidimensional Constellations. Part II: Voronoi Constellations," IEEE J. Select. Areas Commun., SAC-7: 941-58, August 1989. 42, A. R. Calderbank and L. Ozarow, "Nonequiprobable Signaling on the Gaussian Channel," IEEE Trans. Inform. Theory, IT-36: 726-40, July 1990. 4\3, G. D. Forney, Jr., "Trellis Shaping," IEEE Trans. Inform. Theory, IT-38: 281-300, March 1992. 44!. R. Laroia, N. Farvarclin, and S. A. Tretter, "On Optimal Shaping of Multidimensional Constellations," IEEE Trans. Inform. Theory, IT-40: 1044-56, July 1994. 45. M. V. Eyuboglu, G. D. Forney, Jr., P. Dong, and G. Long, "Advanced Modulation Techniques for V. Fast," Eur. Trans. Telecommun., 4: 243-56, May 1993. 46, G. D. Forney, Jr., L. Brown, M. V. Eyuboglu, and J. L. Moran, HI, "The V.34 High-Speed Modem Standard," IEEE Commun. Mag., 34: 28-33, December 1996.

block and channel sets to construct bandwidth-efficient codes is referred to as block coded modulation (BCM). Codes constructed combination are called BCM block coded to TCJVJ[ codes. BCM codes are much easier to construct (or than TCM codes. The most method for BCM codes is the multilevel devised by Imai and Hirakawa in 1976 This is devoted to the nmltilevel construction of BCM codes and of these codes. Also are concritenated coded modulation and noc,,ri,,,<", and high efficiencies vvith reduced and multilevel coded modulation for error

Y(s)) in a two-dimensional Euclidean space with x- and and Y(s), Lets and s 1 be two in . The Euclidean (SE) distance betweens and s1, denoted dy,:(s. s 1). is defined as X(s)

(s) - X(s1))2

Let v the

=

+ (Y(s) -

Y(s1))2.

= (s0. s~ ..... s;1 _ 1) be two Euclidean distance between v and ,,/. denoted

(so. s1 ..... s11 _1) and v 1

over v 1 ), is defined as

11-l

ii(v. v1 ) ~ L(X(s;) - X(s;)) 2 + (Y(s;) - Y(s;)) 2 . i=O

Let S be a two-dimensional modulation set(or as a (X(s). Y(s)) in . An 11-tuple over S is sequence of n signals from S. A BCM code C of n over the signal space S is a collection of 11-tuples over S. The minimum Euclidean (MSE) distance of C denoted C]. is defined as s E Sis

c!H

The mmmrnm Hamming distance of the code, dH[C]. is also called the minimwn distance of the code in coded modulation. For two codewords v = (so. s1 ..... s,,_1) and v1 = (s 0, s~ . ... , S:,_ 1) in C the product distance between v and,/. denoted d~ ,/), is defined as 11-l )

/

Li

c/p(V,"o/)=

Il

WM

Chapter 19

Block Coded Modulation

The minimum product distance of the code, cl~ [ C], is defined as /1() ']\

~"7,.JJ

For an A WGN channel, the error performance of a modulation code depends primarily on its MSE distance and path multiplicity. For the fading channels, such as Rayleigh or Ricean channels, the error performance of a modulation code depends primarily on its minimal symbol and minimal product distances and path multiplicity (nearest neighbors) [29]. H depends on the MSE distance to a lesser degree. These distances are called the distance parameters of the code. H every message of k information bits is encoded into a codeword in C, the spectral efficiency of the code is 17[ C]

k

= - bits/symbol. n

Assume that the channel is an A WGN channel and all the codewords are equally likely to be transmitted. Let Ir= (xo, Yo, x1, Yl, ... , x 11 _1, Yn-1) be the output sequence of the receiver demodulator, where x; and y; are the x- and y-coordinates of the ith received signal r;. For maximum likelihood decoding, the received sequence Jr is decoded into a codeword v* such that

for v; f- v*. 19.2

Ml\JlTillEVIEl IBlOC~ MODUlATiOi'J CODIES Multilevel coding [1] is a very powerful technique for constructing bandwidthefficient modulation codes systematically with arbitrarily large distance parameters from Hamming distance component (block or convolutional) codes in conjunction with a proper bits-to-signal mapping through signal set partitioning. H provides the flexibility to coordinate the distance parameters of a code to attain the best performance for a given channel. Furthermore, the multilevel codes constructed by this method allow the use of multistage decoding procedures that provide good trade-off between error perforrnance and decoding complexity. Multilevel coding is best explained by constructing a class of 3-level BCM codes over the 8-PSK signal set with unit energy. For bits-to-signal mapping, each 8-PSK signals is labeled by a sequence of 3 bits, aoa 1a2. based on the 3-level binary partition chain 8-PSK/QPSK/BPSK/{O}, as shown in Figure 18.12. For convenience, the partition chain is reproduced in Figure 19.1. (Note: The order of bit labeling given here is the opposite of that given in Figure 18.12.) Let Q(ao) denote the set of signal points in S whose labels have ao as the prefix. Q(ao) forms a QPSK. Let Q(aoa1) denote the set of two signal points in S whose labels have aoa1 as the prefix. Each Q(aoai) is a BPSK. Let Q(aoa1a2) denote the set that contains the signal point sin S labeled with aoa1a2. The intraset distances .6.5, .6.f, and .6.3 of S, Q(ao), Q(aoa1), and Q(aoata2) are 0.586, 2, 4, and oo, respectively. The signal set partition and signal labeling process define a one··tO· one mapping f (") that maps each label aoa1a2 into its corresponding signal points; that is, f (aoata2) = s.

.6.i,

Multilevel Block 1\/lodulation Codes

Section 1

1(Q)i'i5

s SFSK

!)

0

q__

7

I f - - - Lln = 0.586

O 0 O 0

ao=

QPSK

0

Q(l) r, 0

C

00 0

c5

O

0

0

0

0

Q(Ol)

Q(OO)

O 0

BPSK o

Q(O) 0 O 0

o

0

0 () 0

0 O 0

a,=

o/

0 O O

0 I)

0 0

0/

0

0

0

0

0

0

o

o

0

0

0

0

o 0

000

000

000

000

000

000

001

010

Oil

WO

o

0

() 0

o 0

0

000

iOl

\

0

0

0 0

For O :s i :s 2, let dimension k;, and JT1inimum

(n, k;.

0

0

0

0 O O

O O 0

110

111

chain and

FIGURE 19.1:

~

'\1

/

o

Q(ll)

0 "'- ~O / _ I 0 ~---,._1_,-L.O. -

0 o'·)o

0

Q(lO)

O O O 0

0

Clf = 2

·~--- -

labeling.

linear block code of

Yo= (vo.o. vo.1- .... vo.J, .... vo.11-1).

= ( v1.o, V!.] . .... VI.j, . • . , V1.11-l),

o/ j

W2 °= (V:2.0· V2.l• .... V2.j, .... v2.11-1).

be three codevvords in Co, 6

vo

* Y1 * Y2 =

'We form the

. and

sequence:

(vo.ov1.ov2.o, .... vo.Jv1, 1v2.J- ... , vo.11-1v1.11-1v2.11-1).

This sequence is obtc1.ined the three codewords vo, v1. and w2. For O :S j < n. we take vo i VJ.j v2.1 as the label for a in the 8-PSK signal space S.

6

-

C=.flCo*

=

{f(,Yo

(19.6)

* YJ * v2): w;

EC;

for O :s i

:s 2}.

n and dimension k = ko + sequences. Because ko + lq + k2 information the spectral efficiency of bits are encoded into a code sequence of n 8-PSK the code is ko + /q + k2 11cc r ] = ----C is a 3-level 8-PSK modulation code of

ki + k2. The code consists of

11

1066

Chapter 19

Block Coded Modulation

I

Encoder for C0

Vo

I

f(Vo*V1*

I

I

Encoder for C 1

Encoder for C2

Vt

~

I

Signal mapper

V, ~

I

FIGURE 19.2: An encoder for a 3-level BCM code.

In this construction of a 3-level 8-PSK code. each binary component code contributes a labeling bit. An encoder for such code is shown in Figure 19.2. In encoding, the first component codeword vo = (vo.o. vo.1 .... , vo. 11 -1) simply selects a sequence of n QPSK signal sets (Q(vo.o). Q(vo.1), ... , Q(vo_,,-1)). From this sequence of QPSK signal sets. the second component codeword v1 = (v3.o. v1.1, ... , v1.,,-1) selects a sequence of 11 BPSK signal sets ( Q( vo.ov1,o), Q( vo.1 vu) ..... Q( vo.n-1 v1 _,,_i) ). where for O :S j < n, Q(vo.Jv1.J) c Q(vo.J),

Then, the third component codeword v2 = (v2.o. v2.1, ... , v2. 11 -1) selects a sequence of n signal points (Q(vo.ov1,ov2.o), Q(vo.1 vu v2.1 ), .... Q(vo.n-1 v1, 11 -1 v2. 11 -1)) from the BPSK sequence (Q(vo.ov1.o), Q(vo1 v1,1) .... , Q(vo.n-1 v1. 11 -1)), with Q(vo.JVI.j v2.J) E Q(vo.JV!.J) for O :S j < n. This sequence is the output signal sequence. With this view of the encoding operation, the encoder of Figure 19.2 can be reconfigured as shown in Figure 19.3. The MSE distance of a 3-level 8-PSK BCM code is given by Theorem 19.1. 19,]_ The MSE distance di( C] of the 3-level 8-PSK block modulation code C = f[Co * C1 * C2] defined by (19.6) is lower bounded as follows: THEOREM

d~[C]::::: min{0.586 >< do, 2 x c/1. 4 x c/2).

Encoder for C0

l

-~1»1111

1112

11

1---v-

-

(19.7)

8-PSK/QPSK

Encoder for C 1

~

---L_~cler for C2 FIGURE 193: Another encoder configuration for a 3-level 8-PSK BCM code.

Seci:ion 19.2

Multilevel Block Modulation Codes

'HJJ67

* vS) be two different signal sequences Let f (vo * 7J * distance bet,Neen these two in C. le deter1r1ine the sequences. we must consider three cases. For the first case, suppose vo / Because the minimum disitance of ,s ,10 ancl must in at least do At each of the labels of the in f (vo * v1 * and differ in the first Because f (-) is a one-to-one f (-c."o ,:, 011 * v2) and f (Y; * Y~ * must differ in these In the whose labels differ cc:.1steHation with unit energy. two a Euclidean distance of at least =::: 0.586

X

do.

For

and v1 / v~. Because the minimum is ., ,1 1 w~ must differ in at least cl1 A1 1he bbels of the m * v1 * , 2) and are identical at the but differ in the These two in the 8-PSK constellation must be Euclidean distance of at least t, = 2. the Euclidean distance between f (vo * ¥1 * v2) and f (v; * * be 8\t least 2 >< di: that 1

i

·<

The last case is that vo = wS, v 1 = v~, and ,v2 / . Because the minimum distance of is d2. v2 and w~ must differ at least d2 places. This * 'fJ * v2) and f (v; * ;~ * differ in at least d2 /'\t the labels of the two corresponding are identical at 'Lhe first ~cNo bits but differ in the third bit These two n1ust be set and by a squared Euclidean distance =4.

we have

This result

the bound of (19.7).

vve want to construct a 3-level 8-PSK BCM code C of length 11 = 8, with MSE distance C] = 4 and efficiency 17 [ C] = 2 bits/symbol. It follows from Theorem 19.1 (19.7) that the minimum Hamming distances of three codes must be 8, 2, and 1, respectively. To achieve 2 the sum ko + k 1 + k2 of dimensions of three component codes must be 16. Under these the three binary component codes may be chosen as foHmvs:

di [

1068

Chapter 19

Block Coded Modulation

lo Co is the (8, 1, 8) repetition code that consists of the all-zero and all-one vectors.

r, .._, ! ic thP (8 7 2\j "'""'Il .._, .._.,J.. - par1'ty_r'hPr-lT {r.r QPI') "Ori" that ""Ilc;ctc ,.f ~11 the> even-weight 8-tuples over GF(2). 3. C2 is the (8, 8, 1) universal code that consists of all the 8-tuples over GF(2).

1. =~

..1.u

-"-

"-'

'

Then, C = f[Co distance

I

l/

'

I.

',.,J....11..._,'-'Jl'-

\ V.L

\..JI.IL

'-..,,

\.I

UV

.Ll

\,.,V

0J.0 l-0

V.i.

U.l..l

LJ..J.\,.,

* C1 * C2] is a 3-level 8-PSK BCM code of length n = 8 and MSE d}[C] ::". min{0.586 x 8, 2 x 2, 4 x 1} = 4.

= 4 (see Problem 19.1). Because ko + k1 + k2 = 1 + 7 + 8 = 16, the spectral efficiency is 17[ CJ = 16/8 = 2 bits/symbol. This 8-PSK BCM code has the same MSE distance and spectral efficiency as the 8-PSK TCM code (code 2) given in Example 18.4. It achieves a 3-dB asymptotic coding gain over the uncoded QPSK system with the same spectral efficiency. This code also has a very simple trellis structure. To construct the trellis for C, we first construct the trellises for the component codes. The trellises for the three component codes are shown in Figure 19.4. Taking the Cartesian product of these three trellises, we obtain the trellis of the interleaved code Co * C1 * C2, as shown in Figure 19.5. Then, we map each branch label into an 8-PSK signal based on the partition and labeling shown in Figure 19.1. The result is the trellis for the 3-level 8-PSK code C shown in Figure 19.6. It consists of two parallel and structurally identical 2-state subtrellises. The code is phase invariant under multiples of 45° rotation [7). Its bit-error performance with Viterbi decoding is shown in Figure 19.7. It achieves a 2-dB real coding gain over the uncoded QPSK at a BER of 10- 6 .

In fact, 4[C)

0

0 0

0

(a) An 8-section trellis for the (8, 1, 8) repetition code C0

1

I

0 0 (b) An 8-section trellis for the (8. 7. 2) even-parity-check code C 1 0

0

0

0

0

0

0

0

1 (c) An 8-section trellis for the (8. 8.1) universal code C 2 FIGURE 19.4:

Trellises for the binary component codes.

Multilevel Block Modulation Codes

Section 1 101

101

100

100

'Hlll6i9

100

000

000

000

000

trellis for the interlecived code (8. 1. 8) * (8, 7, 2) * (8, 1. 1).

l'IGURE 19.5:

J

J

0

FIGURE 19.6:

(8, 1. 1)].

0

0

0

0

0

Eight-section trellis for the 3-level 8-PSK code f[(8, 1. 8)

* (8, 7, 2) *

1070

Chapter 19

Block Coded Modulation

__.QPSK _,,_ · Simulation

""

' ' ' ",

''

"'

' ' .,,

'

"'

'' ''

\

''

'\

10- 6 ~~~~~~~~~~~~~~~~-+-~~~~~-
3

4

5

6

7 E;,/N11 (dB)

8

9

10

11

FIGURE 19.7: Bit-error performance of the 3-level 8-PSK BCM code ![(8, 1, 8)

(8, 7, 2)

* (8, 8, 1)].

*

The preceding example shows that Theorem 19.1 (or (19.7)) can be used as a guideline for constructing good 3-level 8-PSK BCM codes for an A WGN channel. For a given MSE distance, the three component codes should be chosen to maximize the spectral efficiency and minimize the decoding complexity. The other two distance parameters of a 3-level 8-PSK BCM code are given in Theorem 19.2 [9]. THEOREM 19.2 Consider a 3-level 8-PSK BCM code C = f[Co * C1 * C2] with component codes Co, C1, and C2, whose minimum Hamming distances are do, d1, and d2, respectively. Let Ll5, and Ll~ be the intraset distances at the three levels of the partition chain 8-PSK/QPSK/BPSK, respectively. Then, the minimum symbol distance dH [ CJ and the minimum product distanced} [CJ are given by

Lli,

1. dH[CJ = min{d; : 0 :": i :": 2},

(19.11)

2, Let q be the smallest integer in the index set I dq = dH[C]. Then,

{O, 1, 2} for which (19.12)

Proof The proof of the first part, (19.11)~ is similar to the proof af the l\ASE distance d1[CJ given in Theorem 19.1.

Multilevel Block Modulation Codes

Section 19.2

1(Jn

The of the second of the theorem is as follows. For O :S j :s 2, () () let ,(i) = ( v 0.1 • v 0.1 • • • • • denote a codeword in the code Because q is the smallest in I = {O, L 2} for which dq = dH[C], there and ,'y(ql. in Cq such that the distance must exist two behveen them is Consider two ;,; = f(v< 0l*,,,
distance betvreen x and

•x' -

l2p

C

Le} -

(!J,2,t1,1 'J} .

that [C] < ( !J,~ /,1. In the ( L\~ )11,1 cannot be true.

This

and in C such that dH

we prove that d~[C] <

we assume q = 1. do > d1, and d2 :=:: d1. there exist two signal sequences.

Without loss of d~[C] <

Y=

xis

f(i/Ol

y) =

* y
(.vo.

· · ·. S·,,-1).

d1 = dJ-I[C] and 11-l

2(

/ Cp

,2 ( , "!J.:f") = r1 Jl J1 CE y;,_V; < A

)

(

2)dl Lll . ,\

i=O

-1",i'-1'1 This

that there is a j such that c!}(YJ· _vJ) <

Because Yi= can

!},I-

(19.14)

2 1 f(i/I 0li}lli} l) and vi = f(f} °lv<. liP\ the inequality of (19.14) .I .I .1 J I

that i}Ol / v(O) and w
* w
y) ::': dH('j, 7l).

we

(19.15)

Because ,/Ol # v(Ol, dH(:I, 7l) :=::do.It follows from (19.15) that

y) :". that clH(:I, y) = dH("!J,

> c/1,

which contradicts the fact c/1. Hence, the c1i[c] < Cf._I)"1, does not hold, and we must have di[c] = (nfl'1 1. This concludes the (QJ,JEJI]),

1072

Chapter 19

Biock Coded Modulation

EX.AMPllE Hl.2 Suppose we want to construct a 3-level 8-PSK code with minimum symbol distance 4, minimum product distance of at least 4, and spectral efficiency around 2 bits/symbol. From (19 .11) we find that the smallest minimum Hamming distance of the component codes must be 4. From (19.12) we find that for a minimum product distance of at least 4, the first component code cannot be the code with the smallest minimum Hamming distance. In this case, we should choose either the second or the third component code to have the smallest Hamming distance. Possible choices of the component codes are

1. Co is the (32, 16. 8) second-order RM code of length 32. 2, C1 is the third-order (32, 26, 4) RM code of length 32. 3, C2 is the same as C1: that is, C2 = C1. The resultant 3-level 8-PSK code, C = f[ Co * C1 * C2], has the following distance parameters: dHc] = 4.688, dH[C] = 4, d~[C] = 16, and 17[C] = 2.125 bits/symbol. The construction and developments of 3-level 8-PSK codes can be generalized to multilevel codes over any M -ary PSK or QAM signal set. Let S be either an MPSK or a QAM signal space with i signal points. We form a binary partition chain S/Si/· · · / S1 for the signal space S, where S1, S2, · · · . Si = {O} are subspaces of S, and S; consists of i-i points for 1 :s i :s !. For O :s i :s l, let t:,,7 denote the intraset distance at the ith level of partition. For i = 0, t:,,6 is the intraset distance of the signal space S. The partition of Sis carried out such that the intraset distances increase monotonically, that is,

!':,,6 :s

ti.f :s ... :s /':,,f.

(19.16)

Based on this partition chain, each of the i signal points in S is labeled by a unique binary string of length /, denoted by aoa1 · · · G/-1 · This signal labeling defines a one-to-one bits-to-signal mapping f (,). For each label aoai · · · a1-1, f (aoa1 · · · ar-1) is its corresponding signal. For 1 :s i :s l, let Q(aoa1 · · · a i -1) denote the set of l1-i signal points in S whose labels have aoa1 · · · ai-1 as the common prefix. The intraset distance of Q(ooa1 · · · ai-1) is t:,,7An /-level BCM code over a signal space (MPSK or QAM) with l1 signal points is constructed in the same manner as a 3-level 8-PSK BCM code. For O :s i < l, let Ci be a binary (n. ki, d;) linear block code of length n. dimension ki. and minimum Hamming distance di. Let wo w1

'¥/-]

(vo.o, vo.1, · · · . vo.J, · · · . vu.··· . v1.J, · · · .

(v1,o.

(v1-1.o, V/-1.l, ... • V/-1,j, ... '

vo.n-d, Vl,n-1 ).

V/-l.11-1),

be l codewords in Co, C1, · · · , C1_1, respectively. We interleave these l codewords to form the following sequence: wo

* w1 * · ·· * w1_1 = (vo.ov1.o · · · vi-1.0, · · · . vo.JVJ.J · · · V/-1,j, · · · , VO,ll-1 Vl.11-l · · · V/-i,11-1).

Section 1

For O::: j < n, we f(-10

1\i1ultilevel Block 1\/lodulation Codes

(f(vo.ov1.o · · · v1-1.o). · · · V/ - ] j ) • . . . •

•

e

1s a sequence -Ch n

f:mm the

r• 6 L

s ES.

vo.JVLj · · · V/-1.j as the label of a

* v1 * · · · * 1'1-1) =

W13

f(vo.Jvl.i · · ·

f (VQ. II -1 V l, 11 -1 ... V / -

\.11-

i))

space S. 'I'b.e:n~

* C1 * ·· *

= r

for O =:: i < !}

.c

=1.1

is an /-level BC'.IYl code ovt:r S dimension k = ko + /q + · · · + k1-l· Because all the ··,~u, 1,~,.,•••,,,." codes are En.ear. the interleaved code * * · · ·* is linear. For the BCM code C = f[Co * C1 * · · ·* C1_i] as a linear code. of this code are chan1cterized Themem which is of Theoren1s 19.1 and 19.2.

Let S be either an l\1PSI( or a space with l1 rmmmum [CJ be the MSE distance of an !-level BCM code over S.

JL

[CJ 2:

: 0::: i < l}:

·lr-] -- .. Tl , ,1en. (19.20) where l!'6, S/Si/ .. · /S1 = {O}.

L

are the intraset distances of the

The construnion of an /-level BCM code over a consists of four

chain

space S with

of I bits

ciJ.,

each interleaved sequence, vo * v1 * · · · sequence f(,'lo * v1 * · · · * v1-1) over S.

* v1_1,

at

l1 signal a

3 into a signal

An 11-section treHis for an /-level code, C = .f[Co * C1 * · · · * C1-1]. over Scan be constructed in two Fm O::: i
1074

Chapter 19

Block Coded Modulation

ith binary component code C;. We construct an n-section trellis T for the interleaved code Co* C1 * · · · * C1--1 by taking the Cartesian product of r< 0l, y(ll, ... , yU-l) and labeling each branch in T with l bits. These l bits form a label for a signal point in S. We map each branch label in T into a signal point in S. The result is an n-section signal-level trellis for the /-level code C = f[Co * C1 * · · · * C 1_1] over S.

EXAMP'llE 19.3 Suppose we want to construct a 4-level 16-QAM code with MSE distance 8!-.5 and spectral efficiency around 3 bits/symbol. The 4-level binary partition chain for 16-QAM is shown in Figure 19.8. For i = 0, 1, 2, = a possible choice of the component codes is

~7+i 2~7,

1. 2. 3. 4l.

Co is C1 is C2 is C3 is

the the the the

(16, 5, 8) first-order RM code of length 16. (16, 11, 4) second-order RM code of length 16. (16, 15, 2) SPC code of length 16. (16, 16, 1) universal code.

The resultant 4-level 16-QAM code, C = f[ Co * C1 * C2 parameters: CJ = 8~5, and 17[ C] = 2.9375 bits/symbol.

4[

* C3], has the following

Multilevel coded modulation systems can also be designed using convolutional codes ( or a combination of block and convolutional codes) as component codes.

..

~ "' .. ®

" .,

FIGURE 19.8:

.,

The 4-level binary partition chain for 16-QAM.

Section 19.3

Multilevel BCM Codes

U17!5

For short BCM maximum likelihood can be implemented based on their full code trellises a trellis-based The trellises of long BCM ;may be too for of any trellis-based hmvever, the multilevel structure of these codes allows the use of '-"•"'~v,a,rn,I', that an effective trade-off between error performance of a 1T1ultilevel BCM code, component soft-decision l\,fLD. as described in Section 15.3. The decoded information at each The •Nith the first-level '-'VVJ•J"''"'

0

~·~,-.~~

uv·,~·-'""''"'process is by using a 3-level 8-PSK BCM code C = f[Co * C1 * C2]. Assume an A'NGN channel. Let rr = (ro, ro, · · · , r11 _1) be the received sequence at the of the demodulator, where for O ::::c i < n

Let Yo= (vo.o, vo.1, · · , vo.n-1) be a code\;vord in Co. Let c!Hr;, Q(vo.;)] be the MSE distance between the ith received and the signal in Q(vo,;). The SE distance between the received sequence r and the codeword vo is defined as n-1 (

\ 6

,re, 70J

=

(19.21)

, Q (VO_;)]. i=O

For every codeword Yo E Co, we into the codeword vo = (vo o, concludes the

the distance cl~

7o)

and decode

ir

. Do 11 -1) for which d~(rr, vo) is minimum. This

The decoded at the is to the second stage. Let v1 = (v1.o, vu,···. v1. 11 -1) be a codeword in C1. Let dHr;, Q(Do.;vu)] be the MSE distance between the ith received symbol r; and the signal points in Q (Do.; v1.;). For every codeword Y 1 E C 1, we the distance

. Q(Do.;vi.;)].

(19.22)

i=O

Then, we decode Il' into the codeword v1 in This the 1 \9l,3L3l

for which ,4(Il'. vo

* v1) is minimum.

lflharn1-S'u:zil9Jre [lle«:rDlcclarul9J The decoded information at the first and second stages, w'o and v1_, is made available to the third For every codeword v2 E C2, we compute

1076

Chapter 19

Block Coded Modulation

the distance n-l

d~ £, (r,vo

* w'1 * v2) ~ /

2 "\' Q( Do I D1 ,· v2,) L=-1 dE fr,, L ,1 / l J i

,

,

,

i=O

(19.23)

11-l

= I>}[r;,

.f(Do,;Duv2,;)].

i=O

We decoder into the codeword v2 E C2 for which 4(r, vo * v1 * v2) is minimum. This completes the entire decoding process, and {vo, v 1, v2} forms the decoded set. A decoder is shown in Figure 19.9. The foregoing multistage decoding is known as the closest coset decoding. A trellis-based MLD algorithm can be used for decoding each component code. The entire decoding complexity is the sum of decoding complexities of the component codes. Owing to the possibility of error propagation from one stage to the next, the preceding multistage decoding algorithm is not optimum even though each stage of decoding is MLD; it is suboptimum. IEXAMPllE 19.4

Consider a 3-level 8-PSK BCM code, C = f[Co * C1 * C2], of length n = 32, where (1) Co is the first-order (32, 6, 16) RM code, (2) C1 is the third-order (32, 26, 4) RM code, and (3) C2 is the (32, 31, 2) even-parity-check code. The total number of information bits is k = 6 + 26 + 31 = 63. Therefore, the spectral efficiency of C is 17(C] = 63/32 = 1.96875 bits/symbol (almost 2 bits/symbol). From (19.7) we find

r

First-stage decoder

! Second-stage decoder

vo

v1

! Third-stage decoder

FIGURE 19.9:

V7

A three-stage decoder for a 3-level BCM code.

of I\Jlul·Lilevel [KIVI Codes

Section 19.3

'WJT?

clH

that the MSE distance of C is C] = 8. This 3-level 8-PSK code has almost the as the uncoded QPSK, 2 H achieves a 6-dB over the uncoded with MLD. code Co has a 4-section 16-state trellis. The second also has a 4-section 16-state trellis. The third code has a 32-section 2·~state tre11iso The Cartesian of the three trellises vvould L',A,._,u,u""· based on this overaH trellis i0 of this BCM code The bit-error of this is shown in Figure 19.10. There is a 3.6-dB over the uncoded QPSK at a BER of 10- 6 . real The ward manner to of an /-level BCM over a space S. For O :s; .J < L let

iro = (voo. Do.1. · · ·, ~l

vo11-1)

= (D1.o. Du.···. D111-1)

= (vJ-1.0. vi-1.l· · · ·

n-1)

·~~1·

QPSK 3-level 8-PSK·~,~:

10-h._t___ 0

2

4 Et,f/V11 (clB)

FIGURE 19.10:

31) ].

Bit-error~"'·,·~~""""

of the 3-level 8-PSK code f[(32, 6)

* (32, 26) *

1078

Chapter 19

Block Coded ~v1odulation

be the decoded codewords of the first j stages. For every codeword v.i vj,l, · · · . Vj.n-1) E CJ, we compute the distance

(vj.O,

(19.24)

where 4[r;, Q(vo_;vi.;, · · ·, v.i-l.iv.i,;)] denotes the MSE distance between the ith received symbol r; and the signal points in Q(vo,;v 1,;, ... , v1_uvj_;). Then, the (j + l)th stage decoding decodes ir into codeword 1 in C.i for which d7,:(ir, vo * v1 * · ·· * Vj-1 * VJ) is minimum. Decoding stops at the lth stage.

v

IEXAMIP'llE 19.5

Consider a 4-level 16-QAM BCM code, C = f[ Co* C1 * C2 * C3]. of length n = 16, where (1) Co is the first-order (16, 5, 8) RM code, (2) C1 is the second-order (16, 11, 4) RM code, (3) C2 is the (16.15, 2) even-parity-check code, and (4) C3 is the (16.16, 1) universal code. The total number of information bits is k = 5 + 11 + 15 + 16 = 47. Therefore. the spectral efficiency of C is 17[C] = 47 /16 = 2.9375 bits/symbol (almost 3 bits/symbol). From (19.18) we find that the MSE distance of C is di[C] = 81~'5This 4-level 16-QAM code has almost the same spectral efficiency as the uncoded 8-AM-PM constellation obtained at the first partitioning level of Figure 19.8, which achieves 3 bits/symbol. Hence, it achieves a 6-dB asymptotic coding gain over the uncoded 8-AM/PM with optimal MLD. The bit-error performance of this code with 4-stage suboptimum decoding is shown in Figure 19.11. There is a 2.8-dB real coding gain over the uncoded 8-AM/PM considered at a BER of 10- 5 . Note that although the 16-QAM and 8-AM/PM constellations considered in this example have the same average energy. this is no longer the case in general, since several 8-point QAM constellations can be chosen as an uncoded reference. As a result, a given 8-point QAM constellation with the same average energy as the 16-QAM constellation has to be considered as an uncoded reference, which automatically determines the SE distance of this 8-AM/PM constellation.

An upper bound for each stage of multistage decoding of muitiievel codes can be derived based on the union bound. First, consider a i -PSK BCM code, and assume that the all-zero codeword is sent at level 1. Then, a decoding error at stage 1 is made if there exists vo f I!]) with (19.25) Let w represent the Hamming weight of vo. Then, for each nonzero position of vo, each point in Q(O) has two nearest neighbors in Q(l) at SE distance Because at stage 1 of the decoding, it is assumed that any 11-tuple is a valid codeword for the remaining stages, v0 can be mapped into 2cv sequences at SE distance from any given sequence of 11 points in Q(O). Let A;~l represent the number of codewords of Hamming weight w in code C; used at level-i. It follows that the word-error

t..5. wt..5

of Muh:ilevel 3CM Codes

W7·9

0---

FlGURE 1 'i : JB.lt S1Tornp·,·1c;,;;'rn

06.

8

6 E 1,/N"(dB)

2

10

12

code f[(15. 5),:,(16, 11)*

of the i1.-level

,:, (16.

<

fa for 1 < i < l, each Q · · o; _ 10) has two nearest at SE clist;=mce · · · a;-11) H follows that for l :::: i < l, the word-e:uor of the is upper bounded

<

<

'_2'uAUlQ (

- Lc-d

Cu

-

co=d,

\I

+ V0i[cJ~\). ~ 2No I

-1,

(19.27)

\

the -1) is in error (Pc,J,0 = 0). it follows that

(i-l)to of

I

II

,,-_

<

,!_,

4c (/) Q

r,u

I \.

,!

Cu=dt-1

For the 3-leve! 8-PSK BCM code union bound on from

,,[cJ1o,1'/_,) + Pw,1-2. 2No

1

(19.28)

,~,,,-,., . ,~,- 19.4, Figure 19.12 shows the dominates the overall

1081:J

Chapter 19

Block Coded Modulation

QPSI( Upper bound

~.·=~i

~~

Bl~k-orrmrnte - ~ 10-1

£ ii "'

.D 0 0..

10-2

10- 3

1

C

'"' >-< 0

10-4

'"

!'"'

ci5

10-5

10-()

10- 7

C

-

10- 8 1

3

2

4

5

6

7

8

9

10

E 1,/N11 (dB)

FIGURE 19.12: The union bound of the 3-level 8-PSK BCM code of Example 19.4.

union bound for multistage decoding of BCM codes. In general, when a powerful code Co is used al level 1, the union bound on Pw,1 tends to become quite loose at low-to-medium SNR values. This analytical approach for zi-PSK BCM can be generalized for evaluating an upper bound for each stage of multistage decoding of multilevel codes in general. We define N;jj as the average number of nearest neighbors at SE distance ~ 7 corresponding to stage-} of the partitioning. Then, it follows that for O < i < l, the word-error probability at stage-i of the decoding is upper bounded by

Pw,; :S

Ul L (Neff) 11

w=d;

w

Cil

A,v Q

(

2) +

17 [ C ] Eb~;

2

No

Pw,i-1·

(19.29)

For the 4-level 16-QAM BCM code of Example 19.5, we find that

N;~j = (4 · 4 + 3, 8 + 2, 4)/16, = 3, N;~j = (1 · 2 + 2 · 4 + 4 · 2)/8. = 2.25.

(19.30) (19.31)

For this 16-QAM BCM code, Figure 19.13 depicts the union bound on Pw.1 computed from (19.29).

Section 1

Concatenated Coded Modulation

WIH

101

Upper bound Block-error rate~.:=

10"

10-1

~ JScJ

.D 0

10-·

5..

'"0 '""

~

10-,

iii 10--1

10-s

10-(; 0

2

4

6

8

JO

12

E1,iN11 (clB)

FIGURE 19. B: The union bound of the

BCl\11[ code of Example 19.5.

Coded modulation can be used in ·with concatenated coding to achieve gain, high reliability, and with reduced decoding This combination of two is called concatenated coded modulotion (CCM). CCM either with one level or with multilevels, can be constructed in the same manner as concatenated coding systems presented m that BCM codes are used as the inner codes.

In a single-level CCM the outer denoted by B, is an (N, K) linear block say an RS with from GF(2/J). The inner code, denoted by A, is a multilevel BCM code of n and dimension k = Ab over a certain signal space S. The encoding consists of two At the first stage of encoding, A outer-code codewords are formed and stored in a buffer as a Ab x N array, as shown in Figure 19.14. Each column of the army is Ab bits long and consists of A code one from each outer--code codeword. At the second stage of encoding, each column of the array is encoded into a sequence of 11 signals in the signal space S based on the BCM inner code A. This sequence is then transmitted. The outer code is interleaved a of A. The decoding also consists of two stages, the inner and outer decodings. When a sequence of 11 signals is it is decoded into Ah bits based on the BCM inner code A using a soft-decision say, multistage decoding presented in the previous section. These Ah bits are then stored as a column of a Ab x N array in a buffer for the second-stage decoding. After N inner code decodings, the received

1082

Chapter 19

TT b

Block Coded Modulation

o

+f--t---+-~~~~~~~~~----+-+-+-I

code codeword

4--

b ~e-;-+---~~~~~~--~e--+-~

T b

FIGURE 19.14: An array of>,. outer code codewords.

buffer contains a >,.b x N decoded array. Each column of this array consists of >,. estimated symbols, one for each transmitted outer-code codeword. Therefore, the array contains >,. received words for the outer code B, and they are decoded based on the outer code. To maintain low decoding complexity, the outer code is decoded with an algebraic decoding algorithm ( or a reliability-based decoding algorithm). EXAMPLE 19.6

For this example, the outer code B is the NASA standard (255, 223) RS code over GF(28 ) with minimum Hamming distance 33 [13]. The inner code A is the 3-level 8-PSK BCM code of length n = 8 and dimension k = 16 constructed in Example 19.1. Because k = 16 and b = 8, >,. = 2. Therefore, the outer code is interleaved by a depth of >,. = 2. The spectral efficiency of the overall CCM system is (223/255) · 17[A] = (223/255) · 2 = 1.749 bits/symbol. Figure 19.15 depicts the system. Because the inner code has a very simple 4-state trellis (see Figure 19.6), it is decoded with a Viterbi algorithm based on the full-code trellis. The outer RS code is decoded with the Euclidean decoding algorithm presented in Section 7.5. The error performance of this CCM system is shown in Figure 19.16. The system achieves a 5-dB coding gain over the uncoded QPSK at a BER of 10- 6 with 14.3% bandwidth expansion. For SNR Eb/ No = 6.2 dB, the system practically provides error-free data transmission. 19.4.2

Muitiievei Concatenated (oded Mo«foiaition Systems

In a q-level concatenated coded modulation system, q pairs of outer and inner codes areused.Forl::::: i::::: q,letB; bean(N. K;)linearblockcodeoverGF(2111 ;)withminimum Hamming distance D;. These codes are used as the outer codes. The inner codes are constructed from a multilevel BCM code Ao and a sequence of q subspaces of Ao.

Section 19A

Concatenated Coded Modulation

Interleave;depth: A= 2

'Wi33

- - - ~->~;~~;:;~c::~ L ~I encoder 11

I

,I,,'I

l

i----'1 Channel

_j

'I

Ii

i!

,1:,

y

(255,223) RS Outer-code decoder

standard inner code.

Soft-decision inner-code decoder L _ __

concatenated coded modulation RS code as the outer code and zm (8. 16,

I

----1--1---, ~~-QPSK Simulation_ ~~-,,~-- Bound

-"CF''

12

FIGURE 19.16: Bit-error performance of the CCM standard (255, 223) RS code as the outer code and an (8, 16, code as the inner code.

u

14

with the NASA

HlS4

Chapter 19

Block Coded Modulation

Let Ao= f[Co * C1 * · · · * C1-1] be an !-level BCM code over signal space S of length n, rlim~nsinn and MSE di~!a!'.ce d5 ~ dI[Ac]. \Me requfre that (19.32)

c;

For O:::: i < l, let be a linear subcode of the ith component code C; of Ao. Then, the interleaved code Cb* Ci*···* Cf_ 1 is a linear subcode of Co* C1 * · · · * C1_ 1 . After bits-to-signal mapping based on f (-), AS = f [Cb* Ci*···* Cf _1 ] is a subspace of Ao. AS is said to be a linear subcode of Ao. We can partition Ao based on AS by first partitioning Co* C1 * · · · * C1_ 1 based on the subcode Cb* C~ * · · · * 1 and then performing bits-to-signal mapping based on f (,). Let kb be the dimension of AS. Then, the partition Ao/ AS consists of 2k-kf1 cosets ( or cospaces) of AS. Now, we are ready to construct q inner codes for a q-level CCM system. First, we form q linear subcodes of Ao, denoted by A1, A2, · · · , Aq, such that the following conditions are met:

c;_

lo Ao :J A 1 :J · · · :J Aq. Aq = {f(IO)}, where IO is the all-zero codeword in Co* C1 * · · · * Ct-1· 3o For O < i :::: q, let k; be the dimension of A;. Then,

::t

(19.33) It follows from (19.32) and (19.33) that k1=m2+m3+···+mq, k2 = m3

+ 1114 + · · · + mq, (19.34)

kq-1 = 111q, kq = 0.

For O :::: i :::: q, let df ~

4 [A;] be the MSE distance of A;. Then,

We partition Ao into 2m I cosets based on A1 (or modulo-A 1 ). Let Ao/ A1 denote this partition. The MSE distance of each coset in Ai/ Ao is cl[- Ao/ A1 is called the coset code of Ao modulo-Al· Next, we partition each cos et in Ao/ A 1 into 21112 cosets based on A2. Let Ao/ A ii A2 denote this second-level partition of Ao. Then, Ao/ Ai/ A2 consists of 21111 +m 2 cosets of A2 in Ao. The MSE distance of each coset in Ao/ Ai/ A2 is elf Ao/ Ai/ A2 is called the coset code of Ao/ A1 moduloA2. We continue this partition process to form coset codes. For 1 :::: i :::: q, let Ao/ Ai/···/ A;-1 be the coset code of Ao/Ai/···/ A;-2 modulo-A;-1. We partition each coset in Ao/Ai/··· /A;-1 into 2"'; cosets based on A;. Then, Ao/Ail··· /A; is the coset code of Ao/Ai/··· /A;-1 modulo-A;. The MSE distance of a coset in Ao/ A 1 / · · · / A; is c!;2. Each cos et in Ao/ A 1 / · · · / Aq consists of only one codeword in

Section '19.4

Conc;:itenffted Coded I\Jlodula'cion

1(0)35

Ar/Ar Coset-code encoder

encoder

--------~

L __> l(~ 111~-bit

bytes

,outer-code

encoder

~[Ar/Ai/A,

/ m,-bit byte

/

Coset-code encoder

-~J~ ---J

Ar/A 1/-.-.

Coset-code encoder __ _

L__

HGUR:: i 9. 7: An overall multilevel concatenation encoder.

/10, and

00.

The

process results in a sequence of q coset

Ao/A.1. Ao/At/A2.

These q coc;et codes cire used as inner codes in a of the overaH encoder for a OCM inner encoder, for the first one from the the outer-code "'ncode,·, one from the of the inner-code encoder of the level. Fm ::= i ::= q. tbe of ith inner encoder from the of the (i ~ l)th irme:;· encoder is a seqvence of cosets from the coset code Ao/ A1 / ... / A;-1. The ith-level is encoded A 1nessage of m; -bit is encoded into a codeword in the ith-level outer code B;. 1~. The input coset from the of the inner-code level is into 2111 ; cosets . Each 111; the ith-level outer coc1e encoder is encoded into a cosetinthe /10/A.1/ ... /A;. the ith-level inner-code encoder is a sequence of cosets fron1 Ao/ Ai/ ... JL

1086

Chapter 19

Block Coded Modulation

The output of the qth inner-code encoder is a sequence of codewords from the base code Ao.His a sequence of Nn signals from the signal space S. The overall ct-level CCM system generates a concatenated modulation code C of length Nn, dimension

K = m1K1 + m2K2 + · · · + mqKq, and MSE distance (19.35) (see Problem 19.3). The spectral efficiency of C is

-

17[ C]

=

K

~

Nn

bits/symbol.

Multistage decoding of a multilevel concatenated modulation code is similar to the multistage decoding of a multilevel BCM code presented in the previous section. Let V = (vo, v1, ... , "'N-1) be a codeword in C, where for O :::: j < N, "'J is a codeword in the base inner code Ao. Each "'J must be in one of the cosets of the coset code Ao/A1. Let R = (ro,r1, ... ,rN-1) be the received sequence. R is decoded in q steps, from the first level to the qth level, as shown in Figure 19.18. At the first level of decoding. 'ifJ is decoded into one of the cosets in Ao/ A1. Based --

First-level inner-code decoder

JR

First-level outer-code decoder

,,,._

t i--

Second-level inner-code decoder

Second-level outer-code decoder

i

L FIGURE 19.18:

qth level inner-code decoder

qth level outer-code decoder

A decoder for a q-level concatenated modulation code.

Section ·i

Concatenated Coded Modulation

W31

on the decoded coset, the

is identified that is the estimate of the encoder at . Then, the sequence Cll (lJ fl(ll ) . l d' 1 b ' · '·I d T t b. Cll bClJ ) .~e (bCll ( @0 .m 1 •...• wN_ 1 rnoeco.1;a.aseao11t1,,,oute1co.e O • 1 , .... N-l be the decoded codeword. estimated message sequence is retrieved from this decoded codeword. from this decoded a coset sequence 0

O

'·

-

,s

at the where n
J

decoded coset, the is identified that is the estimate of the byte of the second-level outer-code encoder at time-j. The 2l ) 1·n 1"'( 2 ) "( 2 ) -paue11ce ~< 2 l \ ic_u ct·e-~-1-,..1 ;.1+0 a'- cod'eV11o~c1 ('Do<2 ) ' b(12 ) ' ... ' bCN-1 ;j,_, ~ ., \!Plo , '°"l ' .... ilN-1) t,UUC U H l Ji. 2 2 2) ' 1 Ieve l olller " cone 1 n th.e seconc-l.)asea' '.:Jn"(b( ) • l7 < ) , ...• b(N-l), a cose t sequence 1

I

0

-

1

where

~i21

E Ao/ A1/ A2. This coset sequence is then decoder. Other levels of are carried out in the same manner. To low while maintain good error performance, the inner codes are decoded vvith a soft-decision algorithm, and the outer codes are decoded lS

This with m 1 = m2 = 8. The first-level outer code B1 is the J\JASA standard RS code over GF(2 8 ). and the second-level outer code is the (255, 239) RS code over The base modulation inner code Ao is the 3-level 8-PSK BCM code f[Co * C1 * C2], where Co is the (8, 1) code, C1 is the (8. 7) SPC and is the (8. 8) universal code. The dimension of Ao is ko = 16, and (Ao) = 4. To construct a subspace A1 of Ao with dimension kt = ko -1111 = 8, we the three component codes: (1) C61l = (8. 0) is the code of length 8 of the all-zero sequence, (2) = (8. 1) is the code of 8, (3) = (8, 7) is the SPC 1 1 code of length 8. Let A1 = f[C6 l * c/l * J]. The dimension of A1 is /q = 8. 1 1 Because C611 c Co, Ci J c C1, Ci l c is a subspace of Ao. From (19.7) the MSE distance of A 1 is = Ai] = 8. The cos et code Ao/ A 1 consists of 28 cosets modulo-A 1 · Let = {f(lfD)}. the coset code Ao/ J\1/ A2 consists of2 16 cosets moduloA2. Each coset consists of one codevvord in Ao. The overall encoder for the 2-level CCM is shown in Figure 19.19. The 2-level concatenated modulation

cfl

C'i

dH

cf)

1088

Chapter 19 Input

"

I

/

Block Coded Modulation

(255,223) RS encoder

/

)

8-bit byte

A 0 /A 1

Coset-code encoder

\/

Input

"

I

/

FIGURE 19.19:

(255,239) RS encoder

/ 8-bit byte

" /

A 0 /AifA2 Coset-code encoder

Output

An encoder for a 2-level concatenated coded modulation system.

code C; has length 2040, dimension 3696, and MSE distance spectral efficiency is

4 [C]

= 132. The

3696 . 17[C] = = 1.818 bits/symbol. 2040 Ao has a 4-state 8-section trellis, as shown in Figure 19.6. A1 also has a 4-state 8-section trellis but without parallel branches between two adjacent states (see Problem 19.3). Soft-decision decodings of inner codes are relatively simple. The error performance of this 2-level CCM system is shown in Figure 19.20. It achieves a 5.3-dB real coding gain over uncoded QPSK at a BER of 10- 6 . This 2-level CCM system has higher spectral efficiency and better error performance than the single-level CCM system constructed in Example 19.6.

In general, a q-level CCM system can achieve higher spectral efficiency and better error performance than a single-level CCM system. 19.5

PRODUCT CODED MODULATION

Coded modulation also can be combined with the product coding technique to form product coded modulation [22] systems to achieve large coding gains with high spectral efficiencies. Let S be a signal space with i signal points. Each signal point s E S is labeled with a binary string of l bits, a
A(O),

A(l), ... , A(l-ll be l two-dimensional code arrays in the product codes

n,

a?/

Po, P1, ... , P1-1, respectively. For O:::: q < l, 0:::: i < and O:::: j < N, let be the code bit at the ith row and jth column of the qth code array A(q). We form the

Section 19.5 I

·m~S!

Product Coded M::idulation

-T

=QPSK ..,~ - Two-level

~

CCMsystem

10-"

~ j3

"'0

..0 h

CL

10-4

2

!

ic:O

10-j

10-h

~

10-7

s

4

3

6 E1,IN 11 (dB)

FIGURE 'l9.20:

Bit-enor

of the two-level CCM[

nP·rh,n-n

in Exam-

19.7. 11 X

array over S:

JV

~ f (A(O)

* A(l) *···*A(/-[))

= [l(c/Oia(ll ... aU-:-lJ)l ·

r.J

1.1

r.J

10:c:i
Each row of is a codev,10rd in the /-level BCMl code
= f[Co.2 *Cu* ...

* C1-12] over S. Therefore, the

collection of

arrays

* P1 * ... * P1-1] (f(A(O) * A

is the codes of

(19.37) E Pq

with O 2: q < l}

which are called the row and column component BCM code -with Nn and dimension >< k1,2 + .. · + k _u x /q_.1,2. The spectral efficiency of
and the MSE distance of P is 1-1 di, [ p] =

L cl; i=O

1 >< d; 2 X

i}.T,

1090

Chapter 19

Block Coded Modulation

where t:,.f is the intraset distance of the ith level of the binary partition of the signal space S. A signal array A, in ct:> P consists of I layers of binary code arrays. Each layer contributes a la be ling bit in the bits-to-signal mapping. Multistage decoding of

1 and row code
This example demonstrates the construction of a 3-level 8-PSK product code. The three binary row codes are Co.1 = (1023, 648), Cu = (1023, 893), and C2,1 = (1023, 1003) BCH codes with minimum designed distances 83, 27, and 5, respectively. The three binary column codes are as follows: (1) Co.2 is the (16, 5) RM code of minimum distance 8; (2) C1,2 is the (16, 15) even-parity-check code; and (3) C2.2 is the (16, 16) universal code. The resultant 3-level 8-PSK product code P has a spectral efficiency of 1.9968 bits/symbol. In multistage decoding, the column code


19.6

MULTILIEVEl CODED MODUlATmN IFOIR UNEQUAL !ERROR IP'IROTECTION

In certain communication systems an information sequence n1ay consist of several parts that have different degrees of significance and hence require different levels of protection against noise. Codes that are designed to provide different levels of data protection are known as unequal error protection (UEP) codes. UEP codes were first studied by Masnick and Wolf [24] and later by many others. Multilevel BCM codes with multistage decoding are quite suitable for unequal error protection. When such a code is used, different code levels provide different degrees of protection for different parts of a transmitted information sequence. Let m = (mo, m1 ..... m1_1) be a message of k bits, where for O ::::: i < I, m; is the ith part of the message and consists of k; information bits. Suppose mo is

Section

::rrm Frotection

Muldlevel Coded Modulation fm

·mg·J

n-1essage L®l~ arid rrn111_1 is the least

errors than the other be an (n. k;. d;) linear dimension k;, and rninimum distance d;, Considef the i-level BCM code *,,, * C1-1] over a space S, sequence F:11; is encoded into a codeword v;

or mm. mm1-1

in the !-level BC'I'/[ code for the message Lrmi, For we use f (mm) to ) be another message such that with j < i. H follows from the of

for O ~ i Theore1m

2: di

(f

f=

Iilf!l; ,

2

and

=

!Dlllj

for j < i}.

(19.38)

This says that if two messages are identical in the first i ~ 1 in the ith then codevvorcls are a Euclidean distaace of ctt least di that the ith of a rnessage is the SE distanced; for an /-level BCM code to".-'""""' condition on

(19.39)

nmst hoid. construction of TCM and BCNi codes is based on the conventional space S. either an MPSK or a QAM ,w, ~'"'"'- for intraset distance at each level is modulation codes for one-level error multilevel BCM codes constructed based on this conventional clo not well for with for small-to-medium increase in effective error coefficients in the first several of a codeword ciecoc1n1g, the error coefficient at the first stage do 2r1° · A (Ol becomes very large. For becomes • du ,~'"'"'""'"'V is mainly determined by the error increase in error coefficient the error performance the error The most significant The poor performance of the first mmo. of a of stages and results in poor overall

1092

Chapter 19

Block Coded Modulation

100 ,,-------,,-------,-----,------.----,------.---,,

~

=1== 1st level 2nd level 3rd level -QPSK

10-1

~

10-2

E

"'

.D 0

.... 10-1 ~ .... 2 ....

!

i:ci

10-s

10-s

10-6 ~ - ~ - - ~ - - - ~ - - - ~ - - - ~ - - - ~ ~ ~ 0 2 4 6 10 8 E1JN0 (dB)

FIGURE 19.21: Bit-error performances of various levels of a 3-level 8-PSK BCM code for unequal error protection with Ungerboeck's signal partition.

performance in terms of unequal error protection. Figure 19.21 shows the error performances at various levels of a 3-level 8-PSK BCM code with (64, 18, 22), (64, 57, 4), and (64, 63, 2) extended BCH codes as binary component codes. The MSE distances at three levels are do· .6.5 = 22 x 0.586 = 12.892, d1 · .6.f = 4 x 2 = 8, and d2 · .6.~ = 2 x 4 = 8, respectively. The spectral efficiency of the code is 17[ CJ = 2.156 bits/symbol. From Figure 19.21 we see that for large SNR this code does provide two levels of error protection; however, for small-to-medium SNRs, the increase in error coefficient by a factor of 222 at the first decoding stage totally destroys the unequal error protection capability, and all three levels perform poorly; in fact, the first level, with larger MSE distance, performs even worse than the other two levels. For BERs greater than 10- 5 , there is no unequal error protection. Several approaches have been proposed [25-28] for designing good multilevel BCM codes to provide distinct unequal error protection for various levels. These approaches use either nonconventional signal set partition or nonconventional signal constellations or both. In this section we present a nonconventional signal set partition of conventional signal constellations for designing multilevel BCM codes for unequal error protection. The approach is to reduce the error coefficient and to prevent or minimize error propagation from the first stage of decoding. Special examples with different signal spaces are used to explain the signal set partitioning, code construction, and multistage decoding. Consider the conventional 8-PSK signal space. We partition this signal space and label its signal points as shown in Figure 19.22. At the first leveL the signal set is partitioned into two subsets of equal size, denoted by Q(O) and Q(l); the four

Section '19.6

Multilevel Coded Modulation for

Ermr Protection

1093l

y 00

+ Q(OOO)

ol -~I Q(OOl)

f-lGURE 19.22:

+--J-

++ ++

Block

of the 8-PSK constellation.

Q(010)

Q(Oll)

Q(lOO)

Q(lOl)

Q(llO)

Q(111)

of Q(O) lie in the left-half bit oo = 0, and the four of Q(l) lie in the plane with first labeling bit ao = 1. At the second level of Q(O) is into two subsets, denoted Q(OO) and Q(Ol); the two in Q(OO) He in the upper of the left--half plane with second labeling bit 01 = 0, and the two in Q(Ol) lie in the lower of the left-half plane with second bit 01 = 1. Q(l) is also into two subsets of denoted by Q(lO) and Q(ll); the two of Q(lO) lie in the upper of the right-half plane with second = 0, and the two points of Q(ll) lie in the lower quadrant with second labeling bit a 1 = Finally, each subset at the into two each ·with one signal point with the third bit a2 = 0 or 1. Let (x, denote the coordinates of an 8-PSK point in the real . The has the following

lL x > 0 for all the with first bit oo = L and x < 0 for all the points with first labeling bit ao = 0. 2" y > 0 for all the signal points with the second labeling bit o 1 = 0, and y < 0 for all the signal with second labeling bit o 1 = 1. 31c The third labeling bit a2 specifies the point in the quadrant given by aoa1. Therefore, the first and second labeling bits of a point correspond to its x- and structure, the first and second stages of decoding can be carried out and in parallel, as shown in Figure 19.23. This removes the error propagation from the first of decoding to the second stage and also reduces decoding delay; the foregoing partition does not have the monotonically increasing intraset distance in the intraset distances at all three levels are equal: L'l.5 = L'l.f = = 0.586. For smallto-medium SNR this loss of intraset distances of the second and third level is more than compensated for the drastic reduction in error coefficients, as will be shown later.

1094

Chapter 19

Block Coded Modulation

r~I__

D_er__o_de_r_1_~1,----;p--

Vo

Decoder 2

Decoder 3

FIGURE 19.23:

Decoder for 3-level coded 8-PSK modulation.

Let I!' = ((rx,O, ry.o), Crx.l, ry,l), ... , Crx.n-1, ry,n-1)) be the two-dimensional received vector at the output of the matched filter, where (rxJ, rYi) denotes the coordinates of the jth received signal. Let

irx

~

(rx,O, rx,l, ... , rx,n-1)

and

~ (ry,O, ry.l, ... , ry,n-1) denote the projections of iron the x- and y-axes, respectively, called the in-phase and quadrature components of ir. Then, the decoders at the first and second stages simply operate on irx and iry, independently. Let vo = (vo.o, vo.1, ... , vo, 11 -1) and v1 = (v1,o, v1,1 .... , v1. 11 -1) be the decoded codewords at the first and second stages. They are then passed to the third decoding stage for decoding the third component code C2. The third stage of decoding is carried out as follows. First, we form a one-dimensional projection oh= (Irx, iry), R'y

ir'

=

(r

0, r{, ... , r;,),

where for for for for

VO,j VO,j VO.j vo.J

= Vl,j = 0, = 0 and VLj = 1, = Vl,j = 1, = 1 and VJ.J = 0.

(19.40)

Analysis shows that for i = 0 and 1, the bit-error probability of the ith stage of decoding is upper bounded as follows [27]: (19.41)

where 1. AZ) is the number of codewords in Ci with weight w,

2. (19.42)

Section 1

Multilevel

and

~l =

=

~2

l

6

Q(x) = --_-_

,,/r,;No

(19.4A)

~

01' the 3-level 8-PSK BCM

(17 is the The bit-error

the bit-error

17

p
<

~11 A,\?)2'" Q ,i , ~ 2) . , \ VWi:;1_ 1 .

(

\

(19.46)

w=do

The bound of ( shows that the set 19.22 results in an error coefficient of the conventional set results in an error coefficient of 2c1° A co (10) at the as shown in (19.46). Thus, there is a factor of 22c1° reduction in error coefficient. This reduction in error coefficient results in a for small-to-medium SNR that more than for the intraset distance loss. Consider the 3-level 8-PSK BCJ\!l[ code f[Co * * C2] in which the binary 18., (64, extended BCH codes are codes. This BCM code has 17[C] of almost 2 and SE 2 distances dofl,5 = d1Lii = = 1.172 at three levels. Bit-error uHun,v0 at three levels and their upper bounds of this code in Figure which shows that the code possesses three distinct levels of error ,-.r,--.u,roc, very low SNR. At BER = 10~ 5 , an 8.8-dB over the uncoded is attained at the first whereas the 10 log 00 (12.892/2) = 8.02 dB. is called block partitionto a QAM signal space and of the 16-QAM and 64-QAM are shown in y) be the coordinates of a signal point. Again, the signal labeling has the following with first bit ao = L and x < 0 for all the bit CIQ = 0. y > 0 for all the signal with second labeling bit cq = 0, and y < 0 for all the with second bit a 1 = 1.

li, x > 0 for all the

::t

0

If a 6-level 64-QAM BCM code is constructed based on the block partitioning and signal the first and second stages of decoding can be

1096

Chapter 19

Block Coded Modulation

I

\

UB

~ ~-'!

2::~~\ ::1 Level2- Level3 -

10-10 ~ - - - ~ - - - - ~ - - - ~ - - ~ - ~ - - - ~

-5

0

5

10

20

15

Et,!N0 (dB) FIG URE 19.24: Bit-error performances of various levels of an unequal error protection 3-level 8-PSK BCM code with block signal set partition.

+

--f: .

"

"

,

,

€>

y / ~~

+ + Y\: Y\: Y\: Y\: ++++++++: oj o/ o/ o/ o/ oj oj o/ \1

\1

\1

\1

\1

\1

\1

\1

~++++~+~+-++++++++ FIGURE 19.25:

Block partition of a 16-QAM.

Multilevel Coded Modulation for Unequal Error Protection

Section 19.6

Ul!ll1

0000 0000 0000 0000 0000 0000 00000000 000010000 0000 0000 0000 0000 0000 0000

0000 0000 0000 0000 0000 0000 0000 0000

0000 0000 0000 0000

~~~

~1

0000 0000

0000 0000 0000 0000

0000 0000 0000 0000 0000 0000 0000 0000

0000 0000 0000 0000

00 00 00 00

FIGURE 19.26:

second

Block partition of a 64-QAM.

For i = 0 and 1, the bit-error probabilities of the first and of decoding are the same and are upper bounded by

(19.47)

= Mi1s1 + i2s2 + i3s3 + (w = 3s1, s3 = 5s1, and s4 = 7s1-

where d~(il, i2, i3) s2

- i1 -- i2 - i3)s4]2, and

Sl = l/J42,

'Wi!:18

Chapter 19

Block Coded IVloduiation

If we assume correct decoding at the first and second decoding stages, the bit-error probabilities of the third and fourth decoding stages are upper bounded as follows: For i = 2 and 3,

(19.48) where d~(j) =

1

-Lib+ (w -

w

})1;2]2.

If we assume that all the previous decodings are correct, the bit-error probabilities of the fifth and sixth decoding stages are upper bounded by

for i = 4 and 5.

(19.49)

Figure 19.27 shows the bit-error performance of a 6-level 64-QAM BCM code whose binary component codes are (64, 24, 16), (64, 24, 16), (64, 45, 8),(64, 51, 6), (64, 57, 4) and (64, 57, 4) extended BCH codes. This 6-level 64-QAM code provides four levels of error protection. The first and second levels have the same degree of error protection. These two levels consist of 48 information bits. The fifth and sixth 10() .

UB 16-QAM ~ (64,24)-1 ~ (64,24)-2 ~ (64,45)-3 (64, 51)-4 - ~ (64,57)-5 ~ (64,57)-6 ~

""..,,...,~

....

10-1 Level 4.,.....

10-2

R

:.0

"'

..0 0

10-~ '

I-<

0.. I-<

0

I-<

I-<

/ Levels 5 and 6

10-4 //

Levels 1 and 2

.~ Q:l

16QAM

10-5 -

10-6 -

10-7 -5

0

5

10

15

20

25

E1,IN11 (dB)

FIGURE 19.27: Bit-error performance of various levels of a 6-level 64-QAM BCM code for unequal error protection with block signal set partition.

Section 19.6

Multilevel Coded IVlodulation for Unequal Error Protection

110l99

levels also have the same of error The of the code is 4.013125 At a BER of 10- 5 , the first two levels achieve a 12-dB over the uncoded 16-QAM. level is the same. levels may be poor. For shows that L•VVU"'~"''"' results in an error than that of the this can be used. Block first several say decodings and to reduce error effect the conventional levels to increase ~,~v~,.,u,"'"

block signai set as shown in 19.22. the same three v~,u,,~,,v,n codes are used to construct a 3-level 8-PSK BCM code based on the shown in Figure 19.28. The MSE distances at three code levels are then do~5 = 12"892, d1 = and d2L',,.~ = 4. The MSE distance of the third code level is increased from 1.172 to 4. The bit-error of this code at three levels are shown in 19.29. with Figure we see that the results in 4.4-olB coding gain at the third level at a BER of 10- 5 but a loss of 2.3-dB gain at the second level. This is a trade-off if the second of a message is not much more than the third part of the message.

~i

y

Q(OOO)

Q(OOl)

Q(OlO)

FIGURE 19.28:

Q(Oll)

Q(lOO)

Q(lOl)

Q(llO)

Q(lll)

Hybrid partition of the 8-PSK constellation.

1100

Chapter 19

Block Coded Modulation

100 ffi~ .. - - - - ~ - - - ~ ~ - - - ~ - - - - ~ 1 - - - - ~ ~ 10- 1

~

m:f

::=J

18) SI(64,45) SI(64,63) ~

10-111 ~ - - - - ~ - - - ~ - - - - ~ - - ~ - ~ - - - - ~ -5 15 20 0 5 10

FIGURE 19.29: Bit-error performance of various levels of a 3-level 8-PSK BCM code for unequal error protection with hybrid signal set partition.

PROBllEMS 19.1 Prove that the minimum squared Euclidean distance of the 3-level 8-PSK code given in Example 19.1 is equal to 4. 19.2 Construct a 3-level 8-PSK code with the following three binary component codes: (1) C1 is the (16, 1, 16) repetition code; (2) C2 is the (16, 11, 4) second-order RM code; and (3) C3 is the (16, 15, 2) single parity code. a. Determine the spectral efficiency of the code. Ito. Determine the minimum squared Euclidean, symbol, and product distances of the code. c. Analyze the trellis complexity of the code. 19.3 Decode the 3-level 8-PSK code constructed in Problem 19.2 with a single-stage Viterbi decoding, and compute its error performance for an A WGN channel. 19.4 Replace the first component code C 1 in Problem 19.2 with the first-order (16, 5, 8) RM code. Construct a new 3-level 8-PSK code. Determine its spectral efficiency, minimum squared Euclidean, symbol, and product distances. Analyze its trellis complexity. 19.5 Decode the code constructed in Problem 19.4 with a three-stage soft-decision decoding. Each component code is decoded with Viterbi decoding based on its trellis. Compute its error performance for an A WGN channel. 19.6 Design a single-level concatenated coded modulation system with the NASA standard (255, 223) RS code over GF(2 8 ) as the outer code and a 3-level 8-PSK code of length 16 as the inner code. The inner code is constructed using the follov;ing binary codes as the component codes: (1) C1 is the (16, 1, 16) repetition code; (2) C2 is the (16, 15, 2) single-parity-check code; and (3) C2 is the (16, 16, 1)

e>vv~o~rn.,',

Decode and the outer code with an of the for an A V\TGN

channel.

.!oo

'V. V.

"'l/I1JtidirnensionaI 1984.

S.t

,~ IEEE Trans.

B1osl( (Codes in 1043-4.S, r[Jctober 1986. ·· Abstmcts Octobei· 1986 .

IEEE Int.

. ''·Multilevel Channel Codes Based on 35 (1): 1989

" IEEE Trans.

"b!Iultilevel Codes and lV[arch 1989.

''On Linear Structure and Phase Modulation " IEEE 1991. 37 (4):

"On I,AuHilevel Block Modulation 1991.

JlilJl" G.D.

Tmns. ~11L T,

S.

Block MPSK Modulation Codes and Its Perforn1ance Inform. 1993. ]-2"

T. Woerz and J.

and " Proc. IEEE Global Telecommun. Conf. pp.

T. lation Scheme for Error 1990.

of Multilevel " IEEE Trans. for a MPSK San Diego.

"A Concatenated Coded Modu-· and S. "'IEEE Trans. Commun., 38 (6): 752-63, June

no2

Chapter 19

Block Coded Modulation

14!. D. J. Rhee and S. Lin, "Multilevel Concatenated Coded M-DPSK Modulation Schemes for the Shadowed Mobile Satellite Communication ChanneL" IEEE Trans. Vehicular Tech., 48 (5), 1634-49, September 1999. 15. G.D. Forney,Jr., "Approaching the Capacity of the A WGN Channel with Coset Codes and Multilevel Coset Codes," Proc. IEEE Int. Symp. Inform. Theory, p. 164, June 29-July 4, 1997. 16. U. Wachsmann, R. F. H. Fischer, and J.B. Huber, "Multilevel Codes: Theoretical Concepts and Practical Design Rules," IEEE Trans. Inform. Theory, 45 (5): 1361-91, July 1999. 17. S. Rajpal, D. J. Rhee, and S. Lin, "Low-Complexity and High-Performance Multilevel Coded Modulation for the A WGN and Rayleigh Fading Channels," Information Theory, edited by T. A. Gulliver, Springer-Verlag, New York, 1994.

11.8. S. Rajpal, D. J. Rhee, and S. Lin, "Coded MPSK Modulations for the A WGN and Rayleigh Fading Channels," Communications, Coding, and Cryptography, edited by R. Blahut, D. J. Costello, and S. Mauer, Kluwer Academic, Bosten, Mass., 1994. 19. D. J. Rhee, S. Rajpal, and S. Lin, "Some Block- and Trellis--Coded Modulations for the Rayleigh Fading Channel," IEEE Trans. Commun. 44 (1): 34-42, January 1996. 20. S. RajpaL D. J. Rhee, and S. Lin, "Multidimensional Trellis Coded Phase Modulation Using a Multilevel Concatenation Approach, Part I: Code Design," IEEE Trans. Commun., 45 (1): 64-72, January 1997. 21. S. Rajpal, D. J. Rhee, and S. Lin, "Multidimensional Trellis Coded Phase Modulation Using a Multilevel Concatenation Approach, Part U: Codes for the A WGN and Fading Channels," IEEE Trans. Commun., 45 (2): 177-86, March 1997. 22. S. Rajpal and S. Lin, "Product Coded Modulation," IEEE Trans. Commun., 45 (4): 389-92, April 1997. 23. S. Wainberg, "Error-Erasure Decoding of Product Codes," IEEE Trans. Inform. Theory, 8: 821-23, November 1972. 24. B. Masnick and J. K. Wolf. "On Linear Unequal Error Protection Codes," IEEE Trans. Inform. Theory.13 (4): 600-607, October 1967. 25. A. R. Calderbank and N. Seshadri, ''Multilevel Codes for Unequal Error Protection," IEEE Trans. Inform. Theory, 39 (4): 1234-48, July 1993. 26. L. F. Wei, "Coded Modulation with Unequal Error Protection," IEEE Trans. Commun., 41 (10): 1939-49, October 1993. 27. R. H. Morelos-Zaragoza, M. P. C. Fossorier, S. Lin, and H. Imai. "Multilevel Coded Modulation for Unequal Error Protection and Multistage Decoding- Part I: Symmetric Constellations," IEEE Trans. Commun .. 48 (2): 204- 11, February 2000.

Bibliography

n({)3

M. Isaka, }A. P. C. Fossorier, R. H. Morelos-Zaragoza, S. and H. ImaL "Multilevel Coded Modulation for Unequal Error Protection and Multistage Decoding-Part II: " IEEE Trans. Commun., 48 2000. '.);ilJ),

E. D. Divsalar, P. J. and M. K Introduction to Trellis-Coded Modulation with Applications, __ , ..... _,., New York, 1991. ~v~,rn,""

with Multilevel/Phase Signals," IEEE Trans. 1982.

"Bandwidth Efficient Channel: Code Construction and Performance " IEEE J. Select. Area Commun.. 7 1356-68. December 1989.

C H A P T rE

~

20

urst=Er So far we have been concerned primarily with coding techniques for channels on which transmission errors occur independently in digit positions (i.e., each transmitted digit is affected independently by noise); however. there are communication channels that are affected by disturbances that cause transmission errors to cluster into bursts. In general, codes for correcting random errors are not efficient for correcting burst errors, so it is desirable to design codes specifically for correcting burst errors, namely. burst-error-correcting codes. 20.1

INTRODUCTION

A burst of length l is defined as a vector whose nonzero components are confined to l consecutive digit positions, the first and last of which are nonzero. For example, the error vector e = (0 0 0 0 1 0 1 1 0 1 0 0 0 0 0) is a burst of length 6. A linear code that is capable of correcting all error bursts of length I or less but not all error bursts of length l + 1 is called an I-burst-error-correcting code, or the code is said to have burst-error-correcting capability I. It is clear that for given code length 11 and burst-error-correcting capability l, it is desirable to construct an (11. k) code with as small a redundancy n - k as possible. Next, we establish certain restrictions on n - k for given l. or restrictions on l for given 11 - k. THJEOJRJEM 20,]. A necessary condition for an (n, k) linear code to be able to correct all burst errors of length I or less is that no burst of length 2l or less can be a codeword.

Pmof. Suppose that there exists a burst w of length 21 or less as a codeword. This codeword v can be expressed as a vector sum of two bursts llR and w of length l or less (except the degenerate case, in which v is a burst of length 1). Then, llR and w must be in the same coset of a standard array for this code. If one of these two vectors is used as a coset leader (correctable error pattern), the other will be an uncorrectable error burst. As a result, this code will not be able to correct all error bursts of length l or less. Therefore, in order to correct all error bursts of length I or less. no burst of length 2l or less can be a codeword. QJE,ll)), 20,2 The number of parity-check digits of an (11, k) linear code that has no burst of length b or less as a codeword is at least b (i.e., n - k ~ b).

THJEOJRJEM

Proof. Consider the vectors whose nonzero components are confined to the first b digit positions. There are a total of l11 of them. No two such vectors can be in the same coset of a standard array for this code; otherwise, their vector sum, which is a burst of length b or less. would be a codeword. Therefore, these i' vectors must he in 7b rlistinct cosets. There are a total of 2n~k cosets for c:n (n, k) code. Thus, n - k must be at least equal to b (i.e., n - k ~ b).

of

Section 20.2

Codes

nOJSi

h follows from Theorems 20.1 and 20.2 that there must be a restriction on the number of code.

of an !-burst-error-correct-

The number of that code must be at least

'lriHilEOl!F?.lEMI

2@,3l

n-k:::::21. For a

of :m

(20.1)

and k, Theorem 20.3 that the k) code is at most l(n - k)/2J; that is,

11 (11,

!<

-

I

kj

i1 - .

2

(202)

This is an upper bound on the of an (n, k) code and is called the bound [5]. Codes that meet the Reiger bound are said to be The ratio 21 7=--

-

/1 -

k

(20.3)

is used as a measure of the burst-error-correcting efficiency of a code. An ~ , u u u , c u code has efficiency equal to 1. to show that if an (n, k) code is to correct all burst errors of l or less and to detect aH burst errors of length cl :::: l or the number of digits of the code must be at least l + cl (see Problem 20.1). IIJ~oJ0LOUH~

g(il:2

[D)[E((l)[)Jj[,llG ())IF S!uWGllE-!!'l!LJJ~§l-lE[/U'I.O~-((l)~,IR<.IE(Y!u\li\:il O((U( (0[D)[E§

code can most easily be decoded the errorin Section with a slight variation. Suppose that a codevvord v(X) from an /-burst-error-correcting (n, k) cyclic code is transmitted. Let n·(X) and be the received and error vectors, Let s(X)

=so+

be the If the errors in ie(X) are confined to the l check )(ll-k-l, ... ' xn-k- 2 , xn-k-l, then the l syndrome digits, Sn-k-1, · · · , sn-k-2, sn-k-1, match the errors of ie(X), and then - k - l loworder syndrome digits, so, s1, · · · , sn-k-1-1, are zeros. Suppose that the errors in are not confined to the positions xn-k-l, · · . , xn-k- 2 , xn-k-l of Il'(X) but are confined to l consecutive positions of Il'(X) (including the end-around case). Then, after a certain number of cyclic shifts of Il'(X), say i the errors will be shifted to the xn-k-l, · · · , xn-k- 2 , xn-k-! ofrrCi\X), the ith shift ofir(X). Let sU >(X) be the syndrome of •. ul ()(). Then, the first l high-order digits of §(il ()() match the errors at the positions xn-k-l, ... , xn-k- 2 . xn-k-l of Il'Cil(X), and the n - k - l low-order digits of §(il (X) are zeros. Using these facts, we may trap the errors in the syndrome register cyclic shifting rr(X). An error-trapping decoder for an !-burst-correcting code is shown in Figure 20.L where the received vector is shifted into the syndrome register from the left end. The decoding procedure is as follows:

1106

Chapter 20

r(X). Input

Burst-Error-Correcting Codes

~ '-...../

"

1

(n - k)-stage syndrOi:e ;egister

~~.~

~~~cted output

FIGURE 20.1:

An error-trapping decoder for burst-error-correcting codes.

Step Jl, The received vector Ir(X) is shifted into the syndrome and buffer

registers simultaneously. (U we do not want to decode the received parity-check digits, the buffer register needs only k stages.) As soon as ir(X) has been shifted into the syndrome register, the syndrome §(X) is formed. Stell} 2. The syndrome register starts to shift with gate 2 on. As soon as its n - k - l leftmost stages contain only zeros, its l rightmost stages contain the burst-error pattern. The error correction begins. There are three cases to be considered. Step 3. If the n - k - l leftmost stages of the syndrome register contain all zeros after the i th shift for O ::: i ::: n - k - l, the errors of the burst e(X) are confined to the parity-check positions of ir(X). In this event, the k received information digits in the buffer register are error-free. Gate 4 is then activated, and the k error-free information digits in the buffer are shifted out to the data sink. If then - k - / leftmost stages of the syndrome register never contain all zeros during the first n - k - l shifts of the syndrome register, the error burst is not confined to the n - k parity-check positions of Ir(X). Step 41, If the n - k - l leftmost stages of the syndrome register contain all zeros after the (n - k - l + i)th shift of the syndrome register for 1 ::: i :S l, the error burst is confined to positions xn-i, ... , X11 - 1 , XO, ... , x 1- i - l of r(X). (This is an end-around burst.) In this event, the/ - i digits contained in the l - i rightmost stages of the syndrome register match the errors at the parity-check positions, x 0 , X 1, ... , xt-i -l of ir(X), and the i digits contained in the next i stages of the syndrome register match the errors at the positions xn-i, · · · , x 11 - 2 , xn-l of ir(X). At this instant, a clock starts to count from (n -k-l +i + 1). The syndrome register is then shifted (in step with the clock) with gate 2 turned off. As soon as the clock has counted up to n - k, the i rightmost digits in the syndrome register match the errors at the positions xn-i, ... , x 11 - 2 • xn-l of r(X). Gates 3 and 4 are then activated. The

Codes

received information are read out of the corrected ihe error §ten] 5, H the n - k - ! leftmost time that the

'Vii]i7

and

erroneous out from the

2 disabled. If the n - k - ! leftmost never contain all zeros the time the k information have been read out of the buffet, an uncorrectable burst of errors hzi.s been detected. ~V\lith process takes 2n clock

of a small fraction of to the decoder in such a way that it cmrects ali the correctable burst errors of 11 - k or less. That besides all the bursts of I or the decoder ,c,iso corrects those bursts of I + 1 to n - k that are used as ccset leaders. This modified decoder as foHows. The entire .,·eceived vector is first shifted into the the erroi° feedback connections the length shifted 11 times b of the shortest burst that appears in the the syndrome register is recorded a counter. This burst is assumed to be the error burst added the channel. decoder begins as the , the

codes constructed Fire codes were the first class of burst errors. Let jp>(X) be an irreducible

~

'!08

Chapter 20

Burst-Error-Correcting Codes

p be the smallest integer such that )IJ!(X) divides XP + 1. The integer p is called the period of )Pl(X). Let l be a positive integer such that l :=:: m, and 21 - 1 is not divisible by p. An /-burst-error-correcting Fire code is generated by the following polynomial: g(X) == (X 21 - 1 + l)p(X). (20.4)

The length n of this code is the least common multiple (LCM) of 2! - 1 and the period p of p(X), that is, 11 = LCM(2l -1, p). (20.5) The number of parity-check digits of this code is m + 2l - l. Note that the two factors x 21 - 1 + 1 and p(X) of g(X) are relatively prime. IEXAMPllE 20.1

Consider the irreducible polynomial p(X) = 1 + X 2 + X 5 . Because )IJ!(X) is a primitive polynomial, its period is p = 25 - 1 = 31. Let I = 5. Clearly, 31 does not divide 21 - 1 = 9. The Fire code generated by g(X)

= =

+ 1)(1 + X 2 + X 5 ) 1 + x2 + xs + x9 + xn + x14 (X

9

has length n = LCM(9, 31) = 279. Therefore, it is a (279, 265) cyclic code that is capable of correcting any burst error of length 5 or less. To prove that the Fire code generated by the polynomial of (20.4) is capable of correcting any burst of length I or less, it is sufficient to show that all bursts of length I or less are in different cosets of the code. Thus, they can be used as coset leaders and form correctable error patterns. The proof is left as a problem (see Problem 20.3). Fire codes can be decoded with the error-trapping circuit shown in Figure 20.1. The error-trapping decoder for the (279, 265) Fire code considered in Example 20.1 is shown in Figure 20.2. In a data transmission ( or storage) system, if the receiver has some computation capability, a fast decoder for Fire codes may be implemented. Consider a Fire code with generator polynomial g(X) = (X 21 - 1 + l)p(X), where 2/ -1 and the period p of p(X) are relatively prime. Let r(X) be the received polynomial. Let §1 (X) and §2 (X) be the remainders resulting from dividing r(X) by x 21 - 1 + 1 and p(X), respectively. Then, we may take [§1(X),§2(X)] as a syndrome oh(X). We can readily see that s1 (X) = s2(X) = 0 if and only ih(X) is a code polynornial. If r(X) contains a nonzero error burst of length l or less, we must have s1 (X) i= 0 and §2(X) i= 0. If s1 (X) = 0 and §2(X) i= 0 [or §1 (X) i= 0 and §2(X) = O], then r(X) must contain a detectable but uncorrectable error burst of length greater than /. Now, consider an error-trapping decoder as shown in Figure 20.3. This decoder consists of two syndrome registers: the error-pattern register and the error-location register. The feedback connections of the error-pattern register are based on the

Section 20.3

Single-Burst-Error-Correcting Codes

1--->~·~ i ~

O

+

J-1

->-

~

o

~L,

FIGURE 20.2:

Error-trapping decoder for the (279. 265) Fire code.

nrn9J

n 10

Chapter 20

Burst-Error-Correcting Codes

_I

Buffer register

ii~~~~·

Error-pattern register

r(X)

l - 1 stages

l stages

Input

I

Comparator test for a match

'-.--------.---.~ 0

0

0

A

T

I ---1 I

L-+~ L~

:

Computation for 11 - q

I

L__ --i>~ounter 2 A2

~---~

FIGURE 20.3: A high-speed error-trapping decoder for Fire codes.

factor X21 - 1 + L and the feedback connections of the error-location register are based on the factor ]lll(X). The received polynomial r(X) is first read into the two syndrome registers and the buffer register. As soon as the entire 1r(X) has been shifted into the two syndrome registers, §1 (X) and §2 (X) are formed. The decoder tests §1 (X) and §2(X). Ih1 (X) = §2(X) = 0, the received polynomial ir(X) is assumed to be error-free and is then delivered to the user. If §1 (X) = 0 and §2 (X) f=- 0 [or §J (X) f=- 0 and §2 (X) = O], then r(X) contains a detectable but uncorrectable error burst and is therefore discarded. H §1 (X) f=- 0 and §2 (X) f=- 0, then 1r(X) is assumed to contain a correctable error burst and the decoder starts the following error-correction process:

JL Shift the error-pattern register and test for zeros at the l -1 high-order stages. Stop shifting as soon as the l - 1 high-order stages contain all zeros. The error burst is then trapped in the l low-order stages of the error-pattern register. Let )q be the number of shifts performed (in counter 1). Note that no more than 2/ - 2 shifts are needed to trap the error burst. Stiep 2o Shift the error-location register until the contents in its l low-order stages match the burst pattern in the I low-order stages of the errorpattern register. Let the number of shifts be Jc2 (in counter 2). In this step, no more than p - l shifts are required. §tie)lll 2lo Because 2/ - 1 and p are relatively prime, there exists a unique nonnegative integer q less than n ( code length) such that the remainders resulting from dividing q by 21 - 1 and pare Jc 1 and A2, respectively. Determine the integer q by computation. Then, the error burst begins

§tiel_JI

Section 20.3

Single-Burst-Error-Correcting Codes

nH

at xn-q and ends at xn-q+l-l of In the case that q = 0, the error burst begins at position x 0 and ends at x 1- 1 of §urerp 4\, Let in the register. Adel xn-q B(X) to rr(X) in the buffer register. This completes the error-correction process. If in

1 the l - 1 high-order stages of the nevei- contain all zeros the time the has been shifted 21 - 2 an uncorrectable error burst has been detected. In this event, the decoder stops the error-correction process. Because 2/ - 1 and The error-location number n - q can be ,o are there exist two A 1 and such that 0

A1(2l -1)

The q is

"''"""""'

+ A2,o =

1.

the remainder resulting from

by 11. Once A 1 and the numbers A 1 (21 - 1) and A2,o can be stored in the receiver for use in each decoding. n - q two multiplications, one one and one subtraction. We note that the decoder for Fire codes described here at most 21 + ,o - 3 shifts of the two registers and five arithmetic operations to carry out the error-correction process; however, the error-trapping decoder described in Section 20.2 takes n shifts to complete the error-correction process. Because 11 = LCM(2/ - 1, ,o ). it is much greater than 21 + ,o - 3. Therefore, is This improvement in decoding speed is only when the receiver has some capability, or available at the receiver. Furthermore. the fast error-trapping more logic.

Consider the (279, 265) Fire code considered in Example 20.1. This code is capable of correcting any error burst of length l = 5 or less. The fast error-trapping decoder for this code is shown in Figure 20.4. Suppose that the error burst rJc(X) =

x2 + x 3 + x4 + x5 + x 6

has occurred. It is a solid burst of length 5 starting at position 11 - q = 2. The syndromes tl 1 (X) and §2(X) are remainders resulting from dividing ie(X) by X 11 + 1 and l[ll(X) = 1 + X2 + X 5 , respectively:

"1 (X) = x2 + x3 + x4 + xs + x6, 52 ( X)

= 1 + X + X4.

n

12

Chapter 20

Burst-Error-Correcting Codes Error-pattern register

, I

I I

1

:

Counter 1 A1 Computation for n - q

I

I I 1

Counter 2 A2

Error-location register

r(X) Buffer register

Input

FIGURE 20.4: A high-speed error-trapping decoder for the (279, 265) Fire code.

Contents in the error-pattern register of the decoder shown in Figure 20.4 after each shift. TABLE 20.1:

§Ilnillflt

Cl[J)1I111!te1I11fa

0 1 2 3

001111100 000111110 000011111 100001111 110000111 111000011 111100001 111110000

4

5 6 7*

*At the seventh shift, all contents in the four high-order stages are all zeros.

As soon as the entire received polynomial ll'(X) has been shifted into the errorpattern and error-location registers, the contents in the two registers are s1 (X) and s2(X). Because all four high-order stages of the error-pattern register do not contain all zeros, the error burst is not trapped in the five low-order stages. The error-pattern register starts to shift. Table 20.1 shows the contents in the error-pattern register after each shift. We see that the error burst is trapped in the five low-order stages after )q = 7 shifts. Now, the error-location register begins to shift. Table 20.2 displays the contents in the error-location register after each shift. At the 29th shift, the contP,nts in thP P:n-or-lor~tirm rPgister rn~trh thP rnntPnk in thP fivP lnw-orrlPr

stages of the error-pattern register. Therefore, >-.2

= 29. Next, we need to compute

Codes

Section 20.3

m the errorof the decoder 20A after each shift

TM'lLE

1 l OO 1

0 1 2 3 4

11000 01100 00110 00001 10101 11110 011l 1 100 11 11101 11010 01101 100 1 0

s

6

7 9 10 Lt 12 13

01001 10000

14

11B

16 17 10

--'

01000 00100 00010

00001 10100

20 21

0 101 0

00101 1011 0 01011 100 0 1

22 23

24 25

11100 01110

26

0 0 11 l 10 1 1 1l 111

27

28 29*

*At the 29th shift, the contents match the burst pattern m the error-pattern register.

the error-location number

11 -

we find that

CJ.

7 >< 9 + = 7. and

-- -2) -.

>< 31

=1

we 7 >< 9 )( 29

+ (-2)

)( 31 )( 7

= 1393.

1393 n = we obtain q = 277. 11 - q = 2, which the error-location nmrtbeL Errm cmrection is achieved adding the error burst X2 + X 3 + + X. 6 to the received rt(X) in the buffer times. With register. The error-correction process takes at most 8 + 30 = 38 the decoder shown in the error-correction process takes 11 = 279 times.

The fast

decoder for Fire codes was first devised

Peterson

[21] and then refined by Chien [17]. The burst-error-correcting of a Fire code is z = 21 / (111 + 2/ - 1). If l is chosen to be tom. then z = 2111/(3111 - 1). For large m. z is 2/3. Fire codes are not very efficient with to the Reiger they can be A Fire code that is capable of any burst of length l or less and simultaneously detecting any burst of length d ::: l is by ~(X) =

+ 1)[Pl(X).

·Jn4

Chapter 20

Burst-Error-Correcting Codes

where c :=:: l + d - 1, and c is not divisible by the period p of p(X). The length of this code is the LCM of c and p. 20.3.2

Short IEffidernrl: IEh.«rrst-lErmr-(orrediD"lg Codes

Besides Fire codes, some very efficient cyclic codes and shortened cyclic codes for correcting short single bursts have been found either analytically or with the aid of a computer [7, 11, 12]. These codes with their generator polynomials are listed in Table 20.3. These codes and the codes derived from them by interleaving are the most efficient single-burst-error-correction codes known. TABLE 20.3: Some burst-error-correcting cyclic and shortened cyclic codes.

n-k-2l

0

1

Co«lle (n, k)

(7,3) (15, 9) (15, 7) (15, 5) (19, 11) (21, 9) (21, 7) (21, 5) (21, 3) (27, 17) (34,22) (38, 24) (50, 34) (56, 38) (59, 39) (15, 10) (21, 14) (21, 12) (21, 10) (23, 12) (27, 20) (31, 20) (38,29) (48,37) (63, 50) (63,48) (63, 46) (63, 44) (67, 54) (96, 79) (103,88)

BMll'§t-enoJr-\t'.Oll'l!'lediiR11.g \t'.lllllJ!llll!)mty 1

2 3 4 5 4 6 7 8 9 5 6 7 8 9 10 2 3 4 5 5 3 5 4 5 6 7 8 9 6 7 8

Gellllerntor polymmmfall g(X)*

35 171

721 2467 1151 14515 47343 214537 1647235 2671 15173 114361 224531 1505773 4003351 65 171 11663 7707

5343 311 4673 1151 4501 22377 105437 730535 2002353 36365 114361 501001

Single-Burst-Error-Correcting Codes

Section 20.3

TABLE 20.3: (continued)

n-k-2£

CCt1llirl!ie (n, 1k)

2

(51, 35) (55, 35) (57, 39) (63,55) (63,51)

3

4

(73, 63) (85, (85, 73) (105, 91) (131, 119) (169, 155) 42) (63, (85, 76) (89, 78) 82) (121, 112) (151, 136) (164, 153) (195,182) (217,202) (290,277) (43,29) (91, (93. 83) (117, 105) (133, 115) (255. 245) (255,243) (255. 241) (255,239)

JBsun1rni eirrmli' 0

0

(Ct1])Ir11'1l:'dnlili~

li:21JPmll:Dniln~y 1

3 2 2 4 5 5 9 4 7 9 8 3 4 5 6 4 4 5 6 5 6 3 2 3 4 4 3 6 4

5 6 5 5 4 3 4 7 3 4 5 6


471 123 161 3551 13627 13617 6647133 3501 304251 7164555 1341035 711 2263 16447 61303 2343 2651 10131 70521 15163 55725 1455 305 1501 4303 6137 1411 114371 6255 22475 120247 24711 52225 10571 2065 13413 1254355 3523 17667 76305 301565

( continued overleaf)

H'uS

1116

Chapter 20

Burst-Error-Correcting Codes TABLE 20.3: (continued)

Corrlle (ui, k)

n-k-21

R1U11rn1l:-e1rrn1r-,wnedlil!1lg ir!llp!lllbinnn11:y z

(273,261) (511, 499) (595,581) (465,454) (1023, 1010)

4

5

1Gel!1lernfo1r JP)({))Ilyl!1l({))mi\!!1Il g(X)*

4 4 5 3 4

10743 10451 64655 7275 22365

*Generator polynomials are given in octal representation. Each digit represents three binary digits according to the following code: 0 1

+------+ +------+

000 001

2 <-----+ 0 1 0 3 +------+ 0 1 1

4 5

+------+ +------+

100 101

6 7

+------+ +------+

110 111

The binary digits are then the coefficients of the polynomial, with the high-order coefficients at the left. For example, the binary representation of 171 is O O111 100 1, and the corresponding polynomial is g(X) = x6 + x 5+ x 4+ x 3+ 1.

20.3.3

18urst-1Ermr-Con-re«:ting (oidles (onstn..Ki:eidl lby interleaving

Code interleaving, presented in Section 4.8, is a powerful technique for constructing long powerful burst-error-correcting codes from short efficient burst-errorcorrecting codes. Suppose we interleave a burst-error-correcting (n, k) linear code C by a degree A. An (An, Ale) linear code CA results. A code array in C is shown in Figure 4.5. Obviously, a pattern of errors can be corrected for the whole array if and only if the pattern of errors in each row is a correctable pattern for C. No matter where it starts, a burst of length A will affect no more than one digit in each row. Thus, if C corrects single errors, the interleaved code CA will correct single bursts of length A or less. If C corrects any single burst of length l or less, the interleaved code CA will correct any single burst of length Al or less. If C has maximum possible burst-error-correcting capability (i.e., n - k - 21 = 0), the interleaved code CA also has maximum possible burst-error-correcting capability. By interleaving short codes with maximum possible burst-error-correcting capability, it is possible to construct codes of practically any length with maximum possible burst-error-correcting capability. Therefore, the interleaving technique reduces the problem of searching long efficient burst-error-correcting codes to searching good short codes. If the original code C is cyclic, the interleaved code CA is also cyclic. Let g(X) be the generator polynomial of C. Then, the generator polynomial of the interleaved code CA is g(XA) (see Problem 5.15 or Problem 19.7). Therefore, encoding and syndrome computation can be performed by shift registers. The decoder for the interleaved code can be derived from the decoder of the original code C simply by replacing each register stage of the original decoder by A stages without changing the other connections. This essentially allows the decoder circuitry to look at successive rows of the code array in successive decoder cycles. Therefore, if the decoder of the original code is simple, so is the decoder for the interleaved code.

Codes

Section

in Table 20.3. His a (7,

Consider the first code

+1)

= 1-1-

nu

code C generated by

+x+l)

-:-

+

any burst of - k) =:: 2 )( 2/L:· == 1. 'C,,anrsAC 10. The interlea11ed code c 10 is a (70, 30) is

,\ =

-+ of of this code is z.

-1any burst of

= 2 ><

= 1.

20 or less. The it is also optimal.

fa an interleaved code , the decoder first rearranges the received sequence into an array, then decodes each row of the anay based on C. If C is e1e,co1c11r,g can be used to decode each row, or a single errordecoder can be devised for i:he interleaved code by modifying the decoder for C.

l

for constructburst-error-correcting of codes C1 and C2, of the C1 >< C2 of C1 and that a code array as shown in Figure 4.3 is the received digits are back into an array row rovv. No naaHer where it starts, any existing error burst of 11 1 12 or less wiH affect no nwre than h + 1 consecutive rows; when the received back into an array, each column will at most affected a burst of /2. if the array is decoded on a column-by-column basis, the burst will be corrected. the capability of the that a code array is transmitted on a basis. By a similar argument, 11211 or less can be corrected. code is at least 112/1. Consequently, we may conclude that the capability l of the code is at least max {111 /2, 112! i}. Section 5.11). Suppose that the Consider a and burst-error-correcting t2 and burstcapability I of the to max (111 t2 + l 1, n2t1 + /2) [23]; that is, l ~

(20.6)

1118

Chapter 20

Burst-Error-Correcting Codes

This result can be shown as follows. Suppose that an error burst of length 112t1 + l2 or less occurred during the transmission of a code array (see Figure 5.23 for transmission of a code array). When the received digits are rearranged back into an array, all except l2 adjacent rows will contain t1 or fewer errors. Each of these l2 adjacent rows will contain t1 + 1 or fewer errors. If the rows are decoded first, these l2 adjacent rows may contain errors after the row decoding. Therefore, after row decoding, each column of the array contains an error burst of at most length l2. Because the column code C2 is capable of correcting any error burst of length l2 or less, all the remaining errors in the array will be corrected by column decoding. By a similar argument, any error burst of length 111t2 + /1 or less will be corrected if the column decoding is performed before the row decoding. Therefore, we obtain the result as stated by (20.6). 20.4

PHASIED-BURST-IERIROR-CORIRIECT!NG CODES

Consider an (n, k) code whose length n is a multiple of m, say 11 =um.The um digits of each codeword may be grouped into u subblocks; each subblock consists of m consecutive code digits. For example, let v

=

(vo, v1, v2, · · ·, Vu111-1)

be a codeword. Then, the ith subblock consists of the following consecutive code digits: Vim, V;m+l, · · · , V(i+l)m-1,

with O :::: i < u. A burst of length Am or less is called a phased burst if and only if it is confined to A consecutive subblocks, where A is a positive integer less than u. A linear code of length 11 = um that is capable of correcting all phased error bursts confined to A or fewer subblocks is called a Am-phased-burst-error-correcting code. Because a burst of length (A - l)m + 1, no matter where it starts, can affect at most A subblocks, it is clear that a Am-phased-burst-error-correcting code is capable of correcting any single burst of length (A - l)m + 1 or less. Thus, a Am-phased-burst-error-correcting code can be used as a [(A - l)m + 1]-single-bursterror-correcting code. 20.4.1

Burton Codes

Next, we present a class of phased-burst-error-correcting cyclic codes similar to the class of Fire codes and was discovered by Burton [18]. Let p(X) be an irreducible polynomial of degree m and period p. Let n be the LCM of ,n and p. Then, 11 = um. For any positive integer m there exists an m-phased burst-error-correcting Burton code of length 11 = um that is generated by g(X)

= (X + l)p(X). 111

(20.7)

The number of parity-check digits of this code is 2m. Thus, it is a (um, (u - 2)m) cyclic code. Each codeword consists of u subblocks. To show that the Burton code generated by g(X) = (X 111 + l)p(X) is capable of correcting all phased bursts confined to a single subblock of m digits, it is necessary and sufficient to prove that no two bursts are in the same coset of a standard array for the code. The proof is left as an exercise (see Problem 20.8).

Section 20.5

Burst-and-Random-Error-Correcting Codes

11 'u9l

decoder described in Section that the contents of the m leftmost of the are tested for zero at every mth shift. J[f m and the period p of [OJ(X) are and if the receiver has some power, Burton codes can be decoded with the fast described in Section 20.3. His such a way that the interleaved . Ak) code is capable of any burst that is confined to Aconsecutive subblocks. To do we arrange Acodewords code into A ro,Ns of a array as a subblock of each row as a single element the array consists of CJ and each column consists of A subblocks. The array is transmitted column column. one subblock at a time from each row. a code1Nord in the interleaved code consists of Au subblocks. No matter where it starts. any burst confined to X or fewer subblocks wiH affect no more than one subblock in each row. burst of Xm will be corrected if the array is decoded on basis. H the interleaved code is used as a [(X - l)m + its is

z=

2[(A - l)m 2)un

+ 1]

=

1

_

! (m -1). A

m

As the

code is to set up the code array and on rows in and the encoder of the interleaved code consists of the encoder of the original code and a buffer for the of the row vectors of the code array; the decoder consists of the decoder of code and a buffer for the storage of the received code array. Of course. the interleaved code can be decoded with the decoder of Figure 20.1, in which the contents of the Am leftmost are tested for zeros at every mth shift.

On many communication channels. errors occur neither at random nor in well-defined single bursts but in a mixed manner. Random-error-correcting codes or codes wiH be either inefficient or inadequate in combating these mixed errors. it is desirable to design codes that are of random errors and/or single or multiple error bursts. There are several methods of such codes. The most effective method is the technique. By interleaving a (n. k) code to degree A, we obtain a (An. Ak) code capable of correcting any combination oft bursts of length X or less. A code also can be used for simultaneous random-error correction and burst-error correction. Let c/1 and d2 be the minimum distances of codes C1 and respectively. Then. it is possible to show that the product code of C1 and C2 is of correcting any combination oft = L(d1d2 - 1) /2J or fewer random errors and simultaneously correcting any error burst of length l = max(111t2, 11211) or less, where t1 = l(c/1 -1)/2J. and t2 = l(d2 -1)/2J [23. 24]. To prove this assertion it is

1120

Chapter 20

Burst-Error-Correcting Codes

sufficient to show that an error burst of length l or less and a random-error pattern of t er fe";vel~ errcr0 canll.ct be b-1 the sa1ne coset of a standard arr2.y fer the p::.·aduct

code. Suppose that n1t2 ~ n2t1. Then, l = n1t2. Consider a burst of length n1t2 or less. When this vector is arranged as an array of 112 rows and 111 columns, each column contains at most t2 errors. Suppose that this burst and some random-error pattern oft or fewer errors are in the same coset of the product code. Then, the sum of these two error patterns (in array form) is a code array in the product code. As a result, each column of the sum array must either have no nonzero components or have at least d2 nonzero components. Each nonzero column of the sum array must be composed of at least d2 - t2 errors from the random-error pattern and at most t2 errors from the burst-error pattern. Because there are at most t random errors, these errors can be distributed among at most Lt /(d2 - t2)J columns. Thus, the sum array contains at most Lt /(d2 - t2)Jt2 + t nonzero components; however,

Hence, the sum array contains fewer than 2t < d1d2 nonzero components and cannot be a code array in the product code. This contradiction implies that a burst of length l = n 1t2 or less and a random-error pattern of t or fewer errors cannot be in the same coset of a standard array for the product code. Therefore, they can both be used as coset leaders and are correctable error patterns. If n2t1 > n 1 t2, then l = n2t1. The same argument can be applied to rows instead of columns of the sum array. 20.5.1

Codes Deirived from IRS (odes

In Chapter 2 it was pointed out that any element f3 in the Galois field GF(2m) can be expressed uniquely as a sum of 1, a, a 2 , ... , a 111 - 1 in the following form:

/3 =ao + a1a+a2a 2 + ···+a111 _1a m-1 , where a is a primitive element in GF(2m), and a 1 = 0 or 1. Thus, the correspondence between /3 and (ao, a1, · · · , am-1) is one-to-one. We shall call the m-tuple (ao, a1, · · · , a,,,_1) an m-bit byte representation of /3. Consider at-error-correcting RS code with code symbols from GF(2111 ). If each symbol is represented by its corresponding m-bit byte, we obtain a binary linear code with the following parameters: n = 111(2111 11 --

-

1),

k = 2mt.

This binary code is called a binary image of the RS code and is capable of correcting any error pattern that affects t or fewer m-bit bytes. H is immaterial whether a byte has one error, or all the m bits are in error; they are counted as one byte error, as follows. At the channel output the binary received vector is divided into 2"' - 1 bytes; each byte is transformed back into a symbol in GF(2117 ). Thus, if an error pattern affects t or fewer bytes, it affects t or fewer symbols in a RS code. Obviously, the error pattern can be corrected by the decoding methods described in Chapter 7. VV c shall call this binary code a t-bytc correcting code, but it is actually a multiple-phased-burst-error-correcting code.

Codes

1121

codes derived from RS codes are more effective clustered errors than randon1 e:-rcrs, sine<'; dlusternd errors involve several errors per and thus since a burst of length 3m + 1 code can correct any less, H can also correct any combination of because each such burst can affect no more than of four or fewer randmn RS code is any >c

= --------1 + L(l

+m

- 2)/mj

any burst of (t + 1 or corrects any combination of t or fevver random errors,

in Section 15,L are also effective a mixture of not correctable the inner code must for the outer code if the concatenated code is to con:ec1: Scai:tered random errors are corrected the inner code Bursts may affect rel3tive fe-w but so badly that the inner code cannot correct them. These few C3n then be corrected by the ouier code

Let ,8 be an element of order n in the Galois field H follows from Theorem 2.'5 that n is a factor of 2111 - 1. Let be the minimal of fJ. The is n. The of mo, is either tom or a factor of Iii. n has a proper factor b such that b-1-1 0

L,

Let 11 = n · b. Then, X"

+1=

(Xh

+ 1)(1 +

Because the order of fJ is n, and b < of 1 +xi,+ + ... + x
11.

-I-

x.2h -1- ... -1- x
+ 1 and must be a root divides 1 +xi,+ x. 2b + ... + x
fJ cannot be a root of

(20.8)

is a Fire code (b

of

any error burst of length

+ Let

of a cyclic code

of

n that is + 1.

t or fewer random errors. Clearly, ~2 (X) is a factor of X 11

be the least common ~(X)

of 1);1 (X) and

~2

= LCM (1,1;1(X).11;2(X)}.

(20.9)

1122

Chapter 20

Burst-Error-Correcting Codes

Clearly, g(X) divides X 11

+ 1 and can be expressed in the form (20.10)

g(X) = (Xb + l)go(X),

where g0(X) is a factor of 1 + Xb + X 2b + ... + x(X) and the t-error-correcting code C2 generated by g2(X). Because C is a subcode of the Fire code C1, C is capable of correcting any single error burst of length (b + 1)/2 or less, and since C is a subcode of the t-error-correcting code C2, it is capable of correcting any combination oft or fewer random errors. Because g(X) has (X + 1) as a factor, the minimum distance of C is even and is at least 2t + 2. It is possible to show that C is capable of correcting any single error burst of length (b + 1) /2 or less as well as any combination oft or fewer random errors. To show this, it is necessary and sufficient to prove that a burst of length (b + 1) /2 or less and an error pattern of weight t or less cannot be in the same coset of C unless they are identical (see Problem 20.15). IEKAMIPllE 20.4

Let a be a primitive element of the Galois field GF(2 6 ). The order of a is 26 -1 and the minimal polynomial of a is

= 63,

¢(X)=l+x+x6.

The integer 63 can be factored as follows: 63 = 7 . 9. Thus, we have x63 + 1 = (X9 + l)(l + x9 +xis+ x21 + x36 + x4s + xs4).

The code generated by the polynomial g1(X) = (X 9 + 1)(1 + X + X 6)

is a Fire code of length 63 that is capable of correcting any single error burst of length 5 or less. Let g2 (X) be the generator polynomial of the double-error-correcting BCH code of length 63. From Table 6.4 we find that

Note that both factors of g2(X) are factors of 1 + x9 + x 18 + x 27 + x36 + x 45 + xs4. The LCM of g1 (X) and g2(X) is g(X) = (X 9 + 1)(1 + X + X6)(1 + X +

x 2 + X 4 + X 6).

Hence, g(X) generates a (63, 42) cyclic code that is a subcode of both the Fire code generated by g1(X) = (X 9 + 1)(1 + X + X 6) and the double-error-correcting BCH code generated by g2(X) = (1 + X + X 6)(1 + X + x 2 + X 4 + X 6). Therefore, it is capable of correcting any single error burst of length 5 or less as well as any rnrnhln~tlon

of

h;\TA

nr fP,MPr r5inrlnm Prrnr~.

Section

r-----~'>

I_

Decoder I Burst trapping

1123

<"\

I

~~~~~~

II

r(X)

Codes

\

II

---~:<\

I

Inpu,

1-~-
I)ecode:t II P.sndon1-e.cror

correction

cor:rnction of burst and random errors.

in parallel. based on the Fire code based on the The received polynomial ll"(X) is shifted into both decoders to decode ll"(X). The decoder a decoded rnessage if the error pattern is either a burst of (b + 1)/2. or less or an undetectable burst. The random-error decoder a decoded message either contains t of fewer errors or is an undetectable error time when both decoders will decoded messages is is in a coset with a coset leader that is a burst of s (b + 1)/2 and of st. In this case the decoded messages from the two decoders are identicaL If both decoders fail to decode errors are detected_ P'-"H'VH<,~U

3)th-order

let us choose

=

(1 + X + X 6)(1 + X +

the LCM of 1t1

=

(X

9

= (X 9

+X 6 )(l+X+

+

+ 1)(1 + X +

+ 1)(1 + X + X6 )(1 + X +

+

+

is

+

(1 -1- ){ -1-

Hence, 11;(X) a code that is of correcting any single error burst of length S or less as vveH as any three or fewer random errors with majority-logic UvvVUH
1124

Chapter 20

Burst-Error-Correcting Codes

By combining Fire codes and BCH codes and with the aid of a computer, Hsu et al. have constructed several classes of shortened cyclic codes that are capable of correcting burst errors as well as random errors [26]. Other works on constructing burst-and-random error-correcting block codes can be found in [11, 19, and 26-28].

PROBLEMS 20.1 Show that if an (n, k) cyclic code is designed to correct all burst errors of length l or less and simultaneously to detect all burst errors of length d ::: l or less, the number of parity-check digits of the code must be at least l + d. 20.2 Devise an error-trapping decoder for an !-burst-error-correcting cyclic code. The received polynomial is shifted into the syndrome register from the right end. Describe the decoding operation of your decoder. 20.3 Prove that the Fire code generated by (20.4) is capable of correcting any error burst of length l or less. 20.4 The polynomial p(X) = 1 + X + X 4 is a primitive polynomial over GF(2). Find the generator polynomial of a Fire code that is capable of correcting any single error burst of length 4 or less. What is the length of this code? Devise a simple error-trapping decoder for this code. 20.5 Devise a high-speed error-trapping decoder for the Fire code constructed in Problem 20.4. Describe the decoding operation. 20.6 Use a code from Table 20.3 to derive a new code with burst-error-correcting capability l = 51, length n = 255, and burst-error-correcting efficiency z = l. Construct a decoder for this new code. 20.7 Let g(X) be the generator polynomial of an (n, k) cyclic code. Interleave this code to degree J... The resultant code is a (J..n, H) linear code. Show that this interleaved code is cyclic and its generator polynomial is g(X>.). 20.8 Show that the Burton code generated by g(X) = (X 111 + l)p(X), where l!}(X) is an irreducible polynomial of degree m, is capable of correcting all phased bursts confined to a single subblock of m digits. 20.9 Let m = 5. Construct a Burton code that is capable of correcting any phased burst confined to a single subblock of five digits. Suppose that this code is interleaved to degree J.. = 6. What are the length, the number of parity-check digits, and the burst-error-correcting capability of this interleaved code? 2(UO Interleave the (164, 153) code in Table 20.3 to degree J.. = 6. Compare this interleaved code with the interleaved Burton code of Problem 20.9. Which code is more efficient? 20.11 Interieave the (15, 7) BCH code to degree 7. Discuss the error-correcting capability of this interleaved code. Devise a decoder for this code and describe the decoding operation. 20ol2 Consider the (31, 15) RS code with symbols from GF(2 5 ). Convert this RS code to a binary code. Discuss the error-correcting capability of the binary RS code. 20.13 Suppose that the Fire code constructed in Problem 20.4 is shortened by deleting the 15 high-order message digits. Devise a decoder for the shortened code such that the 15 extra shifts of the syndrome register after the received vector has entered can be avoided. 20.14 Find a modified Fire code of length 63 that is capable of correcting any single burst of length 4 or less as well as any combination of two or fewer random errors. Determine its generator polynomial. 20.15 Consider the modified Fire code C generated by g(X) of (20.9). Show that a burst l\f 1Pngth (h + 1) /2 nr lP~~ -:::and Prrnr p-;::attPrn nf UJPlght f C\r les"7;: c~nnnt hP -in thP same coset.

JL N.

"IRE

Codes for IT-4(4): 150-57. December 1959.

Coders and Decoders for

:lo

P. Fire.. "A Class of Electronic Calif.~

Lt C. IVL Data

of Codes for Correction of "' IBM J. Res. Dev., 4: 58-64. 1960. "Codes for the Correction of 'Clustered' ff-6: 16-21.. March 1960.

Errors in " IRE Trans.

Adjacent-Error-Correcting Codes," IRE Trans. March 1960. Burst-Error-Correction 1962. fit J.E.

Res. Dev .. 4: E. AIEE. 80: 708-22.

l\l)o J,

·· Trans.

Bursts of

June 12 . 1961. 11JL J. J. December 1961.

Burst Error

Li_;

324-31.,

]20 T.

Kasami and S. Matoba. "Some Efficient Shortened Error ·· IEEE Trans.

Bo A.J. Odd

"Binary Codes Which Correct in Bursts of Three or Less for "IEEE Trans. IT-8: 356-59, October 1962.

,,u,m,,~,

Jlt!\o PL J. Gross. "A Note on Some Binary Group Codes V\/hich Correct Errors in Bursts of Four or Less." IRE Trans. Inform. IT-8: 384, October 1962. :IL!:lo A. J. Gross,

Bose-Chaudhuri Codes Which Correct Single Bursts of Errors." IEEE Trans. Inform. IT-9: 121. April 1963.

JLto E. Gorog, ''Some New Classes of Codes Used for Burst Error Correc., IBM J. Res. Dev .. 7: 102-11, 1963.

1126

Chapter 20

Burst-Error-Correcting Codes

17. R. T. Chien, "Burst-Correction Codes with High-Speed Decoding," IEEE Trans. inform. Theory, IT-1: 109-13, 1969. 18. H. 0. Burton, "A Ciass of Asymptotically Optimal Burst Correcting Block Codes," presented at the !CCC, Boulder, Colo., June 1969. 19. S. E. Tavares and S. G. S. Shiva, "Detecting and Correction Multiple Bursts for Binary Cyclic Codes," IEEE Trans., Inform. Theory, IT-16: 643-44, 1970. 20. R. G. Gallager, Information Theory and Reliable Communication, John Wiley, New York, 1968. 21. W.W. Peterson, Error-Correcting Codes, MIT Press, Cambridge, 1961. 22. P. Elias, "Error-Free Coding." IRE Trans. Inform. Theory, PGIT-4: 29-37, September 1954.

z::t

H. 0. Burton and E. J. Weldon, Jr., "Cyclic Product Codes," IEEE Trans. Inform. Theory, IT-11: 433-40, July 1964.

24t W. W. Peterson and E. J. Weldon, Jr., Error-Correcting Codes, 2nd ed., MIT Press, Cambridge, 1970. 25. S. M. Reddy and J. P. Robertson, "Random Error and Burst Correction by Iterated Codes,'' IEEE Trans. Inform. Theory, IT-18: 182-85, January 1972. 26. H. T. Hsu, T. Kasami, and R. T. Chien, "Error-Correction Codes for a Compound Channel," IEEE Trans. Inform. Theory, IT-14: 135-39, January 1968. 27. W. Posner, "Simultaneous Error-Correction and Burst-Error Detection Binary Linear Cyclic Codes," J. Soc. Ind. Appl. Math., 13: 1087-95, December 1965. 28. L. Bahl and R. T. Chien, "A Class of Multiple-Burst-Error-Correcting Codes," presented at the IEEE Int. Symp. Inform. Theory, Ellenville, N.Y., 1969.

channels were discussed in the co11vo!11tiona! codes have also ber~n r::o:rtJtr11-cted fo~~ -this purpose. }/lost of these codes are the feedback ,,;ere fixst constructed YNere later constructed codes

15 can also be used to convert codes. both brnrst and randorn errors with

[9] cw,,u,yu,"'H

the overaH code ren1ains convolutional because convolutiorwJ codes

Assume that e

=

the channel error sequence on a BSC.

2JLJ, A sequence of error bits C/+1 • e1+2· · · · . e1+b is called a burst b relative to a space g if

IO%:ci'I[],JTI1I'm01

1L e1+1 = e1+h = l; 2" the g bits 2t the b bits from e1+1

1128

Chapter 2·1

IEXAMPllE 21.1

Burst-Error-Correcting Convolutional Codes

Burst lengths aind Guarrd Spaces

Consider the error sequence e = (· · · 0 0 0 0 0 0 100 111100 0 0 11100 100 0 1 0 0 11 0 11 0 0 0 0 0 0 · · ·).This sequence contains a burst of length b = 28 relative to a guard space of length g = 6. Alternatively, it contains two bursts, one of length 7 and the other of length 17, relative to a guard space of length 4, or three bursts, of lengths 7, 6, and 8, relative to a guard space of length 3. This example illustrates that the length of a burst is always determined relative to some guard space and that the two cannot be specified independently. Gallager [3] has shown that for any convolutional code of rate R that corrects all bursts of length b or less relative to a guard space of length g, (21.1)

The bound of (21.1) is known as the bound on complete burst-error correction. 1 Massey [10] has also shown that if we allow a small fraction of the bursts of length b to be decoded incorrectly, the guard space requirements can be reduced significantly. In particular, for a convolutional code of rate R that corrects all but a fraction c of bursts of length b or less relative to a guard space of length g, g

->

R + [log 2 (1 - c)]/b

b-

1-R

R

~--

1-R

(21.2)

for small c. The bound of (21.2) is known as the bound on "almost all" burst-error correction. IEXAMPllE 21.2

Bounds oro Burrst-lErmr Correction

For R = 1/2, complete burst-error correction requires g /b ~ 3, whereas "almost all" burst-error correction only requires a g /b ratio of approximately 1, a difference of a factor of 3 in the necessary guard space.

21.2

BI\JIRSl-lEIRROIR-COIRRIECTING CONVOllllllONAl COIDIES

In this section we discuss two classes of convolutional codes for correcting burst errors, the Berlekamp-Preparata codes and the Iwadare-Massey codes.

21.2.1

18errlekamp-1Pirepairata Codes

Consider designing an (n, n - 1, m) systematic feedforward convolutional encoder to correct a phased burst error confined to a single block of n bits relative to a guard space of m error-free blocks; that is, a burst can affect at most one block within a span of (m + 1) blocks. The code produced by such an encoder would have a phased-burst-error correcting capability of one block relative to a guard space of m blocks. To design such a code, we must assure that each correctable error sequence 1

Wyner and Ash (4] earlier obtained a special case of this bound for phased-burst-error correction.

1 n!9l

Convolutional Codes

Section 21.2

= (so,s1, ·· .s111 ). This [e ]m = (reo. 01, · · · • ,1::111 ) results in a distinct implies that each error sequence with reo cf= (iJ) and e1 @. I = 1, 2. · · · , m, must must be distinct from a distinct and each of the~e ® and a block any error 0eg_uer:ce the the first erro,· block eo can be e1 I rID. l = 1. 2, · · · , m. Under these most one nonzero block can be decoded in the same way.

An (11. .,(n-ll(P) ll;J V'

n-1. m) lJ

rnatrix as

f

(21.3)

[_ where ("

II

IBlo

--

(11-l)

I

(21.4) (11-J)

(11-l)

,'.>n-1.0

gn-l.1

. -.,n-1.m .

1

0

0

'-

)(

(m

-

,·,[J l.m

I 0

is an n

(11-l)

(11-l)

gll

gl.O

0(11-l)

+ 1)

Im matrix in (ll.45) for shifting 1Bs 1_1 one column this operation can be

111atrix ro tJC1e For O < l :s 111, we obtain and deleting the 0 0

0-1

l O 0 l

0 (21.5)

0 0 where 1f is an (m the as

+ 1)

)( (m

+ 1)

0 0

0 0

uu.,~··'"'h

1 0_J matnx.

+ From For the

for an V•,SAHH,C,case with R f;

+ · · · + e,,,l8l111 + · · · + e111!Slolf111 . g/b = (1

=

(n -

'-b = 111 =

1)/n and g

1+ 1-

~-] 11

11-l n

we can write

this

=

= 211

11111

- L

+ R)/(1

(21.6)

- R).

= mb. this implies that (21.7)

that IB\o is an n x 211 matrix. Vl/e must now choose IBlo such that the conditions for burst-error correction cire scitisfied.

'lBO

Chapter2·1

Burst-Error-Correcting Convolutional Codes

If we choose the first n columns of Bo to be the skewed n x n identity matrix

pl

O1l

l

:

n

then (21.6) implies that each error sequence with ieo # 0 and ie1 = 0, l = 1, 2, · · · , m, will yield a distinct syndrome. In this case, we obtain the estimate of eo simply by reversing the first n bits in the 211-bit syndrome. In addition, for each eo # 0, the condition (21.8) must be satisfied for e1 # 0. This ensures that an error in some other block will not be confused for an error in block zero. We choose the last n columns of Ro to be the n x n matrix 0 0 0

A

B

0 0

C

0

0

0

0

D E F

0

where A, B, C, · · · must be chosen so that (21.8) is satisfied. First, note that for any e1 # 0 and I :::: n, the first n positions in the vector e1 Bo T1must be all O's, since T1shifts Bo such that Bo1'1 has all O's in its first I columns; however, for any ieo # 0, the vector ieolRo cannot have all O's in its first 11 positions. Hence, condition (21.8) is automatically satisfied for 11 ::: l ::: m = 2n - 1, and we can replace (21.8) with the condition that for each ieo -j. 0, (21.9) must be satisfied for all e1 # 8. Now, note the following:

1. The matrix Bo has rank n, since it contains a skewed 11 x n identity matrix in its first 11 columns. 2. For 1 ::: l ::: n - 1, the matrix BoT1 also has rank n, since it also contains a skewed n x n identity matrix in some n columns. 3. Condition (21.9) is equivalent to requiring that the row spaces of Bo and Bo T 1 have only the vector O in common. Hence, we can replace condition (21.9) with the condition that

....... IL rnnk

Bo

i Bi

Convolutional Codes

Section 212

condition (21.10) is

Because

to

that

'10_/e no-\N shovv that fL E. C, ·

for n

= 2, (2L11)

0 •

0 1

0 0

1 0

0

:I

0 0 0

0

1

0

1

0

1

0 0

0 0

0 0

l 0 0

0 0 0

0 0

0 0

l 0

B C

0

0

At 0

0

=L

det

two

1) 0 0

-------------

0 1 0

0

A = 1. For n = 3,

'Nhich is satisfied

0 0

is satisfied.

-- -------

de! [

0 0

be chosen so that

can

the conchtion

0 0 0

0 0 1

0 1 0

l 0 0

0 0 0

-------~----

0 0 0 1 1 0

1 0 0

0

1

B

0

0 0

C

0

0

l

=L

I

J

which are satisfied B = 1 and C = L H can be shown induction thai the elements of lRo can be chosen so that is satisfied Problem

For n

= '1, the 4

>< 8 matrix lRo is

~o

l

given by

~ ~

'01hich results in a rate R = 3/4

0 1 0 1 l 1 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0

0 0

ll

0

convolutional code with + D7, and 7 D + D that is of correcting phased bursts of 11 = 4 bits confined to a block relative to a guard space of m = 7 blocks (g = 11111 = 28 Because g/b = = 7, and (l + R)/(1- = (7 /4)(1/4) = 7, this code meets the for phased-burst-error correction, and is

~i3\D) =

+

+ D6 + D7, ~i3l

1 HZ

Chapter 21

Burst-Error-Correcting Convoiutionai Codes

The foregoing construction, discovered independently by Berlekamp [5] and Preparata f6], always results in a code that meets the bound of (21.1). Hence. the Berlekamp-Preparata codes are optimum for phased-burst-error correction. We can also extend this construction to generate optimum phased-burst-error correcting codes for k < n - 1 (see Problem 21.2). In addition, we can use interleaving (see Section 21.3) to convert any of these codes to phased-burst-error-correcting codes that are capable of correcting bursts confined to Jc blocks relative to a guard space of Jcm blocks, where Jc is the degree of interleaving. Because the ratio of guard space to burst length remains the same, these interleaved codes are still optimum. The Berlekamp-Preparata codes can be decoded using a general decoding technique for burst-error-correcting convolutional codes due to Massey [11 ]. We recall from (21.6) that the set of possible syndromes for a burst confined to block O is simply the row space of the n x 2n matrix lBo. Hence, if eo =f. 0 and e1 = 0, l = 1. 2, · · · , m. [§ ] 111 is a codeword in the (211, n) block code generated by Bo: however, if ieo = 0 and a single block e1 =f. 0 for some I. 1 ::: I ::: m, condition (21.8) ensures that [§] 111 is not a codeword in the block code generated by Bo. Therefore, eo contains a correctable error pattern if and only if [§] 111 is a codeword in the block code generated by Ro. This requires determining if [§] 111 = OJ, where IH!o is the n >< 211 block code parity-check matrix corresponding to Bo. Because

H5

Bo

=

0

001,0AB

D

0

010,00C

E

o

100'000

F

1

000'000

(21.15)

0

the corresponding block code parity-check matrix is given

r Ho=

0

'

0

r

E

Di

0

0

0

C

B,

0

1 0 0

0

0

0

A

0

0

0

0 ' 1

0 0 0

(21.16)

If [§] 111 lHI5 = OJ, the decoder must then find the correctable error pattern eo that produces the syndrome [§] 111 • Because in this case [§] 111 = eoBo, we obtain the estimate of eo simply by reversing the first 11 bits in [§ ]i,,. For a feedback decoder, the syndrome must then be modified to remove the effect of ieo. But for a correctable error pattern, [s] 111 = eolllio depends only on ieo, and hence when the effect of en is removed the syndrome will be reset to all zeros.

Section 21

ii.

)(

Corwolutional Codes

8 block code r· 0 0 0 I 0

I

1 B:3:

matrix lBio is 1 1 0 1 0 0 0 0

l l

1

0 0 0

0 0 1 C

1

0

L

l

1 1 0 0 0 0 0 _I 0

(21.17)

2U the observer canonical ) 11.5.- Because it . the received information sequence and thus it is units before error conection can use a controller canonical fmm realiwtion to "encode'' the received in the decode,- has as At time unit f = 7. if the [§h to a \i\/hen this activates the connections ,1 ,' " :,(2) - • ana e • me esnm::ues e,-(ll A·. e,(7) 0 0 0 is fed back to the .AND block.

note that unlimited error cann.ot occur in a decoder described. '\1\fhen the lS to O's. Hence. if the received sequence is error-free for must be cleared to aH zeros. Because :::;paces

ctT~

nee.:]ed to restore a decoder of this

eiror. codes of and also can of b relative to space of g, where not confined to an number of blocks. In general. though . the bou1J.d ~v~1iH in this case~ that the codes wHl not achieve the ratio of space to burst for a code interleaved to X. the shortest burst >c + 1 blocks contains (J.c -1)11 + 2 bits. the code will correct aH bursts of b = (J.c - 1)11 + 1 bits. the :opace to cover Jcm blocks is g = + -1 ThFo be JJserl band g

g

(:>cm+

-1

(Jc--l)n-1-1

,\m

+ n-1 i7

Jc_

11-l 11

when Jc is

~ l1Z

==

l+R 1+ R

(21.18)

codes are almost >c is large.

2 In this chapteL as in Sections 13.5-13.7. we use the labeling order of input and output sequences for rate R = (11 - 1) / 11 systematic feedforward encoder realizations given in Figure 11.5 for reasons of consistency with the published literature.

1134

Chapter 21

Burst-Error-Correcting Convolutional Codes

A

=, <~

,;::

+..+Cr; <~

':'L= +

=1 <~

>-<

Q)

"O

0

() Q)

Q

FIGURE 21.1: Complete system block diagram for the (4, 3, 7) Berlekamp-Preparata

code.

Convolutional Codes

iBS

burst errors of length b by Iwadare [2] codes. For

:n

==

)\ -\-

-

(21.19)

u/..., ·- -

g

= 11A . The

vvhere 'A is any (n-1) a(i)

1=

-

(n -

(m--\- ) -

1) 2 .....

-1-

<J]J

~ ('A+l)(n-i) where these codes can best be

- i)

Consider the rate R = this case m = 18. b = 9,

11 -

1.

+i -

11

(21.20)

3. The decoding of

= 3 and 'A = 3. In 2 l (D) = are

H~HH,.u"

~i

+ D and (D) = + . ThF: otseXYFor canonical form encoding circuit for this code is shown in P~ssume that a burst er:for of the first bit of block 0: 18

00 .. · ).

(21.21)

from

= ,;,JOJ (2) -eo

L (2) n _L -,·e1 ,c,-,

+ e~ll Dl7

(21.22a)

L

circuit for the

2, 18)

code with 'A

= 3.

1136

Chapter21

Burst-Error-Correcting Convolutional Codes

or (2l.22b) Examining this syndrome sequence, we note the following: 1. Each information error bit appears twice in §. 2. e;1l appears in§ before e;0), l = 0, 1, 2. 3. The number of positions between the two appearances of e;il is 10 + j, for j = 0, 1 and l = 0, 1, 2. The integer 10+ j is called the repeat distance of efil, and we see that the information error bits e;0l and e; 1) have distinct repeat distances of 10 and 11, respectively. This fact will be useful in decoding. If the burst starts with the second bit of block 0, then c1i <2> coi ) e = Co eo eo el el el e2 ez ez e3 000 ...

(21.23)

and (21.24) If the burst starts with the third bit of block 0, e - (00/2le(O)e(ll/2l/O)e(l)/2le(O)e(1)000·. ·) 0 1 1 1 2 2 2 3 3

(21.25)

and (21.26)

In each of these cases, the repeat distance for the information error bit e;il is still 10 + j. The decoding circuit for this code is shown in Figure 21.3. It consists of a systematic feedforward encoder (in controller canonical form) for syndrome calculation and a correction circuit. The two inputs of AND gate A1 are separated by 11 stages of the syndrome register (the repeat distance of e; 1l). Two of the inputs of AND gate Ao are separated by 10 stages of the syndrome register (the repeat 0 distance of e; l). The third input to Ao ensures that both AND gates cannot have a 1 output at the same time. A careful examination of the syndromes reveals that A 1 1 cannot be activated until error bit l appears at both its inputs, and then its output 1 will be the correct value of ). At the same time, the received information bit r6 1) is at the output of the fifteenth stage of the buffer register that stores the received information sequence rCll. Correction is then achieved by adding the output of A 1 to 1). The output of A1 is also fed back to reset the syndrome register. After one shift of the syndrome register, appears at both inputs of A1 . Hence, the output of A1 will be the correct value of eil), which is then used to correct the received information bit r;1 1. After the next shift of the syndrome register, the decoder estimates eil) and corrects rJ 1l in exactly the same way.

e6

e6

r6

e?)

Section 21 .2

Con vol utiona I Cocles

ll 131]'

Systematic feedforward encoder

2. 18)

FIGURE 21.3:

code with >..

= 3.

is inverted and any erroneous estimates while when e61 ) appears at both land ei1l appear as and if e61) = ej1J = 1. then the of Ao will be a 1, which would cause an erroneous correction at the of the buffer that stores the received information r/OJ. After eill is corrected, the register is shifted once, and at two inputs of Ao. The last of the register contains a 0, and hence A 1 is prevented from making any erroneous estimates, and I of Ao is a 1. the of P10 is the correct value of e6°J. At the same the received information bit is at the of the last stage of the buffer that stores the received information sequence r/Ol. Correction is then achieved the of Ao to rci°). The of Ao is also fed back to reset the after one the last stage of the register contains a 0, and the of Ao are 1. eiOl, and eiOl. of is the correct value of eiO), which is then used to correct the received information bi.t ri°). After the next shift of the register, the decoder estimates eiO) and corrects r~Ol in the same way. Because a correctable error burst of b = 9 mu-st be followed space of length g = 56, a careful examination of the error bursts and reveals that the next burst cannot

fed into Ao as (1) ( l) e0 ,e 1 ,and

,,,_,,,,,,,h

rtJ

1B8

Chapter 21

Burst-Error-Correcting Convoiutionai Codes

"'I

"' Q)

bl)

~

"'bl) Q)

-<

~

01

-<

I ~

n

;;-+ I

-

:::: I

~i.::-

+ Cl

I

"'c:-1 FIGURE 21.4: General decoding circuit for an Iwadare-Massey burst-errorcorrecting code.

S'c';ction 2'1 .3

afte, the decoder corrects

r1°>. at which time the

21A Note 2re fed into P~ND . · · · . An~3 estimates. H can be shown that unlimited error error is will be cleared and burst-error correction is g b

>

l+R 1-

1 "_1_ n-1 I 11 --1- II_:::__1

=2n-L

(2L27)

11

For the -2) -1

n(m+l)-1 b

nX

nX

(2L28) - 1) - 1 bits

and ;;ve see that these

vvith an

There is a second class of space than the class described ,.,.,,,v"-"H'" circuits Problems 2L5 and

The last to

encoders for The received bits "'"'-'m.~,_,, and sent to 'A decoders. A burst of length 'A on the channel vviH then look like errors to each of the decoders. Hence. if each decoder is •,u,.wun.~ 11A, then, -with (nA - l)A will be corrected. decoders. Hence, if each decoder of with interleaving, all sequences oft or fewer bursts of A or less relative to a space of at most will be corrected. 3 More a burst of length b'Jc on the channel will look like bursts of b' or less to each of the decoders. In this case :1The actual guard space requirements between groups of I or fewer bursts depends on how the bursts are distributed. In essence. there can be no more than t bursts of length !c or less in any A decoding lengths of received bits (see Problem 21.8).

1140

-

Chapter 21

Burst-Error-Correcting Convolutional Codes

1

2 -:,- Original encoder with k

r-

-3...,__

(n - l)(A - 1) n

A_ 1 ~1 ~ - -11- - ~

2 Original decoder with

0

k

Channel replaced

replaced

A-1 n

~-

by

by

--------/l>-11

-E}-

-E}- Astage delay unit FIGURE 21.5: An (n, k, m) convolutional coding system with interleaving degree A.

if each decoder is capable of correcting bursts of length b' relative to a guard space g', then the interleaved code will correct all bursts of length b' A or less relative to a guard space g' A. In practice it is not necessary to use A separate encoders and decoders but only to implement one encoder and one decoder in such a way that their operation is equivalent to A separate encoders and decoders. An (n, k, m) convolutional coding system with interleaving degree A is shown in Figure 21.5, where it is assumed that A - 1 is a multiple of n. The interleaver is placed between the encoder and the multiplexer and separates the n encoded bits in a block by A - 1 interleaved bits prior to transmission over the channel. In addition, the encoder is modified by replacing each delay unit with a string of A delay units. This makes the encoder equivalent to A separate encoders whose n-bit encoded blocks are formed in succession and ensures that there will be A - 1 interleaved bits between the last bit in one block and the first bit in the next block corresponding to the same encoder. Hence, the encoder of Figure 21.5 achieves an interleaving degree of A relative to the original convolutional code. The interleaver of Figure 21.5 requires (Jc-1)

(Jc-1)

n

n

- - + 2 - - + · · · + (n -

(Jc-1)

1)-n

(Jc-l)(n-1)

= ----

2

(21.29)

delay units. In addition, assuming k m-bit registers for encoding the received information sequences in controller canonical form and (n - k) m-bit syndrome registers, the decoder in Figure 21.5 requires a total of Jcnm delay units. Hence, the total memory required in the interleaved decoder is given by (A - l)(n -1)/2 + Jcnm delay units.

Seccion 21.3

Interleaved Convolutional Codes

1'u""·u

··>Channel->

(a)

(b)

FIGURE 21.6: An interleaved system for a

1. 1) convolutional code with A = 5.

Consider the (2. L 1) convolutional code with 1!;(1\D) = 1 + D. This code errors in a decoding of nA = n + 1) = 4. bits the circuit shown in when interleaved A, where A - 1 is a multiple of n = 2, this code will correct all bursts of length A or less with a space of 3A. Because 3A/A = 3, the ratio of to burst length meets the Gallager bound for rate R = 1/2 and this interleaved code is for complete burst-error correction! The total memory required in the interleaved decoder is (A - 1)(11 + Amn = - 1)/2, ·which 12 for A = 5. The complete interleaved convolutional is shown in Figure for A = 5. ~,vv~O,UUC;',

Consider the (4. 3. 7) Berlekamp-Preparata code of Example 21.3 and 21.1. which is capable of correcting phased bursts of one block relative to a space of m = 7 blocks. H this code is interleaved to degree A, it can correct bursts confined to A blocks relative to a guard space of 7A blocks. it can correct all

n42

Chapter 21

Burst-Error-Correcting Convolutional Codes

bursts of length b = 4(>c - 1) + 1 bits, relative to a guard space of g = 4(7>c + 1) - 1 bits, and for Jc = 5, g/b = M3/17 ~ 8.~, v1hi~h is about 20% above the Gallager bound of g/b = 7 for rate R = 3/4 codes.

Several convolutional coding techniques are available for correcting errors on channels that are subject to a combination of random and burst errors. As noted in the previous section, interleaving a code with random-error-correcting capability t to degree >c results in a code that can correct t or fewer bursts of length >c or less. This is called multiple-burst-error correction and requires only that there be no more than t bursts of length >c or less in any >c decoding lengths of received bits. Because some of the bursts may contain only scattered errors, this code in effect corrects a combination of burst and random errors. Codes also can be constructed to correct a specific combination of burst and random errors. The diffuse convolutional codes of Kohlenberg and Forney [7] and Massey [8] are an example of this type of construction. Adaptive decoding algorithms also can be employed to determine which type of error pattern has been received and then switch to the appropriate correction circuit. Gallager's [3] burstfinding codes and Tong's [9] burst-trapping codes make use of adaptive decoding algorithms. 21.4.1

Diffosie (l(l)des

Consider the (2, 1, m) systematic convolutional code with m = 3>c + 1 and 11;C 1l(D) = 1 + D'- + Dn + D 3'-+ 1 , where >c is any positive integer greater than 1. The syndrome sequence is given by §(D) = (<';CO) (D)g(l) (D) + (l';Cl\D), (21.30) and four orthogonal check-sums on e60) can be formed as follows:

so

s,.

(0)

= eo = e0(0) + e,_(0) (0)

=

eo

=

eo

(0)

(1)

+ eo (1) + e,_ J_ ,,(]) _L 1

~2A

I

,,(1)

t-3A

(1)

+ e3Hl · (21.31)

Hence, if there are two or fewer errors among the 11 error bits checked in (21.31), they can be corrected by a majority-logic decoder. Now, suppose thal a burst of length 2>c or less appears on the channel. H e6°> = 1, then the only other error bit in (21.31) that could have value 1 is 1l, since the other error bits in (21.31) are at least 2>c positions away from however, if = 0, a burst of length 2>c can affect at most two of the four check-sums. In either case, the estimate made by a majority-logic decoder will be correct. Hence, the

e6°l

e6°l;

e6°l

e6

Section 21.4

Convolutione:1 Codes

block

FIGURE 21.7:

3

rate R

2-error-

A-diffuse code. code corrects any tML = 2 or fewer r2mdom errors arnong the 11 error bits in (21.31) as well as bursts of b = 2)c or less with a space g = + 1) = 6?c + 2. With a feedback decoder, if all decisions have been correct. the same ,.,~vv·~"-""' of all information error bits. A 21.7. Note g 1

o

which is

6)c +2

== - - -

2)c 1/2 code

~3,

(21.32)

for a rate R to the Gallager bound for burst-errnr correction. The code of 21.7 is an diffuse convolutional code. A convolution.al cod<: is A-diffuse ar::_d tivfL --error if 2tML checksums can be found on each block-0 information error bit e& 1 such that for each i. 0 :s i :S k - 1:

eg

n

lo Error bits other than 1 from a burst of X or less that starts in block 1 0 and includes e& are checked no more than tML - 1 of the check-sums on ;t Error bits from a burst of nJI. or less that starts after the ith position in block O are checked no naore than fML of the check-smns on UcP,J;c.CJH
HnJ;"'-o,nHH

eg) eg)

Hence, a

decoder vvill

estimate each information error bit

eg) when there are tML or fewer random errors in the 2tML sums checking egi, or when the first of received bits contains a burst of length b

= n'A or less with a

space g

= nA

- iML· "With feedback decoding, the same

U44

Chapter 21

Burst-Error-Correcting Convoiutional Codes

error-correcting capability applies to all information error bits if all past decoding decisions have been correct. The error-propagation of diffuse codes are examined in Problem 21.13. For the special case in which fML = 1 and R = (n ~ 1)/n, the Iwadare-Massey codes discussed in Section 21.2 form a class of A-diffuse, single-error-correcting codes. For any fML, Ferguson [12] has constructed a class of A-diffuse codes with rate R = 1/2 and g/b ~ 4, and Tong [13] has constructed a similar class of asymptotically optimum rate R = 1/2, A-diffuse codes for which g/b -+ 3 as ."A. becomes large. Some of Tong's rate R = 1/2 diffuse codes are listed in Table 21.1. In addition, Tong [14] has constructed a class of A-diffuse, fML-error-correcting. self-orthogonal codes. These codes are easy to impiement, have limited error propagation, and their g/b ratio, although much larger than the Gallager bound, is optimum within the class of self-orthogonal diffuse codes. A list of Tong's selforthogonal diffuse codes with rates R = 1/2, 2/3, 3/4. and 4/5 is given in Table 21.2. Note that the g /b ratio for the diffuse codes in Table 21.1 is much less than for the TABLE 21.1: Rate R lML

m

A.*. mm

2 3

3A +3 3A + 12

2 5

4

3A+37

17

5

3A + 88

44

6

3A + 217 120

7

3A + 374 233

= 1/2, A-diffuse, IML -error-correcting orthogonalizable codes.

il1) {0.A,2A+3.3A+3} {0, LA+ 1. 2A + 7, 3A + 9, 3A + 12) (0. 2. 3, A + 3. 2A + 18, 2A+23,3A+27, 3A + 37} {O, 3, 4. 5, A+ 5, 2A + 40, 2Jc + 54, 3A + 60, 3A + 67, 3A + 88} (0, 2. 3, 7, 8, A+ 8, 2A + 88, 2A + 118, 2A + 138, 3A + 147, 3A + 157, 3A + 217} {O, 6, 7, 9, 10, 11, A+ 11, 2A + 141. 2A + 154, 2A + 245, 3A + 257, 31 + 296, 3A + 322, 3A + 374}

01r\l:l!nogorrn21Iln1m\l:fonu nnlles-r

(3A + 3, 2A + 1) (3A + 9, 2A + 3)(3A + 12. A+ 4) (2A + 23, A+ 8) (3A + 27. 2A + 12, 2A + 7) (3A+37.A+13) (5, 1)(2A + 54, A+ 19)(2A + 67, A+ 12)(3A + 60, 2A + 11, 2A + 25) (3A + 88, A + 33, A + 26) (8. 4, 5, 6)(2A + 118, A+ 38) (2A + 138, A + 58, A + 28) (3A + 147, 2A + 17, 2A + 67, 2A + 37)(3A + 157, A+ 18) (3A + 217, A + 68, A + 78) (10, 3, 1)(11, 8, 4)(2A + 154, A+ 24)(2A + 245, A+ 115, A+ 102)(3A + 257, 2A + 23, 2X + 127, >,. + 114) +296, A+ 50)(3A + 322, A+ 76, A+ 37)(3A + 374, A+ 89, A+ 63)

Adapted from [13]. *The minimum value of Jc for which these codes are Jc-diffuse and tM L -error-correcting. ·i- (x. y, ···)indicates that the sum sx + sv + · · · forms an orthogonal check-sum on e&O). Only those orthogonal equations that require a sum of syndrome bits are listed.

Section 21.4

Burst-and-Random-Error-Correcting Convolutional Codes

·u it!S,

fML-error-correcting self-orthogonal codes. m

fML

1 2 3 4 5

3Jc 4Jc + 1 5Jc +4 6Jc + 10 7A+19

1 2 4 8 13

{O.Jc.3Jc} {O, Jc, 3Jc. 4Jc + 1) (0. L Jc+ 3, 3Jc + 3, 4Jc + 3, 5Jc + 4) (0. 1. 3. Jc+ 7, 3Jc + 7, 4Jc + 7. 5Jc + 8. 6Jc + 10) (0. 1. 4. 6, Jc+ 12. 3Jc + 12. 4Jc + 12. 5Jc + 13, 6Jc + 15. 7."A. + 19}

(a) Rate R = 1/2 codes itiiE

8Jc +3 lOJc + 10

3

3 4

12A

+ 26

12

2

7

n

10, 2,1c. 6Jc + 1, + 21 {O, 2, 2Jc + 2, 6>, + 4. n + 4. 9Jc +7} {0. 2, 7. 2Jc + 7, 6Jc + 11, + 11, 10Jc + 20. 12Jc + 26J

(0. Jc. 4Jc. 8Jc + 3) (0, l, Jc+ 2, 4), + 4. 8Jc + 5. lOJc + 10) (0, 1. 4. Jc+ 7. 4Jc + 11, 8Jc + 12. 9Jc + H lU + 24)

n

(b) Rate R = 2/3 codes 1ML

2

3

12Jc + 5 15Jc + 12

6 9

(0. Jc, 6Jc. 9Jc + 2) (0, 2Jc. lH + 3. 12A + 5) (0, 1. 2Jc + 3, lH + 6, (0, 2. 4Jc + 3, 7A + 3, 12Jc + 7, 15Jc + 12) 8Jc + 5, 13Jc + 7)

(c) Rate R

2

16Jc

+8

7

{O, Jc, lOJc + 3, l2Jc + 4)

=

ro, 4Jc, n. 8Jc + 1J (0. 3, Jc, 6Jc. 9Jc + 2. 14Jc + 8)

3/4 codes

{O. 2Jc, 13Jc 14Jc + 7)

+ 5,

(0, 3Jc,

n, 16Jc + 8)

(0. 5Jc. 8Jc + 1. 9Jc + 2)

(d) Rate R = 4/5 codes Adapted from [14]. *The minimum value of Jc for which these codes are Jc-diffuse and fML-error-correcting.

,-,~."'~.,,.,,_, rate R = 1/2 self-orthogonal diffuse codes in Table 21.2; however, the self-orthogonal diffuse codes are easier to implement and less sensitive to error propagation.

Consider the (2, 1, 9) systematic code with r,0\D) triangle for this code is given by

= 1 + D 3 + D 7 + D 9 . The parity

1146

Chapter 21

Burst-Error-Correcting Convolutional Codes

1 0 1 0 0 1 0 0 1 0 0 0 0 1 0 0 1 1 0

---:>

---:>

---:>

~>

1 0 0 1 0

IT] 0 0 1

1 0 0

1 0

0 0

0 0

IT]

0

0

IT]

0

0

1

IT]

1

0 0

0

IT]

0

0

1 0

[D

and we see that the code is self-orthogonal with majority-logic error-correctingcapability fML = 2. The four syndrome bits orthogonal on e6°> are given by (0)

so= e0 (0) s3 = e0 (0) s7 = eo s9 = e60)

+ €3(0) +

(0)

e2

+ €4(0)

+ e6(0)

+ €7(1)

+ e9(0)

+ eo(1) + €3(1) + €7(1) + e9(1)

(21.33)

The error bits other than e6°l that belong to a burst of length b = 4 or less including e6°> can affect only syndrome bit so. In addition, error bits from a burst of length b = 4 or less that starts after e6°> can affect at most two of the syndrome bits orthogonal on e6°>. Hence, this is a A = 2 diffuse, fML = 2 error-correcting code that can correct any two or fewer random errors in a decoding length or any burst of length b = 4 or less with a guard space g = 18.

2VL2

18!..!lrs'i:-fkJ1dlnrnig Codes

Consider the (2, 1, L + M +5) systematic convolutional code with il 1l (D) = 1 + D 3 + D 4 + D 5 + vL+M+s. In general, a Gallager burst-finding code corrects "almost all" bursts of length b = 2(L - 5) or less with a guard space g = 2(L + M + 5) as well as tML or fewer random errors in a decoding length. In all cases tML < 'ML, and in this example tML = 1. Typically, L is on the order of hundreds of bits, whereas M is on the order of tens of bits. The encoding circuit for this code is shown in Figure 21.8. The first five delay units in the encoder along with their associated connections form a set of J = 4 orthogonal check-sums on each information error bit (see Example 13.11). By themselves, these orthogonal check-sums could be used to correct fML = 2 or fewer random errors in a decoding length. The key to the Gallager burst-finding code is that only patterns of tML or fewer random errors are corrected, where tML < fML, and the additional error-correcting capability of the orthogonal check-sums is used to detect bursts. The decoding circuit for this burst-finding code is shown in Figure 21.9. To understand the functioning of the decoder, assume that the /th block has just been received~ that a!! past decoding estimates have been correct, and that the decoder is in the "random mode" (or r-mode) rather than the "burst mode" (or b-mode ).

Section 21.4

circuit for the rate R = 1/2

Output = 0 when zero or one of the four inputs are J. Output = l when two. three, or four of the four inputs are ! . Mode selector

Notes: 1. Mode selector switches to b-rnode when 2 or 3 of the 4 check-sums have value 1. 2. Mode selector switches to r-rnocle when M consecutive r-rnocle estimates have value 0.

circuit for the rate R

=

In the r-mode, the input to the mode selector with taken 3.S the decoding estimate, whereas in the b-mode, the with b is taken as the decoding estimate. Because all estimates have been correct, the feedback of these estimates along the ""main feedback line'" removes the effect of e;~L-M-S from the syndrome, since at this time e;~L-M-S is the error bit that is fed to the from the last stage of the encoder replica. Hence, the modified bits entering the syndrome register are precisely those that would enter if the last M +L of the encoder replica and the associated connection were removed.

n4:S

Chapter 21

Burst-Error-Correcting Convolutional Codes

This modified code is decoded just as if it were a random-error-correcting code, except that the bits are by time units before Now, assume that there are r;l!.L = 1 or fewer errors in the 17£ = 11 error bits affecting the J = 4 orthogonal check-sums on e;~L-s· Then, if e;~L-S = 1, all four check-sums will equal 1, and if e;~L-S = 0, at most one of the check-sums will equal 1. Hence, the estimate [e;~L--s]r, which is the decision accepted by the mode selector in the r-mode, will be correct, and the decoder will stay in the r-mode and decode correctly as long as there is at most one error in a decoding length. Note also that there is an additional M time unit delay before the r-mode decision is actually accepted by the mode selector. Now, suppose that at some time two or three of the four check-sums have value 1. This situation will always occur when there are two or three errors in the 17£ = 11 error bits affecting the check-smns, and the r-mode estimate is incorrect. (Note that it is possible for two or three errors to cause zero, one, or all four of the check-sums to equal L but in this case the estimate will be correct, since two of the errors must have canceled their effect on the check-sums.) When this occurs, the mode selector changes to the b-mode, and [eflL-M-S]b is now chosen as the estimate. This causes the preceding M decisions of the r-mode decoder to be rejected, which ensures that when a burst occurs the r-mode decoder will have detected the burst before any of its estimates of error bits in the burst are accepted by the mode selector. If the bits in a burst have probability of being in error, then when the first bit in the burst reaches the end of the syndrome register, all 24 = 16 possible outcomes for the four check-sums are equally likely. Because

i

( 1) + ( ~ ) =

10 of these outcomes cause the mode selector to switch to the

b-mode, the probability that the burst is not detected in time for the mode selector to switch to the b-mode before accepting an r-mode estimate for any error bit in the burst is 6 Pr (undetected burst) """ ( ) M+l (21.34) 16 Clearly, this probability is quite small for M greater than about 10. The decoding estimate in the b-mode is given by ,(0) ] [ el-L-M-5 b

(0)

= s1 = el-L-M-5

(0)

(0)

(0)

(0)

(1)

+ el-5 + el-4 + el-3 + el + el

·

(21.35)

If e;~L-M-S is part of a burst, the other error bits in (21.35) must come from the guard space if the guard space has length g = 2(L + M + 5) or more. Hence, the

decoding estimate in the b-mode will be correct. Note that it is possible for a burst to cause a switch to the b-mode as soon as its first bit reaches the first input to a check-sum in the syndrome register, which would cause up to M + 5 guard-space error bits to be estimated in the b-mode. For these estimates to be correct, the burst length should not exceed L - 5 blocks, or b = 2(L - 5) bits, since otherwise, error bits from the burst would affect the decoding of these guard-space error bits. While the decoder is in the b-mode, the r-mode estimates continue to be mnni1nrPrl ::is ::in inrlir::itnr nf whPn tlw rlPrnrler shnnlrl ~m;trh bac1'

tn

the r-rnode.

When M consecutive r-mode estimates have value 0, this is taken as evidence that

Section 21.4

the most recent L

Convolutional Codes

+JV!+ 5received blocks have been in a

and the mode \

( 4 \

o)+\1J= 5

selector is returned to the r-m0de. of the r-mocle

ntl!9

to the r-111ode

C,~!C!lHICH.G

( 5 \.M ~11-1\

Pr

\ 16)

a value of O for the a burst is

.

than about 10. desc:ribed corrects '' almost aff' bursts of of

2( vvhich meets the lmver bound of rate R = 1/2 codes.

+ lVi + :S)

----""L - 5) on "almost-all'" burs,t-erroT correction for

. "'""·""~.codes forrateR = (11-l)/11, = k / 11. AH these codes have the on ''almos,t all'" burst-error conection is met that Yvhen the decoder is SuBivan [16] has shovvn ho"\,V errors in the

that the lower bound of ·with near Note that it is sensitive to errois in tJ:ie codes can be modified to the code rate.

codes are on block codes rather than convolntional codes. 'The overall code remains convolutionaL hovvever, since there memmy in the encodeL Consider an (11 = 3M. k = 2M) b1ock code with rate R = t., and

l

J

(21.38)

l. 1uPl]G = ·where G 1 and G2 are Mf x lVi sub1natrices of G. The codevvord v = [llii. rru(2). Jlll], where Rn< 11 and ur;( 21 contain IV/ information bits and fP is the M-bit vector

n»

=

+

The encoder for this code is shown in 21.10. Note ll1 addition to the block memory has been added to the encoder. This converts the block code into an (n = 3M.k = 2M.m = vU.v~•u .d

1150

Chapter 21

Burst-Error-Correcting Convolutionai Codes vi11 ~'">D

u-;;,,.@)-""uG

FIGURE 21.10:

Encoding circuit for the rate R

= 2/3 Tong burst-trapping code.

convolutional code. The code is still systematic, and the encoding equations at time unit l are given by .,(0) _ lill(l) 'I I .,(]) -

VI

(2)

""1

--

l!ll(2)

(21.40)

I

(1)

(2)

(1)

(2)

= Rill G1 + RU/ G2 + lilll-L + Illll-2L.

The decoding circuit for the Tong burst-trapping code is shown in Figure 21.11. Assume that the 1th block has just been received, that all past decoding decisions have been correct, and that the decoder is in the r-mode. The feedback of ihe decoding estimates lUl!~L and lUlj:_l2L to the parity input line of the block decoder removes the effect of past decisions from the encoding equations, so that the block decoder simply decodes the original block code; however, the block decoder is designed to correct only t' or fewer random errors, where t' < t, and the additional error-correcting capability of the code is used to detect patterns of t' + 1 or more errors. As long as the block decoder estimates t' or fewer errors in a block it remains

Section 21.4

Burst-·and-Random-Erro1·-Coi-recting Convolutional Codes

1151

FIGURE 21. ·11: Decoding circuit for the rate R = 2/3 Tong burst-trapping code.

in the r-mode. Note that, as for burst-finding codes, the decoding estimates of the block decoder are delayed by 2L time units before they are accepted as final. When the block decoder detects a pattern oft' + 1 or more errors, say at time unit l -- L, it emits a 1 from its "burst detected" output. L time units later. at time unit l, this 1 reaches the first mode selector in Figure 21.11 and causes it to switch to the b-mode for that block only. At this time the output of the first mode selector is A(l) lJ!l-L

[A(l) ]

] + [A(2) 1ll!l-2L r = (1ll1j 1l + e)0l)G1 + (un;2l + e; 0 )G2 + lll1j 1JG1 + lll1j2lG2

= ~.nl-L

(1)

/J

=

§/

j2)

(2)

[ (2)

]

(21.41)

+ Ril/-L + 1!Il/-2L + <el + lll11-2L r = un;~L + un}:n + [@f:n],. + e}0 JG1 + e71)G2 + efl. From our earlier assumption that past decoding decisions are all correct, [1ill):nJ,. (2) Also, assuming that time unit l comes from the error-free guard space

lll1 _ L. 1 2

1152

Chapter21

Burst-Error-Correcting Convolutional Codes

following the burst at time unit l - L, it follows that (21.42) and the b-mode decoding estimate will be correct. A similar argument shows that at time unit l + L, when the "burst detected" output reaches the second mode selector and causes it to switch to the b-mode for that block only, the output of the second mode selector will be the correct estimate for Mj~L, provided that time unit l + L also comes from the error-free guard space. The probability of failing to switch to the b-mode during a burst can be estimated as follows. Assume that an (n, k) block code is designed to correct t'

6.(11) + (/1) + · · · + (11) t' 1 0

or fewer errors. There are then a total of N (t') =

correctable error patterns. Hence, for each of the 2" codewords, there are N (t') received blocks that will be corrected and not cause a switch to the b-mode. Assuming that during a burst all 2" received blocks are equally likely, the probability of failing to switch to the b-mode during a burst is given by 2k

Pr(undetected burst)~ N(t')~ 211

=

. N(t')/2"-k.

(21.43)

IEXAMIPILIE 2Ul:

Consider a (30, 20) shortened BCH code with t' = 1. Then, N(t') = 1 + 30 = 31, and Pr(undetected burst)~ 31/2 10

= 3.0 x

10- 2 ,

(21.44)

which implies about a 3 'Yo failure rate in detecting bursts. The burst-trapping code just described corrects "almost all'' bursts that can affect at most L consecutive received blocks. Hence, b = (11 - l)L + 1 bits. The guard space must include at least 2L consecutive error-free blocks following the burst. Hence, g = 211L + (n - 1). Therefore, for large Land 11, ~ L

-

b-

211L+(11-l) ~ 2 (11 - l)L + 1 ~ '

(21.45)

which meets the lower bound of (21.2) on "almost all" burst-error correction for rate R = 2/3 codes. Although the preceding discussion concerned only rate R = 2/3 codes, bursttrapping codes can easily be generalized to any rate R = ').M / (A + l)M. These systems all meet the lower bound of (21.2) on "almost all" burst-error correction with near equality. Like the closely related burst-finding codes, burst-trapping codes are sensitive to errors in the guard space when the decoder is in the bmode. Burton et al. [17] have shown, however, that the system can be modified to provid~ so111P protPrtion Rgainst errors in the guard space by !ovJering the code rate.

Problems

'u u52:,

rr,,ol8l!E~~s 2JL]_

mathematical induction, show that the unknown elements of the matrix can be chosen so that (21. 11) is satisfied. 2,]_J; Show how to construct codes with k < 11 - l. 2lio2l Consider the Berlekamp-Preparata code with n = 3. m, Find m, b, and g for this code. [b), Find the JB.; 0 matrix.
ei

m

= (211 - l)Jc + (11 2

b

= 11A.

-

11 -

2)/2

g=n(m+l)-1

The 11 -1 generator polynomials are given by (21.20), where a(i) ~

i (n -

11 - 1, and b(i) ~ !(11 - i)(4Jc + n - i - 1) + n + Jc - 2. Consider the code with 11 = 3 and A = 3. m, Find 111, b, and g for this code. 2 [b), Find the generator polynomials g~ ) (D) and (D). 0 1'.o Find the repeat distance of the information error bits e/ ) and e}1) [Ilo Construct a decoding circuit for this code. Construct a general circuit for the class of codes iD Problem 21.5. For the two classes of codes: m, compare the excess guard space required beyond the Gallager bound; and lbio compare the number of register to a decoder. Show that for the Iwadare-Masseycode of Example 21.4, ifn[m+(Jc+2)n- l]-l = 95 consecutive error-free bits follow a decoding error, the syndrome will return to the all-zero state. Consider the (2, 1. 5) double-error-correcting code from Table 13.3 interleaved to degree ), = 7. in, Completely characterize the multiple-burst-error-correcting and the associated guard-space requirements of this interleaved code. [b), Find the maximum single-burst length that can be corrected and the associated guard space.
11 - i - 3) +

0

2liAii

Zlio't

2Ul

2]}\»

i )(4,\ +

-i-i54

Chapter 2·1

Burst-Error-Correcting Convolutional Codes

2L10 Consider the Berlekamp-Preparata code of Problem 21.3 interleaved to degree ?<. = 7. l!lo Find the g/b ratio and compare it with the Gallager bound. ibio Draw a biock diagram of the complete interleaved system. 2L11 Consider the n = 3 Berlekamp-Preparata code interleaved to degree ?<. = 7 and then= 3 Iwadare-Massey code with>..= 7. :ii. Compare the g / b ratios of the two codes. lb>o Compare the number of register stages required to implement the decoder in both cases. 21.12 Consider the (2, 1, 9) systematic code with g( 1)(D) = 1 + D 2 + Ds + D9. !llo Is this code self-orthogonal? What is fML for this code? lbio Is this a diffuse code? What is the burst-error-correcting capability b and the required guard space g? 1t'.o Draw a complete encoder/decoder block diagram for this code. 21013 For the diffuse code of Figure 21.7, find the minimum number of error-free bits that must be received following a decoding error to guarantee that the syndrome returns to the all-zero state. 21.14! Consider using the (2, 1, 11) triple-error-correcting orthogonalizable code from Table 13.3 in the Gallager burst-finding system. ru. Draw a block diagram of the encoder. ibio Draw a block diagram of the decoder. 11:o With t~L = 1, choose M and L such that the probabilities of an undetected burst and of a false return to the r-mode are less than 10- 2 and the g / b ratio is within 1 % of the bound on "almost all" burst-error correction for rate R = 1/2 codes. irllo Repeat (c) for t~L = 2. 21015 Consider the rate R = 2/3 burst-trapping code of Example 21.8. :llo Choose L such that the g / b ratio is within 1 % of the bound on "almost all" burst-error correction for rate R = 2/3 codes. !ho Describe the generator matrix G of the (30, 20, 2L) convolutional code.

BilBUOGRArHY

lo D. W. Hagelbarger, "Recurrent Codes: Easily Mechanized Burst-Correcting, Binary Codes," Bell System Tech. J., 38: 969-84, July 1959. 20 Y. Iwadare, "On Type Bl Burst-Error-Correcting Convolutional Codes," IEEE Trans. Inform. Theory, ff-14: 577-83, July 1968.

3. R. G. Gallager, Information Theory and Reliable Communication. McGraw-Hill, New York, 1968. 4t A. D. Wyner and R. B. Ash, "Analysis of Recurrent Codes," IEEE Trans. Inform. Theory, ff-9: 143-56, July 1963. So E. R. Berlekamp, "Note on Recurrent Codes," IEEE Trans. Inform. Theory, IT-10: 257-58, July 1964. 16io

F. P. Preparata, "Systematic Construction of Optimal Linear Recurrent Codes for Rnrst Error Correction," Ca!co!o, 2: 1-7, 1964.

andG, D. 1968. ft J. I.,. tion 1968.

Con1rnunica-

Ne·01

IEJE~E lriternational (Conference OE

:0/ash.<, June 1973, J~~L J. l,.

Con-vo}irtional Codes,'"

Jl2o Ml. J.

March 1971.

fl3lo S. Y. "Construction of in Proc. IEEE Intl.

June 1970. "IEEE

December 1971. :ilrti D.D. ~~

"A Generalization of Error Control lliE.t~ Trans. ff.17: 727-35. November 1971.

Jt1, H. 0. Burton. D. D. November 1971.

C H A P T rE fi

Ult-. is

o··. m ,, ·~·

,

22

411";

~ ~E~

Strate As pointed out in Chapter l, there are two categories of techniques for controlling transmission errors in data transmission systems: the forward-error-control (FEC) scheme and the automatic-repeat-request (ARQ) scheme. In an FEC system an error-correcting code is used. When the receiver detects the presence of errors in a received vector it attempts to determine the error locations and then corrects the errors. If the exact locations of errors are determined, the received vector will be correctly decoded; if the receiver fails to determine the exact locations of errors, the received vector will be decoded incorrectly, and erroneous data will be delivered to the user (or data sink). In an ARQ system a code with good error-detecting capability is used. At the receiver, the syndrome of the received vector is computed. H the syndrome is zero, the received vector is assumed to be error-free and is accepted by the receiver. At the same time, the receiver notifies the transmitter, via a return channel, that the transmitted codeword has been successfully received. If the syndrome is not zero, errors are detected in the received vector. Then, the transmitter is instructed, through the return channel, to retransmit the same codeword. Retransmission continues until the codeword is successfully received. With this system, erroneous data are delivered to the data sink only if the receiver fails to detect the presence of errors. Ha proper linear code is used, the probability of an undetected error can be made very small. We have devoted many chapters to various types of error-correcting codes and decoding methods for FEC. fo this chapter we present various types of ARQ schemes and the combinations of ARQ and FEC, called hybrid-ARQ schemes. Major references on ARQ are listed at the end of this chapter.

22.1

BASK ARQ S(HIEMIES There are three basic types of ARQ schemes: the stop-and-wait ARQ, the go-back-N ARQ, and the selective-repeat ARQ. In a stop-and-wait ARQ data transmission system, the transmitter sends a codeword to the receiver and waits for an acknowledgment from the receiver, as shown in Figure 22.1. A positive acknowledgment (ACK) from the receiver signals that the codeword has been successfully received (i.e., no errors were detected), and the transmitter sends the next codeword. A negative acknowledgement (NAK) from the receiver indicates that the received vector has been detected in error; the transmitter resends the codeword. Retransmissions continue until an ACK is received by the transmitter. The stop-and-wait ARQ scheme is simple and is used in many data communication systems; however, this scheme is inherently inefficient because of the idle time spent waiting for an acknowledgment for e::ich transmitted codeword. Unless the cede ler1gth n i:; extremely long, the fraction of idle time can be large; however, using a very long

Section 22" 1

Basic ARQ Schemes

'u 157

Retransmission

Idle time

l

Transmitter Transmission Receiver Error

Error

FIGURE 22" 1: Stop-and-wait ARO"

a since the probability that a block contains errors increases with the block length" Hence, idle time is reduced at the expense of increased of retransmissions for each codeword" Moreover, a long block may be impractical in many applications because of restrictions imposed the datao fa a go-back-N ARO codewords are transmitted The transmitter does not wait for an acknowledgment after sending a as soon as it has one, it begins sending the next codeword, as shown in Figure 222" The acknowledgment for a codeword arrives after a round-trip delay, which is defined as the time interval between the transmission of a codeword and the receipt of an acknowledgment for that codeword" During this interval, N - 1 other codewords have also been transmitted" When a NAK is received, the transmitter backs up to the codeword that was negatively acknowledged and resends that codeword and N - 1 succeeding codewords that were transmitted during the Of the transmitter pulls back and resends N course, a buffer must be provided at the transmitter for these codewords" At the the N - 1 received vectors following an erroneously received vector ( a received vector detected in error) are discarded regardless of whether they are error-free or not. Therefore, the receiver needs to store only one received vector at a time" Because of the continuous transmission and retransmission of codewords, the go-back-N ARO scheme is more effective than the stop-and-wait ARO, and it can be implemented at moderate cost The go-back-N ARO scheme becomes Round-trip delay Transmitter Transmission

Retransmission ( Go-back-7)

t

1 2 3 4 5

~ --,-~~-+-+-----f'----f~F----~~-f---f'--(--+-f'------f'----t~~~+--+---+--+-+-----f'----f'--r-

Receiver Error

FIGURE 222: GO-back-NARO with N =

70

1158

Chapter 22

Automatic-Repeat-Request Strategies

Transmitter Transmission Receiver Error

Error

FIGURE 22.3: Selective-repeat ARQ.

ineffective when the round-trip delay is large and the data transmission rate is high. This inefficiency is caused by the retransmission of many error-free codewords following a codeword detected in error. This inefficiency can be overcome by using the selective-repeat strategy. In a selective-repeat ARQ system, codewords are also transmitted continuously; however, the transmitter resends (or repeats) only those codewords that are negatively acknowledged, as shown in Figure 22.3. Because codewords must ordinarily be delivered to the user in correct order, a buffer must be provided at the receiver to store the error-free received vectors following a received vector detected in error. When the first negatively acknowledged codeword is successfully received, the receiver then releases the error-free received vectors in consecutive order until the next erroneously received vector is encountered. Sufficient receiver buffer must be provided; otherwise, the buffer may overflow, and data may be lost. The selective-repeat ARQ is the most efficient one among the three basic ARQ schemes; however, it is also the most complex one to implement. In an ARQ system the receiver commits a decoding error whenever it accepts a received vector with undetected errors. Such an event is called an error event. Let P(E) denote the probability of an error event. Clearly, for an ARQ system to be reliable, P(E) should be very small. Therefore, the reliability of an ARQ system is measured by its error probability P(E). Suppose that an (n, k) linear code C is used for error detection in an ARQ system. We define the following probabilities: Pc Pd Pe

= probability that a received vector contains no error; = probability that a received vector contains a detectable error pattern; = probability that a received vector contains an undetectable error pattern.

These probabilities add to 1 (i.e., Pc + Pd + Pe = 1). The probability Pc depends on the channel error statistics, the probabilities Pd and Pe depend on both the channel error statistics and the choice of the (n. k) error-detecting code C. A received vector will be accepted by the receiver only if it either contains no error or contains an undetectable error pattern. Therefore, the probability P(E) that the receiver commits an error is given by Pe P(E)=-(22.1) Pc + Pe The probability Pe can be mude ·very small relative to I'c by choosing the code C properly (e.g., a long Hamming code or a long double-error-correcting primitive

Section 22.1

Basic ARQ Schemes

1 i 5Sll

can be made very small. For

= (1 - p)".

fo Section 3.6 ·we showed that for an average (n, k) linear ::: r(n-k)

[l - (1 -

(22.2), and (22.3). ·we can obtain an upper bound on P

Consider a BSC with transition p = that vve use the (1023, 1003) BCH code for error detection in an

and

::: 2- 20 for p ::: 1/2]. We see that

the bound we find that

:::: 2-
p

<

This result illustrates that using a relatively small number of ARO can achieve very

an

Another measure of the performance of an ARO is its efficiency ( or simply which is defined as the ratio of the average number of information successfully by the receiver per unit of time to the total number of that could be transmitted per unit of time [L 2]. AH three basic ARO schemes achieve the same however, have different for each of the three basic efficiencies. Next, we derive the ARO schemes. For simplicity, we assume that the feedback channel i.s noiseless. The assumption may not be realistic, but the results give a good indication of the effectiveness of each ARO scheme. For throughput of various ARO schemes with noisy feedback channel, the reader is referred to [l]. We first derive the throughput of the selective-repeat ARO scheme. VVe recall that with this scheme the transmitter sends codewords to the receiver and resends those codewords that are negatively The that a received vector will be accepted the receiver is P

=

For the usual situation where << codeword will be retransmitted is simply

+ Pe. then P

Pc1=l-P~l-

~

Pc. The

that a

1'16(]

Chapter 22

Automatic-Repeat-Request Strategies

For a codeword to be successfully accepted by the receiver, the average number of retranr,mission::; (includi1.1.g the 0Iigb:1al t1-~ausu1issio:a) ;e,quired is TsR

= 1 · P + 2 · P(l

- P)

+ 3 · P(l -

P)2

+ ·· · + l

· P(l - p)1-l

+ · ··

1 p

Then, the throughput of the selective-repeat ARQ is l)SR

= _l (~) = (~) P,

TsR

11

(22.4)

11

where k / 11 is the rate of the (n, k) code used in the system. We see that the throughput 11sR depends only on the channel error rate. In a go-back-N ARQ system, when a codeword is negatively acknowledged, the transmitter resends that codeword and the N - 1 subsequent codewords that were transmitted earlier. Therefore, for a codeword to be successfully accepted by the receiver, the average number of retransmissions (including the original transmission) required is TcsN

= 1 · P + (N + l)P(l +(IN+ l)P(l -

+ (2N + l)P(l P)1 + · · · P)

P)

2

+ ...

= l + N(l-P). p

Consequently, the throughput efficiency of a go-back-N ARQ system is l)GBN =

1

TcsN

(k) = -;;

p

PP)N (k) -;; .

+ (1 -

(22.5)

We see that the throughput 7JGBN depends on both channel block error rate 1 - P and the round-trip delay N. When the channel error rate is low, the effect of the round-trip delay, (1 - P)N, is insignificant and the throughput is high; however, the effect of (1 - P)N becomes significant when the channel error rate is high, and the throughput is sharply reduced. The go-back-N ARQ provides satisfactory throughput performance for systems in which the round-trip delay is small and the data transmission rate is not too high. Otherwise, its throughput performance becomes inadequate. In deriving the throughput of a stop-and-wait ARQ system, we must consider the idle time spent waiting for an acknowledgment for each transmitted codeword. Let D be the idle time from the end of transmission of one codeword to the beginning of transmission of the next. Let r be the signaling rate of the transmitter in bits per second. In one round-trip delay time, the transmitter can transmit a total of n + D · r digits if it does not stay idle. During ihe interval from the beginning of transmission of one codeword to the receipt of a positive acknowledgment for that codeword, the average number of digits (including the idling effect) that the transmitter could have transmitted is Tsw

=

+ Dr)P + 2(11 + Dr)P(l n + Dr

(n

p

P)

+ 3(11 + Dr)P(l - P)2 + · · ·

Basic P.RQ Schemes

Secdon 22.1

ARQ

of a

the

,D

' f( /)SW=

(~~)'. \11 '

+

n©1

is (22.6)

The factor may be as the number of codevvorcls that could be trans1nitted the idle time of the transmitter. "We see that the can For never achieve the maximum value data transmission in which the data tn1nsrnission rate is low and the roundis with the code

communication

scheme is the least efficient one. The not ,P>"JVU~'

0.8

>-, Cl ;::l 0)

·u 0.6 ~~

-"' _,

0.

..c: bl) ;::l

0

a::'"

0.4

r-,

0.2

Bit-error rate

receiver (the solid triangles represent simulation

ARQ with infinite a receiver buffer of size N = 128 and the go-back-N ARQ.

1162

Chapter 22

Automatic-Repeat-Request Strategies

and the parameter Dr/ n for the stop-and-wait ARQ may become very large. In thlc:

r~cp

the thrnHghpnt

fnr

th,,::. g.0,-h?irk_.~_7\f b. lDQ rlir.0.p~

?C:

tlhP

,-,,h~;1no1

error rate increases, whereas the throughput of the stop--and-wait ARQ becomes unacceptable. The high throughput of the selective-repeat ARQ is achieved at the expense of extensive buffering at the receiver and more complex logic at both transmitter and receiver. Theoretically, infinite buffering is needed to achieve the efficiency (k/n)P. H a finite buffer is used at the receiver, the buffer may overflow, which reduces the throughput of the system; however, if sufficient buffer (say, a buffer that is capable of storing N codewords) is used at the receiver, even with a reduction in throughput. the selective-repeat ARQ still significantly outperforms the other two ARQ schemes in systems for which data transmission rate is high and round-trip delay is large [4, 5, 10-12]. This situation is discussed in the next section. Figures 22.4 to 22.6 show the throughput efficiencies of the selective-repeat ARQ and the go-back-N ARQ for various code lengths and round-trip delays. The channel is a BSC. Several variations of the go-backN ARQ scheme have been proposed [3, 6, 8, 9]. These variations improve the throughput efficiency; however, they are still less efficient than the selective-repeat ARQ scheme.

0.2

Selective-repeat ARQ with a receiver buffer of size N

IO

10

c,

-1

Bit-error rate

FIGURE 22.5: Throughput efficiencies: the ideal selective-repeat ARQ with infinite receiver buffer. the selective-repeat A RQ with a receiver buffer of size N = 512 (the solid triangles represent simulation results), and the go-back-N ARQ.

Section 222

Selective-Repeat ARQ System with Finite Receiver Buffer

i %3

l.O ---- ---T--------1-1<-~

"\ -""

-"

"\

O.S

f

N

=

11

= 1024

1024 \

06

\ \

\

\

Go-back-N ARQ \

0.2

0 10-X

) Selective-repeat with a receiver buffer

,

o[sizeN

\

I

. \

\

..;_.J_.il!~~~w._~_...i_..L..W:::f;;.~-----..J=6l,3 ~ ~ J d d , /

10-r,

10--l

10-"

Bit-error rate

FIGURE 22.6: Throughput efficiencies: the ideal selective-repeat ARQ with infinite receiver buffer., the selective-repeat ARQ with a receiver buffer of size N = 1024 (the solid represent simulation results), and the go-back-N ARQ.

fo this section we present a practical selective-repeat ARQ system in which the receiver employs a finite-size buffer [10, 11]. Let N be the number of codewords that can be transmitted during a round-trip delay period. For simplicity, we consider only the case for which the size of the receiver buffer is N (i.e., the receiver buffer is capable of storing N Each codeword to be transmitted has a sequence number. The range of the sequence number is set to 3N (i.e., codewords are numbered from 1 to 3N). These numbers are used cyclically (i.e., a new codeword following a codeword with sequence number 3N is numbered with 1 again). For a received buffer of size N, if the range of the sequence numbers is 3N, the receiver will be able to distinguish whether a received vector with sequence number q is a new vector or a vector that has been accepted and delivered [5]. (This will become clear in the following discussion.) Using the numbering system described, m = Pog2 3Nl bits must be appended to each codeword. Suppose that an (n. k) linear code is used for error detection. When a vector is received its syndrome is computed. If no errors are detected, the received vector is either delivered to the user or stored in the receiver buffer until it is ready to be delivered in the correct order. H the received vector is detected in error, it is discarded and a space in the receiver buffer is reserved for storing that vector at a later time.

1164

Chapter 22

Automatic-Repeat-Request Strategies

When a codeword is ready for transmission it is numbered and stored in Lin:, iupUi qvu:uc

v; l;1C uau;:,u1~Lic..1.

A[Le1

~l-' uau".111~.-,,;:.,~vu, tl1v L,vdvVVv1d ;~ ;:,aVcd

in the retransmission buffer of the transmitter until it is positively acknowledged. The acknowledgment arrives after a round-trip delay. During this interval N - 1 other codewords are transmitted. A codeword is said to be a time-out vector if it is transmitted ( or retransmitted) for a round-trip delay time. When a codeword in the retransmission buffer becomes a time-out vector, it must be either positively or negatively acknowledged. When an ACK for a codeword in the retransmission buffer is received, the codeword is released from the retransmission buffer; however, when a NAK for a codeword in the retransmission buffer is received, a retransmission of that vector is initiated. A third possibility that neither an ACK nor a NAK is received when a codeword becomes a time-out vector ( e.g., the acknowledgement is lost in the return channel). In this event the transmitter regards that the unacknowledged time-out vector was unsuccessfully transmitted or lost, and retransmission is initiated for this vector. Before we detail the transmission and retransmission process, we introduce another concept. Let qo be the sequence number of the earliest NAK'ed (negatively acknowledged) or unACK'ed (unacknowledged) codeword in the retransmission buffer. The forward index (FWX). denoted by jy, of a codeword with sequence number q in the retransmission buffer or input queue with respect to the earliest NAK'ed or unACK'ed codeword in the retransmission buffer is defined as the remainder resulting from dividing q - qo by 3N. Mathematically,

.h = q - qo(mod 3N). Clearly, 0 22.2.1

s: h

(22.7)

< 3N.

Trn1111smissio11 am:I ~etrrzm1smissio1111 lfl>rnceroh.11rn

When the transmitter sends a codeword it also computes the forward index fy of the codeword in the retransmission buffer that is to become a time-out vector (we will refer to this vector as the current time-out vector). Then, the transmitter decides whether the next codeword in the input queue is to be transmitted or a retransmission is to be initiated. The decision rules follow: :Il.. If the current time-out vector is positively acknowledged and its forward index fr is less than N (or there is no time-out vector in the retransmission buffer), the first codeword in the input queue is to be transmitted. If the input queue is empty, the transmitter sits idle until a new codeword arrives or a retransmission is initiated. 2. If the current time-out vector, say VJ, is either NAK'ed or unACK'ed and its forward index fr is less than N, a retransmission for VJ is initiated. If the current NAK'ed (or unACK'ed) time-out vector is the earliest vector in the retransmission buffer that has not been positively acknowledged (h = 0), all the codewords in the retransmission buffer with forward indices equal to or greater than N are moved back to the input queue for retransmission at a later time. These are the codewords that have teen transmitted; however, whe11 they arrive at the receiver, the receiver buffer is full and has no space to store

Section 22.2

Selective-Repeat ARQ System with Finite Receiver Buffer

Hili5

Retransmission Round-trip delay ~

Transmitter Transmission Receiver Error

Error

Cock,Yord 3 is recovered. Receiver releases coclc\\'ords :\, ..J.. and 5.

Error Cocle,\,orcis 13 to 17 arc rejected by the receiver o\\'ing to buffer overflow

Codeword 6 is recovered. Receiver releases codewords 6 to 12.

them (this event is referred to as Therefore. these codewords must be retransmitted 21c If the forward index .fr of the current time-out vector is to or greater than N. the first codeword in the queue is the next to be transmitted (this vector may be a codeword that ,vas moved back to the input queue from the retransmission buffer to the receiver buffer overflow). The transmission and retransmission Figures 22.7 and 22.8.

J

l

of the transmitter are illustrated in

the transmitter sends codewords to the receiver. The receiver checks the received vector and sends an ACK to the received vector. When the channel is quiet the error-free codewords are delivered to the user in consecutive order, and the receiver buffer is The receiver is said to be in the normal state when the receiver buffer is and no space is reserved for any NAK'ed block. If the received vector is detected in error, or a received vector with an outof-order sequence number is detected while the receiver is in the normal state. the receiver sends a NAK to the transmitter and enters the blocked state. In the blocked state the receiver of the incoming received vectors, it stores those vectors that have zero at the proper locations in the receiver buffer. and reserves proper locations for the vectors whose syndromes are not zero until the receiver buffer is full. No vectors are delivered to the user. When the retransmitted codewords arrive after a round-trip their syndromes are checked again. The vectors that are received (zero syndrome) are stored at the reserved locations in the receiver buffer. Once the earliest NAK'ed vector is successfully received. the receiver outputs that vector and all the subsequent consecutive zero-syndrome vectors (held in the receiver buffer) until the next NAK'ed vector is encountered (see Figure 22.7). If all the vectors in the receiver buffer are released and all the 1·eserved locations are freed after the earliest NAK'ed vector has been successfully received. the receiver returns to the normal state.

1166

Chapter 22

Automatic-Repeat-Request Strategies Transmitter

No time-out vector?

No

..----"-'-'---cAcknowledgment received?

ACK

Transmit the first vector in the input queue and save it in the retransmission buffer

NAK

No

Retransmit the current time-out vector

Is the current time-out vector the ------!----"N_o'----<: earliest vector that has not been positively acknowledged?

Yes Move all the codewords in the retransmission buffer that have FWX f, 2- N back to input queue

FIGURE 22.8: Transmission and retransmission procedure.

If not all the vectors in the receiver buffer can be released to the user, or if there are still reserved locations in the receiver buffer for the NAK'ed vectors, the receiver continues its error recovery process until all the NAK'ed vectors are successfully received. Next, we describe the detailed operation of the receiver in both the normal and the blocked state.

22.2.3

Normal-State Operation

When a vector with sequence number q is received, the receiver checks its syndrome and computes its forward index (FWX), denoted by f R, with respect to the last accepted and delivered vector. Let qo be the sequence nun1ber of the last accepted and delivered vector. The forward index f R of the current received vector is defined

Section 22.2

Selective-Repeat ARQ System with Finite Receiver Buffer

as the remainder resulting from

!R = q

CJ -

- qo(

qo

ni57

3N; that is,

mod 3N).

(22.8)

H no errors in the current received vector are detected . and f R = 1 UR = 1 indicates that the current received vector has the correct sequence the received vector is and delivered to the user. The receiver then checks the next ""'-,,..... ,.., received vector. If either the current received vector is detected in error, or its forward index .fR is not equal to 1 but less than N + 1(1 < f R :::: N indicates that the sequence number of the current received vector is out of order), the receiver enters the blocked state. There are three cases to be considered. fo the first case, for which the is not zero and .fa = 1. the receiver discards the erroneously received vector and reserves the first location of the receiver buffer for that vector. In the second case, for which the syndrome is not zero and fR > l, f R - 1 vectors between the last delivered vector and the current received vector are lost. The receiver rejects the current erroneously received vector and reserves the first ./R locations of the receiver buffer for the lost vectors and the rejected vector. In the third case, for which the syndrome is zero but .lR > 1, .lR - 1 vectors between the last delivered vector and the current received vector are lost. The receiver then reserves the first {R - 1 locations of the receiver buffer for the lost vectors and saves the current received vector at the .fRth location of the receiver buffer. H the forward index .fR of the current received vector is greater than N, it is as a vector that was previously and it is then ignored, and an ACK is sent to the transmitter. The receiver in the normal state and processes the next vector. The flowchart shown in Figure 22.9 details the receiver operations in the normal state. 22.2.t!J.

~k:»df:e!Dl-Stzr~e O(Olerrzru:i
The receiver enters the blocked state when either a received vector is detected in error or a received vector with an out-of-order sequence number is detected. When the receiver enters the blocked state, each subsequent zero-syndrome received vector is held in the receiver buffer at a proper location, and a space in the receiver buffer is reserved for each erroneously received or lost vector. When the buffer is fuU, all subsequently received new vectors are rejected and the buffer overflows. Let w1 be the earliest vector that has not been successively received. When a retransmitted copy of w1 is successfully received, the receiver releases w1 and the subsequent zero-syndrome vectors held in the receiver buffer (see Figure 22.7). Suppose that the receiver can release L + 1 consecutive vectors including Wj (i.e., , wJ+ 1 , · · ·WJ+L, where O::: L < N). Because the retransmission is selective, and since wi+ 1 to Wj+L were successfully received before Wj, the vectors following the retransmitted copy of Wj are new vectors WJ+N, wi+N+l, · · · , 'VJ+N+L-1· When these L new vectors arrive their syndromes are computed. The vectors that have zero syndrome are then temporarily stored in the receiver buffer until they are ready to be released, and proper locations in the receiver buffer are reserved for those vectors that have nonzero syndromes.

1168

Chapter 22

Automatic-Repeat-Request Strategies

Yes

Send ACK(j) to the transmitter and store vi in the receiver buffer

Send ACK(j) to the transmitter and deliver vector vi to the user

Send ACK(j) to the transmitter and reserve a space in the buffer for vi

Enter the blocked state

FIGURE 22.9: Normal-state operation of the receiver.

If the retransmitted copy of VJ is not successfully received, the zero-syndrome vectors held in the receiver buffer cannot be released to the user. Therefore, there are no places in the receiver buffer to store the new vectors, VJ+N, vi+N+l, · · · (since the buffer size is N). In this event, the buffer overflows and the new vectors, "IJ+N, vi+N+l, ···,will be rejected no matter whether they are successfully received or not. Let qo be the sequence number of the earliest vector that has not been successfully received. In the blocked state, vihen a vector \vith sequence nun1ber q is received its forward index with respect to the earliest unsuccessfully received vector, denoted by If, is computed. This forward index If is defined as the remainder resulting from dividing q - qo by 3N; that is, ff

=

q - qo(

mod 3N).

(22.9)

If the current received vector has zero syndrome and its forward index lf < N, it is stored in the receiver buffer and an ACK is sent to the transmitter. If the current received vector is detected in error and its forward index f.t < N, it is rejected and a space in the receiver buffer is reserved for this vector; also, a NAK is sent to the transmitter. If lf 2:: N, the receiver then computes the backward index (BWX) h of thP. rP.r-P1-uPr1 "\TP.r'tQr u:1-itl, rP.C'pPr-t tfl tl,p. l';lct n:"r-P-iuP.r1 'iJPr>tnr th<:lt U.l'.'.:lC' ctnrP.rl ln thP

receiver buffer or for which a space in the receiver buffer is reserved. This backward

Section 22.2

nG!ll

Selective-Repeat ARQ System with Finite Receiver Buffer

index h is defined as the remainder

from

qo - q by 3N; that

(22.10)

q( mod 3N),

lb= qo

where qo is the sequence number of the last received vector that was stored in the receiver buffer or for which a space in the receiver buffer is and q is the sequence number of the current received vector. H l1, < the current received vector is regarded as a vector that was and delivered. H is then and an ACK is sent to the transrn.itter. If lb ::: 2N, the current received vector is a new vector; however, since the received buffer is the buffer overflows. In this event, the current received vector is rejected and a NAK is sent to the transmitter. The receiver's in the blocked state is detailed the flowchart shown in 22.10 .

•~ ~ - - - - - - - - - - - - - - , ' " > - - ~

Receive vi

Has v1 already been recovered and stored in the buffer? Send NAK(f) to the transmitter

Send ACK(j) to the transmitter

l

-----1 No

V

----

'i/ Reject V; and send NAK(j) to the transmitter

Yes

Is v; the earliest vector held in the buffer?

Output v1 and all the subsequent consecutive error-free codewords

No

Store v;

Return to the normal state

Yes

FIGURE 22.10: Blocked-state operation of the receiver.

1170

Automatic-Repeat-Request Strategies

Chapter 22

22.2.5 Thrnughput Effidency If we assume that the channel errors are randomly distributed (i.e., a BSC) and

the feedback channel is noiseless, we can derive a lower bound on the throughput efficiency of the selective-repeat ARQ system [11]. Suppose that a proper (n, k) code is used for error detection. Let p be the transition probability of the BSC. Then, the probability P that a received vector will be accepted by the receiver is P ~Pc= (1 - p)11.

Let ¢0

=

1 - (1 -

¢2 = 1 - (1 - P) 4 .

P)2,

These are the probabilities of success of the second, third, and fourth retransmissions of a codeword, respectively. We define A

=

>c1

= p2(cpt-2 + (1 - P)cpf-2 -

>c2

= 3 - P 2¢t- 2 -

0

___
¢5¢f- 2

-

(1 - P)2¢i-2},

c/Jic/Ji- 2 ·

Then, the throughput efficiency of the selective ARQ with a receiver buffer of size N is lower bounded as follows: 1}SRF ::':

AO+ A~O+ >c2N (~).

(22.11)

(Assuming that the number of bits used for sequence numbers is small compared with the code length n, its effect on the throughput is ignored.) For various n and N, the lower bound on the throughput given by (22.11) is compared with the throughput of the ideal selective-repeat ARQ with infinite receiver buffer and the throughput of the go-back-N ARQ, as shown in Figures 22.4 to 22.6. We see that the selective-repeat ARQ with a receiver buffer of size N is less efficient than the ideal selective-repeat ARQ because of the reduction in buffer size at the receiver; however, it significantly outperforms the go-back-N ARQ, particularly for comrnunication systems in which the round-trip deiay is large and data rate is high. If the round-trip delay is taken to be 700 ms (suitable for satellite channels), the data transmission rate corresponding ton = 524 and N = 128 (Figure 22.4) is 100 kbps, the data rate corresponding to n = 2024 and N = 512 (Figure 22.5) is 1.54 Mbps, and the data rate corresponding ton = 1024 (Figure 22.6) is also l.54 Mbps. The selective-repeat ARQ described in this section can be extended for any receiver buffer of size greater than N, say 2N, 3N, · · ·. Of course, the throughput efficiency increases as the buffer size increases. Figure 22.11 shows the throughput with buffer size equal to 2N. Selective-repeat ARQ that operates with a finite receiver buffer and a finite range of sequence numbers was first studied by Metzner [5]. Other selective-repeat i-\..RQ schemes employing finite receiver buffers can be found in [~, 12, 13, 18-20, and 25].

Section 22.3

ARQ Schemes with Mixed Modes o-F Retransmission

nn

0.8 N = 512 2024

11 =

G

5

Tl

0.6 -

8

Q)

0.2

Bit-error rate

Throughput of the (1) size N and (2) size where N = 512.

FIGURE 22.11:

22.3

ARQ with a finite receiver buffer:

~,ri.Q S(Mt!VitS WffM M!}{[EI) MODIE:S Or f'.l!Entfolj\JSM!SS!O~! It is possible to devise ARQ schemes with mixed modes of retransmission. One such

scheme is the selective-repeat plus go-back-N + GBN) ARQ [12]. When a codeword v in the retransmission buffer becomes the earliest negatively acknowledged vector, the transmitter resends wand other vectors in the selective-repeat (SR) mode (i.e., only 01 and other negatively vectors are retransmitted). During the SR mode, the receiver stores those vectors that are successfully received. The transmitter remains in SR mode until either "!/ is positively acknowledged or has been retransmitted v times but the transmitter fails to receive an ACK for v. In the first event, the transmitter to send new codewords. When the second event occurs the transmitter switches to the (GBN) retransmission mode. The transmitter sending new backs up to w, and resends v and N - 1 succeeding vectors that were transmitted the vth SR retransmission attempt for v. The transmitter in the GEN mode until vis positively acknowledged. At the receiving when the vth retransmission of v is detected in error all the subsequent l\f - 1 received vectors are discarded regardless of whether they are successfully received or not. The receiver continues this operation until v is successfully received. Then, the receiver releases \Y and all the consecutive error-free vectors following v that were held in the receiver buffer. When the SR+ GBN ARQ

1172

Chapter 22

Automatic-Repeat-Request Strategies

is used the receiver buffer must be able to store v(N - 1) + 1 vectors to prevent buffer overflow. Figure 22.12 illustrates the transmission/retransmission procedure for SR + GBN with v = 1, namely one retransmission for a NAK'ed vector in SR mode and all subsequent retransmissions of the same vector in GBN mode. Following the second consecutive NAK for a given vector (e.g., block 7 in Figure 22.12) the receiver simply discards all N - 1 subsequent vectors until the successful reception of the erroneous vector. In the transmitter and in the receiver, counters would be required to keep check on the number of transmission/retransmission attempts made for each

FIGURE 22.12: SR+ GBN ARQ for v

= 1 and N = 5.

Section 22.3

nn

A.RQ Schemes with Mixed Modes of Retransmission

vector to determine whether an SR or GEN type of retransmission is If more than one vector that has been transmitted twice unsuccessfully is stored in the transmitter retransmission buffer. these vectors must be for successive retransmissions. The earliest double-NAK"ed vector, say v.i, is retransmitted in the GEN followed N - 1 vectors that were transmitted after vj and discarded by the receiver. This action is repeated until an ACK for VJ is received. Then. the same procedure is for any double-NAK"ed vectors in the retransmission buffer. The of the SR+ GBN ARQ has been [12]. For

171

where P is the

=

(k)-

(22.12)

11'

that a vector will be successfully received. For v > 1, l)v -

For v

p

1-(N-1)(1-P)2

p

(k)

(22.13)

1- (N-1)(1- P)v+l \;;-) ·

= 0, the SR+ GBN scheme becomes the conventional go-back-JV ARO, and 170 =

_P_-1--_(l_P_--(~).

Figure 22.13 demonstrates the throughput performance of the SR + GBN ARO for v = 1 and 2. Throughput efficiencies of other ARO schemes are included

I

0.2

Ideal select-repeat ARQ

O'--~~~-'--~~~-'---~~~=:s""'"--=-"'-~-'----~-""----~--'-~~ 10-7 10-r. 10-s Bit-error rate

FIGURE 22.13: Throughput efficiencies of the SR+ GBN with v and n = 2024.

= 1 and 2,

N

= 512

1114

Chapter 22

Automatic-Repeat-Request Strategies

in the same figure for comparison. We see that SR + GBN ARQ significantly outperforms the conventional go-back-NARO in throughput for high bit-error rate and large round-trip delay. We also see that with the same receiver buffer size, the SR+ GBN ARQ is inferior to the selective-repeat ARQ described in Section 22.2 for high channel error rate; however, SR+ GBN ARO requires simpler logic at the transmitter and receiver than the selective-repeat ARO. In selective-repeat ARQ, when a codeword in the retransmission buffer is negatively acknowledged a single copy of this NAK'ed codeword is retransmitted. A simple variation of this single-copy retransmission is to repeat a NAK'ed codeword multiple times to increase the probability that a retransmitted codeword will be received successfully. This multiple-repetitions strategy is called stuttering (ST). The first selective-repeat ARQ incorporated with a stuttering mode of retransmission, known as SR+ST ARO, was proposed by Miller and Lin [12] and was later improved by Weldon [13]. The basic idea of Weldon's SR+ ST ARO scheme is to repeat a NAK'ed codeword multiple times, with the number of repetitions increasing each time until the buffer overflows. The scheme operates as follows:

1. First retransmission When a codeword is negatively acknowledged for the first time it is repeated n 1 times in its first retransmission. Z. Second retransmission Hall n 1 copies in the first retransmission of a codeword are negatively acknowledged, the codeword is repeated 112 times in its second retransmission. q. qth retransmission If all the nq-l copies in the (q - l)th retransmission of a codeword are negatively acknowledged, the codeword is repeated nq times.

After q retransmissions, if a codeword is still negatively acknowledged, the buffer overflows at the receiver. The transmitter continues to retransmit the codeword. In each retransmission the codeword is repeated nq times. The transmitter sends new codewords whenever it does not have repetitions to send. This process continues until the codeword is positively acknowledged. For q = 1 and n 1 = 1, the SR+ ST ARQ becomes the conventional selective-repeat ARO. The throughput performance of Weldon's SR+ ST ARO depends on the choices of the repetition parameters 111. n2. · · · • nq. These parameters can be chosen to optimize the throughput performance for a given receiver buffer size. This AR + ST ARO provides some improvement in throughput efficiency over the selective-repeat ARO with finite receiver buffer presented in Section 22.1; however, it is harder to implement. 22.4

HYBRm ARQ SCHIEMIES

Comparing the FEC and ARQ error-control schemes, we see that ARQ is simple and provides high system reliability; however, ARO systems have a severe drawback-their throughputs fall rapidly with increasing channel error rate. Systems using FEC maintain constant throughput (equal to the code rate R = k / n) regardless of the channel error rate; however, FEC systems have two drawbacks: First, when a received vector is detected in error it must be decoded and the decoded message must be delivered to the user regardless of whether it is correct or incorrect.

Section 22.4

Hybrid ARQ Schemes

'u 'u 75

Because the of a decoding error is much an undetected error, it is hard to achieve high to obtain a long code must be used and a large must be corrected. This makes hard to impledata netvvorks and computer communication in communication data storage) where return channels are not available or retransmission is not possible for some reason, FEC is the choice. The drawbacks in both ARO and FEC can be overcome if two error control schemes are combined. Such a combination of the two basic control A_ consists of an schemes is referred to as a FEC contained in an ARO The function of the FEC subsystem is to reduce the correcting the error that occur most throughout VVhen a less error occurs and is the receiver a retransmission rather than passing the unreliably decoded message to the user. This increases the system reliability. As a a proper combination of FEC and ARO higher reliability than an FEC alone and a higher than the system with ARO since the decoder is designed to correct a small collection of error patterns, it can be simple. The FEC scheme can be with any of the three basic schemes. A ARO scheme is to use a code, say, an (11, k) linear code that is designed to simultaneously correct and detect errors. When a received vector is detected in error the receiver first attempts to locate and correct the errors. If the number of errors the length of an error burst) is within the designed of the the errors are and the decoded message is passed to the user or saved in a buffer until it is ready to be delivered. If an uncorrectable error is the receiver rejects the received vector and a retransmission. 'When the retransmitted vector is received the receiver to correct the errors H the is not successful, the receiver again the received vector and asks for another retransmission. This error-correction and retransmission process continues until the vector is successfully received or decoded. For example, one may use the (1023, BCH code in a hybrid ARO system. This code has a minimum distance of 21; it can be used for correcting five or fewer errors and simultaneously detecting any combination of 15 or fewer errors (and many other error patterns). Han error pattern with 5 or fewer errors occurs, it will be detected and corrected. H an error pattern with more than 5 but less than 16 errors occurs, it will be detected. In this event the receiver will request a retransmission of the erroneous vector. The hybrid ARO scheme described is referred to as the type-I hybrid ARQ scheme. Because a code used in this scheme must be able to correct a certain collection of error patterns and simultaneously detect other error patterns, more parity-check digits are needed, which increases the overhead for each transmission and retransmission. As a result, when the channel error rate is low, the type-I hybrid ARO has a lower throughput than its corresponding ARO scheme; however, when the channel error rate increases, the throughput of the ARO scheme drops rapidly, HD.m,Afo

1 H6

Chapter 22

Automatic-Repeat-Request Strategies

t

Re~~c;~~~ i~!~/;~y~~?~\_due 1.-V

0

VL"-\..LU 1-'U..Ll.\.

V.1.\...J .1.V.1.

error correction

Type-1 selective-repeat hybridARQ

Selective-repeat ARQ

0

E

Bit-error rate

FIGURE 22.14:

Throughput performance of the type-I selective-repeat ARQ.

and the hybrid ARQ scheme provides higher throughput, as shown in Figure 22.14. The type-I hybrid ARQ scheme is capable of maintaining significant high throughput over a wide range of channel error rates if the designed error-correcting capability of the code is sufficiently large. Let P1 be the probability that a received vector will be decoded successfully. This probability depends on the designed error-correcting capability of the code being used. Replacing P with P1 in (22.4), (22.5), (22.6), and (22.11), we obtain the throughput efficiencies for the type-I hybrid selective-repeat ARQ with infinite receiver buffer, the type-I hybrid go-back-N ARQ, the type-I hybrid stop-and-wait ARQ, and the type-I hybrid selective-repeat ARQ with receiver buffer of size N, respectively. The second type (or type II) of hybrid ARQ scheme is based on the concept that the parity-check digits for error correction are sent to the receiver only when they are needed [30, 34-36, 39]. Two linear codes are used in this type of scheme: one is a high-rate (n, k) code Co designed for error detection only; the other is an invertible half-rate (2k, k) code C1 designed for simultaneous error correction and error detection. A code is said to be invertible if when only the parity-check digits of a codeword are known, the corresponding information digits can be uniquely determined by an inversion process. In the next section we present a class of invertible half-rate linear codes, and we show that the inversion can be accomplished by a linear sequential circuit. When a message u of k information digits is ready for transmission, it is encoded into a codeword v = (f (u), u) of n digits based on the error-detecting code Co, where f (u) denotes then - k parity-check digits. The codeword v = (f (u), u) is then transmitted. At the same time the transmitter computes the k parity-check digits, denoted by q(u), based on the message u and the half-rate invertible code C1. Clearly, the 2k-tuple (q (u), u) is a codeword in C1 . The k-bit parity block q (u) is not transmitted but stored in the retransmission buffer of the transmitter for later use.

Seci:ion

v

ARQ Schemes

117/1

Let = (/(un), fili) denote the received vector to v = (f (un), illl). When vis received the receiver the of based on Co. H the syndrome is zero, fili is assumed to be error-free and is the receiver. H the syndrome is not zero, errors are detected in v. The erroneous message fili is then saved in the receiver and a NAK is sent to the transmitter. On receiving this NAK the transmitter encodes the k-bit block q(1m) into a codeword w* = (f[q(11Il)], q(illl)) of n bits based on Co, where f[q(illl)] denotes the 11 - k parity-check digits for q(un). This vector ,1* = (f[q(nu)], q(uu)) is then transmitted the retransmission is a = (/[q (nu)], q(rn)) denote the received vector corresponding to parity vector). Let w* = (f[q(un)], q(uu)). When is received the is computed based on Co. If the is zero, ij (Illl) is assumed to be and the message 11.Il is recovered from q(11.Il) by inversion.Uthe is not zero, q(Illl) and the erroneous message fili (stored in the receiver are used for error correction based on the half-rate code . If the errors in (q (1rn). im) form a correctable error pattern, they are corrected. The decoded message 11.Il is then accepted the receiver. If the errors in (q (Illl). mi) form a detectable but not a correctable error pattern, iiii is discarded and the erroneous block q(iu) is stored in the receiver buffer; also, a NAK is sent to the transmitter. On receiving the second NAK for the codeword v = (j(uu), 11.Il), the transmitter resends v = (f (uu), @). When w = (/(an). fili) is received the syndrome of vis again computed based on Co. H the is zero, fili is assumed to be error-free and is accepted the receiver; the erroneous block q(11.Il) is then discarded. If the syndrome is not zero, mi :md the erroneous block q(11.Il) (stored in the receiver buffer) together are used for error correction based on C1. If the errors in (q(uu), un) are corrected, the decoded message lUl is then the receiver and an ACK is sent to the transmitter; however, if the errors in (q (un), mi) are detectable but not correctable, q(rn) is fili is stored in the receiver buffer, and a NAK is sent to the transmitter. The next retransmission is the vector v* = (f[q(uu)], q(uu)). Therefore, the retransmissions are repetitions of the parity codeword w* = (f[q(uu)], q(Ru)) and the information codeword w = (f(illl), 11.Il). The receiver stores the received message fili and the received block q(Illl) alternately. The retransmissions continue until QTI is recovered. The most important feature of the ARQ is the parity retransmission for error correction based on a half-rate invertible code C1. This parityretransmission strategy can be with any of the three basic types of ARQ. H is particularly effective when it is used in conjunction with the selectiverepeat ARQ. Because of the invertible of C1, the message nu can be reconstructed uniquely from the parity block q(ru1) inversion. Hence, the parity block q (Illl) contains the same amount of information as the message. As a result, the overhead per transmission or retransmission is the number of parity-check digits, n -k, needed for error detection based on the (n, k) code Co, which is required by any ARQ scheme. Therefore, when the channel is quiet or the channel error rate is low, the type-H hybrid ARQ has the same throughput efficiency as its corresponding ARQ scheme. 'When the channel error rate is high, the error correction provided by the half-rate code C1 maintains the high. In Section 22.5 we present a type-H hybrid ARQ that the parity-retransmission strategy with the selective-repeat ARQ with finite receiver buffer. We will show that for

v

1; 78

Chapter 22

Automatic-Repeat-Request Strategies

a BSC, the throughput efficiency of the ideal selective-repeat ARQ with infinite receiver buffer can be achieved by a half-rate invertible code C1 with a very small designed error-correcting capability, say t = 3 to 5. With a larger t, the type-II hybrid selective-repeat ARQ with finite receiver buffer is far superior to the ideal selective-repeat ARQ with infinite receiver buffer. The decoding complexity for a type-II hybrid ARQ is only slightly greater than that of a corresponding type-I hybrid ARQ with the same designed error-correcting capability. The extra circuits needed for a type-II hybrid ARO scheme are an inversion circuit based on C1, which is simply a linear sequential circuit and an error-detection circuit based on Co. The disadvantage of the type-I hybrid ARQ is that the overhead due to the extra parity-check digits for error correction must be included in each transmission or retransmission regardless of the channel error rate. When the channel is quiet, this inclusion represents a waste; however, the type-II hybrid ARQ removes this disadvantage, because is an adaptive scheme. This scheme is particularly attractive for high-speed data communication systems for which round-trip delay is large and error rate is nonstationary, such as satellite communication systems. It is also attractive for wireless communications. Various hybrid ARQ schemes and their analyses can be found in [23, 28-68]. 22.5

A CLASS OF HALF-RATE iNVERTIBllE CODIES

In a type-II hybrid ARQ system, C1 is chosen as a half-rate invertible code. The invertible property facilitates the data recovery process. During a retransmission, if the parity block q(u) is successfully received (no errors are detected), the message M can be reconstructed from q (u) by a simple inversion process rather than by a more complicated decoding process. The inversion process also reduces the frequency of retransmission. For example, if the received message ii contains more than t errors, and the received parity block q(u) is error-free, the decoding process based on (q(u), ii) will not be able to recover the message u. Hence, another retransmission will be required; however, taking the inverse of the error-free parity block q(illl) allows us to be able to recover 11ll and thus avoid another retransmission. Next, we present a class of half-rate invertible block codes and show that inversion can be accomplished by a linear sequential circuit. Let C be an (n, k) cyclic code with n - k :s k. Let g(X) be the generator polynomial of C with the form g(X)

= 1 + g1X + g2X 2 + · · · + g

11 -k-1X

11

+ xn-k_

-k-l

Let v(X)

=

vo

+ v1X + v2X 2 + · · · + V11-1X

11 -

1

be a code polynomial. In systematic form, the k leading high-order coefficients Vn-k, V 11 -k+I, · · · , v11 _1 are identical to k information digits; the n - k low-order coefficients vo, v1, · · · , vn-k-l are parity-check digits. Consider the set of those codewords in C whose 2k - n leading high-order components v2cn-k), v2cn-k)+l, · · · , v11 _1 are zeros. There are 2 11 -k such codewords in C. If the 2k - n high-order zero components are removed from these codewords . we ohtain a set of 7..n-k vedors of length 2(n k). These vectors form a half-rate (2n - 2k, n - k) shortened cyclic code C1

A Class of Half-Rate Invertible Codes

Section 22.5

in'!:ll

Section 5.10). This shortened code has at least the same as C. We showed in Section 5.10 that C1 can be encoded and decoded by the same circuits with a as employed C. code C 1 has the invertible property. Let un(X)

= uo +

iqX + · · · + Un-k-lxn-k-l

be the message to be encoded. i(X), we have

the generator

xn-kll.Il(X)

=

:iI!(X)~(X)

+ lb(X).

where ru(X) and lb(X) are the quotient and the remainder, for u.n(X) is then w(X) = lb(X)

and fui(X) is the of Cl·

+X

11

(22.14) The codeword

-"un(X).

portion. The following theorem proves the invertible

nu,nPrtu

22Jl No two codewords in a half-rate shortened cyclic code C 1 have the same parity-check digits.

'fJHIJEOJP/.JEM

J?moJ: Let

and

x

11

11.Il 1 (X) and M2(X) be -"Qu2(X) by the generator

xn-k @1 (X)

xn-k[n2(X)

two distinct messages. Dividing xn-kll.ll 1 (X) ~(X), respectively, we obtain

= 211 (X)~(X) + (X), = m2(X)~(X) + f0l2(X).

(22.15) (22.16)

Then, the codewords for RU1 (X) and ll.ll2(X) are '¥Y1 (X)

= lbi1 (X)

+

M1 (X).

Suppose that lhi1(X)

= lhi2(X) = lb(X).

Adding (22.15) and (22.16), we obtain [11Il1 (X)

+ 11Il2 (X) ]xn-k = [1J11 (X) + m2 (X) ]~(X).

Because ~(X) and xn-k are relatively prime, uu 1(X) +M2(X) must be divisible by !J;(X): however, this is in1possible, since 1lll1 (X) + uu2 (X) =I- 0 and its degree is less than n - k but the degree of g(X) is 11 - k. Therefore, \hi 1 (X) =I- lb2 (X). (Q)JEJ!]), Because the remainder ibJ(X) resulting from dividing xn-k11Il(X) by g(X) is unique, Theorem 22.1 implies that there is one-to-one correspondence between a message nn(X) and its parity check lb(X). Therefore, knowing lb(X), we can uniquely determine M(X). Next, we show how to recover the message 11Il(X) from the parity lhi(X). First, we multiply both sides of (22.14) by x": X"11Il(X) = a(X)!);(X)Xk

+ \hi(X)Xk.

(22.17)

1180

Chapter 22

Automatic-Repeat-Request Strategies

Rearranging (22.17), we obtain u(X)(X 11

+ 1) + u(X) =

Because g(X) is a factor of X"

ru(X)g(X)Xk

+ lb(X)Xk.

(22.18)

+ 1, we can rewrite (21.18) as

b(X)Xk = [u(X)h(X)

+ ru(X)Xk]g(X) + u(X),

(22.19)

where b(X) = (X' + 1)/g(X). From (21.19) we see that the message u(X) is simply the remainder resulting from dividing lb(X)Xk by the generator polynomial g(X). This operation can be performed by using a division circuit with feedback connection based on g(X). The process of finding the message u(X) from its parity-check digits b(X) is called an inversion process. A faster circuit for inverting lbi(X) can be implemented as follows. Dividing Xk by g(X), we have xk = c(X)g(X) + p(X), (22.20) where the remainder P (X) =Po+P1 X +···+Pn-k-2 X

n-/c-2

+Pn-k-1

xn-k-1

·

(22.21)

Multiplying both sides of (22.20) by b(X) and using the equality of (22.19), we obtain b(X)p(X)

=

[u(X)lrn(X)

+ a(X)Xk + b(X).r(X)]g(X) + u(X).

(22.22)

The preceding expression suggests that we can obtain the message u(X) by multiplying the parity b(X) by p(X) and dividing the product b(X)p(X) by g(X). We can do this with the circuit shown in Figure 22.15.

FIGURE 22.15: Inversion circuit.

Section 22.6

Selective-Repeat ARQ with Finite Receiver Buffer

1i~'u

For consider the (1023, 523) BCH code. This code has a minimum distance of at least 111. We can use this code to correct all combinations of 5 or fewer errors and detect all combinations of 105 or fewer errors. In this case the of an undetected error wiH be small. Shortening this code by 23 we obtain a (1000, 500) invertible code. 22.(5

lVIP'IE-11 ~lVl8mD §IELIEICl!VIE-i?ilEIPIEAl Alf«Q 1)ftfffH IF!NfflE i?ilEICHVIE!?i 18UIFIFIE!f«

retransmission for error correction can be with the ARO with finite receiver buffer described in Section 22.2. vve the case for which the size of the receiver buffer is N (the number of vectors transmitted in one round-trip The range of sequence numbers is still 3N. Let w1 be a k-bit message to be transmitted. There are three codewords associated with this message: 1L The codeword w = (f(un). un) based on an (n, k) error-detecting code Co.

:t The codeword 'W = ], The codeword,-,,* the code Co.

based on a half-rate invertible (2k, k) code C1. (f[q(un)], q(un)) based on the k-bit parity block q(un) and

(q(un), un)

=

we call v = (f (un), llli) the information codeword of un, and the parity codeword of un. We win use ;;; = (/(un), ill!) and = Cf[q(un)]. q(w1)) to denote the received vectors to v and w*, fo transmission or retransmission the information codeword wand the parity codeword w* of the message uu have the same sequence number. When an information codeword w is ready for it is numbered and stored in the queue of the transmitter. After its transmission, w and its codeword w* are saved in the retransmission buffer until w is "''°"'F,V"'· When an ACK is received after a round-trip delay, both v and v* are released. \Nhen a NAK (or no is received after a v* is sent to the receiver for error correction (if necessary). After another round-trip delay, if an ACK is received, both w and w* are released; otherwise, wis retransmitted. The transmitter resends w and w* alternately until w is acknowledged, as shown in Figure 22.16. For

v*

(f[q(un)], q(uu))

When the transmitter is sending a codeword, information, or parity it also computes the forward index fr of the codeword in the retransmission buffer that is to become a time-out vector. Based on this forward index fr. the transmitter decides whether the next information codeword in the input queue is to be transmitted or retransmission is to be initiated. The decision rule is as follows: acknowledged and fr < N ( or if there is no current time-out the first information codeword in the input queue is to be transmitted. ;2o H the current time-out codeword, say w1 , is either negatively acknowledged or unacknowledged and fr < N, a retransmission for w.i is initiated. The

Jt H the current time--out codeword is

<

::!!

II

C

:-J

.... d>

G\

eo i'J

:::0 m

n o.,

N

::,

!'-J

~·

Retransmission (parity)

~

?:'

""Ci

rt>

Retransmission (parity) Retransmission (information)

>-l

-;

N

'--<: 'v

N

~

Round-trip delay

"'""

J>

[/J

~,

(D

C

(D ~

v3 I v-1

~-

(D

...,' (D

'"O

Transmission

(D

~ :::r' '--<:

u ...,

/

~

I

/

/

I

I

I

I

I

I

/

I

/

/

/

/

I

/

/

/

I

/

/

/

11 ti ti ti ti ff I

/

I

/

/

I

/

/

/

I

/

I

/

/

I

/

/

/

/

/

/

/

I

/

/

/

/

/

I

/

/

I

/

/

I

I

/

I

/

/

/

/

/

/

/

/

I

/

/

/

/

/

I

I

/

I

I

/

I

/

I

I

/

I

/

I

/

I

/

/

I

/

/

I

I

I

I

/

I

/

/

/

:::r'

..., (D () (D

:::;·

..., (D

u

i::

~

'0"'

-

"t=::l"' (D

/

/

/

I

/

?

I

/

I

v21 I v1 I v2I v3 I v-1 Iv, I vr, I v7 Iv:~ I v9 Iv; lv1olv11 j v6 jv12lv11lv1-1lv1,lv1r,jv17j vr, jvulv1-1lv1,lv1r,lv17lv18lv19lv2olv21 I v1 I ir2

3 0.,

r>

i","

;b (t)

"O (t) 0.,

';"'

::x,

(t)

.0 C

(I)

~-

>-~

-

/

(JIIJl1Jli7l1Jl1J/i7l1JIIJIIJIIJIIJl(Jli71(51~:>i71i71i71i71(J';(J'//J>CJ'l6"9/(J'/IJ/i:JI / Z'/\/ 1 '\'/~/Z,/Z,/~/Z,/"?;//"?;/Z,/"?;//"?;//~/Z,/~/~/~//~/~/Z,/Z,//'Z°'/"?;/'\;/"?;/Z,/"?;/ /

0.: 0 :§.

/

/ I I / / / / / / / / / / I / I / / / I / / / I / / / / .L/,L/,L/.L/,L/,L/.L/.L/~_/,L/L/.L/.L/,L/.L/,L/.L/,L/,L/.L/.L/,L/L/.L/.L/,L/.L/,L/

I

Receiver

0

V\

r.·

Qj

r.·

Error

Error

Information codeword v3 is recovered. Receiver outputs v3 • v-1, and v, .

Information codeword v6 is recovered. Receiver outputs v6 to v 11 . Information codewords v 13 to v 17 are rejected by the receiver owing to buffer overflow.

Parity codeword v6 fails to recover information code vector v6 . Erroneous information codeword v6 is replaced by the received parity codeword v6.

(I)

I.Q

ii," V,

Section 22.5

ARQ with Finite Receiver Buffer

nS\3

retransmission for is the if vi was previously transmitted, and is a of ·y i if was transmitted. U the current NAK'ed (or unACK'ed) time-out codeword is the earliest codeword in the retransmission buffer that has not been acknowledged Ur = 0), all the information codewords in the retransmission buffer with forward indices fy :::: N are moved back to queue for retransmission at a later time the receiver owing to the receiver buffer C)UPrlc>CYm

:t H fy 2: N, the first information codeword in the

queue is the next to be

transmitted. The transmission and retransmission of the hybrid selectiverepeat ARQ with receiver buffer of size N are detailed by the flowchart in Figure 22.17. 22Uii.2

~ie«:reo~rerr's; OLQl®rrsrii:om1

o~ ·,d1c~ ~!mniucaill St;;rii:e

fo the normal state the receiver receives information and the receiver buffer is empty. vVhen an information codeword is received the receiver computes its syndrome based on Co and its forward index f R with respect to the last accepted and delivered information codevvord. H the is zero and f R = 1, the received information codeword is and delivered to the user. If f R > N, the received vector is regarded as an information codeword that was previously accepted and delivered; it is then and an ACK is sent to the transmitter. Heither the received information codeword is detected in error, or 1 < f R :::: N (i.e., its sequence number is out of the receiver enters the blocked state. The receiver's in the normal state is detailed the flowchart in Figure 22.18. 22J».3l

iM;te«:ieivrer':£ O(Qliernii:l~llil 0111 ·itrrue

H the receiver enters the blocked to the detection of errors in a received information codevvord v = v1ith f R = 1, the erroneous message llil is then stored at the first location of the receiver buffer and a NAK is sent to the transmitter. The receiver then checks the received information codewords and stores them in the receiver buffer at the proper locations according to their sequence numbers until the buffer is full; an appropriate acknowledgment is sent to the transmitter for each received vector. H the receiver enters the blocked state with JR > 1, then f R - 1 information codewords between the last delivered information codeword and the current received information codeword are lost. The receiver then reserves the first fl?. - 1 locations of the receiver buffer for the lost vectors and saves the current received vector the message part) at the CfR)th location of the buffer. The received information codewords are stored in the remaining locations of the receiver buffer. When the first retransmitted say ,?1 = (f[q(llll1)], q(llllj)), is received it is used to recover the earliest erroneously received message (or the earliest lost message) llllJ. H the syndrome of is zero, is recovered by taking the inversion of q(llllJ ). Uthe syndrome ofw'1is not zero, q(uui) and lillJ (stored in the buffer) together are used for error correction based on the half-rate-error-correcting code C1 . If the

1184

Chapter 22

Automatic-Repeat-Request Strategies

No

Yes

.---------<:. current time-out >----, vector? No

Yes

ACK

ACKorNAK?

Transmit the first information codeword in the input queue and save it in the retransmission buffer

Yes

No current time-out vector an information vector?

Retransmit the parity-code vector of the current time-out vector

No

:::]_

__

time-out information codeword

Is the current time-out vector the earliest information codeword that has not been positive! acknowledged?

Yes Move all the information code vectors in the retransmission buffer that have FWX fr 2: N back to the input queue

FIGURE 22.17: Transmission and retransmission procedure of the type-II selectiverepeat hybrid with receiver buffer of size N. errors are correctable, ui J is recovered. When u 1 is recovered, the receiver releases and the subsequent error-free (or zero-syndrome) messages in consecutive order, say 11.llJ, l!llj+ 1, · · · , llllJ+L with O :::: L < N. Because the retransmission is selective, and since llllJ+l to UJ+L are successfully recovered, the vectors following vj are new information codewords VJ+N, vi+N+l, · · · , vi+N+L-1· When these new information 1.1. J

Hybrid Selective-Repeat ARQ with Finite Receiver Buffer

Section 22.6

--

No

v 1

V

-~~1 [

~:cs __ ,

1 Send ACK(i) to the transmitter and store vj in the receiver buffer

l

l

Yes ~ r r o r ~ No

lj',lc,_.~~yes

V SendACK(i) to the transmitter

Yes

1·rns

l _____

Send A~K(j) to the transmitter and deliver vj to the user

Send NAK(i) to the transmitter and store v1 in the receiver buffer

c_En-te-rt,he 1-cl-,ed---)

FIGURE 22.18: Normal-state operation of the receiver.

codewords arrive their are checked and are temporarily stored in the receiver buffer until they are to be released to the user or to be corrected. If q(fillj) fails to recover uni. the receiver discards iin1 in the receiver buffer; also, a NAK is sent to the transmitter. Because error-free messages cannot be released, there are no places in the receiver buffer to store the new received information codewords VJ+N to VJ+N+L-1 · fa this event, the buffer overflows, and these new information codewords are rejected. On receiving the second NAK for Yj = (f(Mj). u 1). the transmitter resends w.i to the receiver. On receiving = ( / (nu.i). illlj). the receiver again attempts to recover nu1. If the syndrome of v1 is zero, illlJ is assumed to be error-free and is released. If the syndrome of is not zero. im1 and q(nni) ( stored in the receiver buffer) together are used for error correction based on Cl· If the receiver again fails to recover nni, then q(RTIJ) is discarded and imi is stored in the receiver buffer. Vector w.i ( and any other erroneously received information codewords) continue to be retransmitted until nu.i is successfully recovered. The retransmissions are alternate repetitions of the parity vector and the information vector Yj. and the receiver stores illlj and q(nuj) alternately. In the blocked state, when an information codeword is received its forward index lf is computed. If lf < N, the vector is stored in the receiver buffer. H no errors are detected, an ACK is set to the transmitter; otherwise, a NAK is sent to the

1186

Automatic-Repeat-Request Strategies

Chapter 22

transmitter. If l1 ::': N, the receiver computes its backward index lb. If lb< 2N, the current received vector is regarded as an information codeword that was previously accepted and delivered. It is then ignored, and an ACK is sent to the transmitter. If lb ::': 2N, the received vector is a new information codeword; however, the buffer is full and overflows. In this event the received vector is rejected and a NAK is sent to the transmitter. When a retransmitted NAK'ed vector arrives, the receiver checks whether this received vector and its corresponding vector stored in the receiver buffer form a message and parity pair. If they do form such a pair, the receiver then attempts to recover the original message either by inversion or by decoding process. If the newly received vector and its corresponding vector in the buffer do not form a message and parity pair, the receiver checks whether both are information vectors or parity vectors. If they are both information vectors, the receiver stores the one that is error-free; if both are erroneous, the receiver replaces the old one with the newly received one. If they are both parity vectors, the receiver replaces the old one in the buffer with the newly received parity vector (or its inversion). If the received vector corresponds to a lost vector, it is stored at a reserved location in the receiver buffer. The receiver's operation in the blocked state is detailed in the flowchart in Figure 22.19.

22.6.4 Throughput: Efficiency To analyze the throughput efficiency of the hybrid ARQ described is very difficult; however, if we assume that the channel is a BSC and the return channel is noiseless, we can obtain a lower bound on the throughput. Again, let P be the probability that a codeword, information or parity, will be received successfully (i.e., the syndrome based on Co is zero). Suppose that C1 is capable of correcting any combination oft or fewer errors and simultaneously detecting any combination d(d > t) or fewer errors. Let Q1 be the conditional probability that a message lll!J will be recovered from the first received parity block q(l!Hj) either by inversion or by decoding based on C1 , given that errors are detected in the received information codeword Vj = (j(anj), iiJ), Let Q2 be the conditional probability that MJ will be successfully recovered from the second retransmission of YJ = (f (lll!J), lll!J ). given that errors are detected in the first received parity codeword vj = (/[q(MJ)], q(lll!J)) and the first received information codeword VJ = (f (uni), ii.i ), and that the first received parity block q(u i) fails to recover ui but detects the presence of errors in (q (111 i), iii). Then, it is possible to show that [39] Q1

=

p

+ 1~

p

(;ta (2t)p'(l -

-2(1 - p)'1[(1 -

0? > -

P

+ .

_l

Pl+ .

p)2k-i

t G)i

f 1-1

+

(1-

p)2n

(1 - p/-i]),

(22.23)

t

1=1

l

··1 .

~ ) ,6.;S,_;(l - ,6.n - S,_;) (1-f')(l-{h) !"-".. .. " .

(22.24)

Section 22.6

Type-II

ARQ with Finite Receiv2r Buffer

n21

No

I

.i

y~~ r~--T~

y'7

No/U~wvz )AT

V

V

Send NAK(i) to the transmitter

Send ACK(i) to the transmitter

I

v_1

j

c<~--'-b-~---------d

No

Any previous transmission or retransmission Yi in the buffer corresponding toX/ Yes

l

transmitter

<----

Errors in X; and Y; form a correctable error pattern1

No

I

V

Send NAK(i) to the transmitter and replace Y; by X;in the buffer

Recover uni either by inversion or by decoding and send ACK(i) to the transmitter

\! ---().--

Yes

,I

V Output un; and all the subsequent consecutive error-free messages

Store uni

t

>--------~~

~ e~~~~~~

.. , . ( Return ,o the nmmal s t a t e ) ~

FIGURE 22.19: Blocked state of the receiver, where information codeword VJ or the parity codeword vj.

No

and Yj represent either the

'1188

Chapter 22

,L'<,utomatic-Repeat-Request Strategies

where

j

Sj

=

I>'l/. i=l

We define 80 81

6

1 - (1 - P)(l - Q1),

6

1 - (l - P)(l - Q1)(l - Q2),

Then, the throughput of the type-II hybrid selective-repeat ARQ with receiver buffer of size N is lower bounded as follows: 80

1.00

(22.25)

r--=~=---r---,;;,.,,_;;;;,,_=11.-=-,----r---,-------, ~~

'A

',,\

\A

0.80

N= 128 n = 524

~A

~A

4 ~~5

;:i

0.60

'----\\\\\\

0..

..c:: bi) ::; 0

~ O®~

""'l ,

~ 2 ~\~\ \

0·?o.___ 10_.___ 8 --10--'--7 --1-'o---r-,_ _ 5 _ _1_,_o---"1--1-0=-= 3 """------"10- 2 Bil-error rate

FIGURE 22.20: Throughput of various ARQ schemes with N = 128 and n = 524: (1) ideal selective-repeat ARQ with infinite receiver buffer; (2) and (3) selective-repeat ARQ with receiver buffer of size N and 2N; ( 4), (5), and (6) type-II selective-repeat hybrid ARQ with receiver buffer of size N and error-correction parameters t = 3, 5, and 10, respectively.

Section 22.6

Type-II Hybrid Selective-Repeat ARQ with Finite Receiv,2r Buffer

11~9

where 8

= _!!3___(1 - e eN- 1) 1 - 81 2 1 .

81

=

82

=2-

0

PQ1ef:- 2 + P(l - Q1)Q2et- 2. PQ1er:- 2 - e1e2et- 2.

For various n, k, t, and N, the lower bound on rate is plotted in Figures 22.20 to 22.22. We see that the of the ideal ARQ with infinite receiver buffer can be achieved the type-H ARQ scheme described with a small errorcapability t (t :s 5). For t > 5, ideal selective-repeat ARQ can be obtained. 22.5.!5

11eiaat(l)amty

Uthe error-detecting capability d of the half-rate code C1 is type-H hybrid ARQ scheme the same

large, the scheme.

l 0.80 N 11

= 1024 = 1024

\ Go-back-JV-\ \

0.20

Bit-error rate

FIGURE 22.21: Throughput of various ARQ schemes with N = 1024 and n ideal selective-repeat ARQ with infinite receiver buffer; (2) and ARQ with receiver buffer of size N and 21V; (4), (5), and (6) type-H hybrid ARQ with receiver buffer of size N and error-correction 5, and 10, respectively.

= 1024.: (1)

1190

Chapter 22

Automatic-Repeat-Request Strategies

1.00~"1

\ 0.80

N= 512 n = 2024 "

0.60

0..

..c::

bIJ

::l 0

~

0.40

0.20

10-0 Bit-error rate

FIGURE 22.22: Throughput of various ARQ schemes with N = 512 and n = 2024: (1) ideal selective-repeat ARQ with infinite receiver buffer; (2) and (3) selective-repeat ARQ with receiver buffer of size N and 2N; ( 4), (5), and (6) type-H selective-repeat hybrid ARQ with receiver buffer of size N and error-correction parameters t = 3, 5, and 10, respectively.

Let Pe be the probability that Co will fail to detect the presence of errors. Let 2k

(J"

= I: (

2ik ) / (l _

v>2"-i,

(22.26)

l>d

which is the probability that the number of errors in (q(llll), mi) wiH exceed the designed error-detecting capability d of C1. Then, the probability that the receiver will commit a decoding error is bounded as follows [39]: Pe Pe

+ Pc

< P(E) <

-

- Pe

Pe

+ Pc

+

u

Pe

+ Pc '

(22.27)

where Pc = (1 p )11. If we choose d sufficiently large, we can make a of the same order as Pe or even smaller than Pe. As a result, the probability of a decoding error, P(E), is the same order as the error probability of a pure ARQ. 22.7

HVIBRiD ARQ SYSTEMS USING CONVOU..mONAl CODIES

So far we have considered only hybrid ARQ systems that employ block codes for both error detection and correction. Hybrid ARQ systems using convolutional codes

Section 22.7

Convolutional Codes

·i u91

ARQ for error correction can also be devised. One such is a in [4ll Consider a rate-1/2 L m) convolution.al code C 1 of memory order m. Let g
and

i

2

l (X)

= 11J1(X)gC 2 l

based on the convolutional code C 1, each 11 + m bits The + m )-bit sequence v(X) obtained interleaving vClJ (X) and is a code sequence in C 1. The sequence v( 1l (X) is then and the sequence (X) is stored in the transmitter buffer for possible retransmission at a later time. Let 1;;(1\X) denote the received sequence corresponding to Vi/hen is received it is divided by the generator 1!i(ll Let ,rnm and be the and remainder of the H t 0 J (X) = 0, llll(lJ (X) is then checked based on the error-detecting code Co. If the s/ll(X) of nn< 1l(X) is zero, illlC 1l(X) is assumed to be error-free and identical to the transmitted codeword uu(X). Then. the information sequence IT(X) is recovered from 1uPl (X) and the receiver. H 1bJOl =I- 0, or §(ll (X) =I- 0. errors are detected in (X). is saved in the receiver buffer for at a later time. At the same the receiver requests a retransm1ss10n a NAK to the transmitter. On receiving this NAK the transmitter sends the sequence This forms the first retransmission for IT(X). Let ~;;C 2\X) be the received sequence to (X). When w< 2\X) 2 is received it is divided the i:g< J (X). Let llll( 2l ()() and 2 ll:D( J ()() be the quotient and ren1ainder of the H !bJ( 2 l (X) = 0, 2 the syndrome rP\X) of illl< l(X) is based on Co. If (X) = 0, then illl( 2l (X) is assumed to be error-free and identical to :rn(X). fo this case. the receiver recovers IT(X) from illl( 2\X) and discards vrll (X). H n:P\X) =I- O. or §(2) (X) =I- 0, then w< 2l (X) together with vcx) is discarded, and w< 2J(X) is stored in the receiver buffer for reprocessing at a later time. The receiver requests a second retransmission for Il(X). The second retransmission is vCll (X). The retransmissions for IT(X) are alternate of the sequences w< 2l (X) and w(ll (X). Retransmission and decoding continue until IT(X) is finally recovered. The throughput efficiency of the described hybrid ARQ system has been analyzed in [41] for various receiver buffer sizes.

1192

Chapter 22

Automatic-Repeat-Request Strategies

A type-II hybrid ARQ system using a single rate-1/2 code for parity retransmission and error correction is simple and easy to implement; however. its throughput is reduced by 50% whenever the first retransmission is requested. This large percentage drop of throughput can be reduced by using a family of rate-compatible punctured convolutional (RCPC) codes [46]. This family of RCPC codes is obtained by puncturing a single rate-1/n (n, 1. m) convolutional code C with rates ranging from T /(T + 1) to l/n, where Tis the punctured period. All these punctured codes can be decoded with the decoder for the mother code C. A type-H hybrid ARQ system using a family of RCPC codes has been proposed by Hagenauer [46]. The principle of this system is to transmit additional code bits corresponding to the next-lower-rate RCPC code in the family to enhance the error-correcting capability of the code at the receiving end. These additional code bits are actually certain code bits of the mother code that were punctured in the earlier transmission or retransmissions. When these additional code bits are received they, together with the undecoded code bits stored in the receiver buffer. form a lower-rate RCPC code and are then decoded. After decoding. error checking is performed based on an error-detecting block code Co. If no error is detected. the decoded sequence is assumed to be error-free; otherwise, a retransmission is requested. Retransmission continues until the mother code is reached. completing a full retransmission cycle. U the decoding is still not successful at the end of a full retransmission cycle, then the next retransmission cycle for the same information sequence begins. starting with the highest-rate RCPC code; or the decoded sequence is simply accepted at the end of decoding the mother code C. This, of course, degrades the reliability. Other hybrid ARQ systems using convolutional codes can be found in [23, 36, 38, 42, 47, and 49]. 22.8

A CONCATIENATIED CODIED MODULATION HYIBIRID ARQ SYSTIEM

The concatenated coded modulation error control scheme presented in Section 19.4 can be incorporated in an ARQ system to achieve both high reliability and high throughput efficiency. Such a hybrid ARQ system [51] is presented in this section. This system demonstrates the integration of concatenated coding, coded modulation, and ARQ. The FEC subsystem is achieved by concatenating an interleaved Reed-Solomon outer code and a bandwidth-efficient BCM inner code. Parity retransmission is used to maintain throughput efficiency when the communication channel is noisy. 22.8.1

Codles lllsedl in ithe System

Three codes are used in the system. The inner code C1 for the concatenated coded modulation FEC subsystem is a bandwidth-efficient BCM code of length 111 and dimension k1 with symbols from a signal set S. say, an 8-PSK signal set. The outer code C2 is a shortened (n2, k2) RS code over GF(2m) with generator polynomial g2(X). This outer code has a minimum distance of d2 = 112 - k2 + 1 and is used to correct t2 or fewer symbol errors and simultaneously detect p2 (P2 2: t2) or fewer symbol errors, where t2 + P2 + 1 s d2. The dimension k1 of the BCM inner code C1 and the length 112 of the RS outer code C2 are chosen to satisfythe following

Section 22.8

A Concatenated Coded IVlodulation Hybrid ARQ

conditions: lq = "Am,

with "A ::: 1 and q ::: 2. The third code, is designed for parity retransm1ss1on. H is a rate-1/2 (2(n2 - k2), 112 - k2) shortened RS code obtained from the outer RS code deleting the first 2k2 - 112 information from each codeword in Both C2 and C,. are generated by .\\;2 (X) and can be encoded and decoded by the same circuits. C,. is an invertible code. Let llll be an information sequence of 112 - k2 Let R,. (11Il) denoted the sequence of 112 h parity symbols formed based on illl and C,.. Then, (llll, (1!Il)) is a codeword in C,., and there is a one-to-one correspondence between illl and R,.(llll). R,.(llll), we can recover llll by an inversion In fact, the invertible property implies that (R,. (llll). un) is also a codeword in C,. [see Problem 22.9]. Let v(X) = vo + v1 + ... + v112 _1X 112 - 1 be a codeword in C2. Because 112 = q · (112 - k2), we can divide v(X) into q equal subsections, v1 (X), v2(X) .... Vq with each consisting of 112 - k2 such that (22.28)

where (22.29)

For 1 :s: i :s: q, let [w1 (X)] be the parity polynomial obtained by encoding the ith section v; (X) based on C,.. R,.[w; (X)] is simply the remainder obtained from xn,-k2v, (X) by g2 (X). Therefore. [v,(X)]

+ w;(X)X 112 - "2

is a codeword in C,.. We form the following polynomial:

Then, R[w(X)] is also a codeword in C2 [see Problem 22.10]. Therefore, v(X) and R[w(X)] form a codeword pair in C2 , and one can be obtained from the other by a unique inversion operation. We call R[ v(X)] the parity codeword ( or polynomial) of w(X). We will use this property in the hybrid ARQ system to be described. 2:Z.8.2

lraie (©6111:zr\tieu1zr1tiedl (©\Dledl M©ci!.!li©Jto©u1 IF!EC '.S!.!lbx1;y:1,tiem amid ~ietr,©111:1,mi5:1,irn11

The FEC subsystem is a concatenation of the RS outer code C2 and the BCM inner code C1 . The outer code is interleaved by degree "A, as shown in Figure 22.23, where "A outer codewords, lbJ 1 • ibJ2, ... , ibJA, are arranged as a "Ax 112 array over GF(2 111 ). vVe represeot each symbol in GF(2111 ) with a binary m-tuple in column form. Then, the "A x 11 2 array over GF(2111 ) becomes a "Arn x 112 binary array, with each column consisting of lq = "Am binary digits. We encode each column of this binary array into a code sequence of 111 signals based on the BCM inner code C1. The result

1194

Chapter 22

Automatic-Repeat-Request Strategies 1, - < - - - - - - - - - - 1 1 2 - - - - - - - - - - - > - I 1

I<

TT m-bit byte

...

J_

A outer codewords

FIGURE 22.23:

A data array.

is an n1 x n2 signal array over the signal set S, denoted by I[b1, b2, ... , bJ,.]. This array, called a data array, is transmitted column by column. At the receiving end, the received signal sequence is rearranged into an array i[b 1 , b2, ... , li'DJ,.]. First, each column of i[b1, b2, ... , IJJ,.] is decoded based on C1 using a soft-decision decoding algorithm. After n2 inner decoding, we obtain an array of )cm x n2 decoded bits. Each column of this decoded array is regarded as a column of )c symbols from GF(2111 ). Then, each row of this array is decoded into a codeword in the RS outer code C2. If the decoding of each row is successful, the decoded information symbols are delivered to the user. If any row of a received data array i[b1, b2, ... , bA] is not decoded successfully, the data array obtained from i[b1, b2, ... , bA] after the inner code decoding is saved in a buffer and a retransmission is requested. The retransmission is an array of parity symbols, denoted by P(R(b1), R(b2), ... , R(l-»J,.)), that is formed based on the original set of codewords, b1, b2, ... , bJ,., and the half-rate invertible code C,.. First, each codeword b; is divided into q sections bi.1, b;,2, ... , b;,q· The parity word R(lb;) = (R,.[bu], R,.[b;.2], ... , R,.[b;,q])

is formed based on b;,1, b;,2, ... , b;,q and code C,.. An array is constructed with R(b1), R(b2), ... , R(bJ,.) as rows. Each column of this array is encoded into

Section 22.8

a code sequence of

A Concatenated Coded Modulzrtion

m

111

ARQ

n 1 >( 112 signal array set S, which 1s transmitted column

P(R(ibi1), R(ibi2), ... , R(ibi;J) over the

column. Let denote the received P is decoded just like a data array If the U'-''-'V'""rn'"" based on R(ibi1), R(lbi2), ... , R(lbi.d to obtain ibl1, delivered to the user or saved in the receiver buffer until Hthe decoderfailstodecodeeachrovvof P(R(ibJ 1), R(lbJ2), ... , then the erroneous words R(ibii), , ... , R(lbi") and the erroneous data words ibi1, lbi2, ... , lbJ" stored in the receiver buffer are used for errnr correction based on the half-rate code C,.. For 1 ::: i ::: >,. and 1 ::: .f < q, the (lbi;.J. [lbJ;.J]) is formed and decoded based on . Let be the decoded estimate of llD;.J. After >,. >< q such decodings, the receiver contains the estimated data array:

l

llD~

]

[

llD*1.2

:bi*l.q

fui·' · "'2,11

!bi*2.2

11:D2.q *

lbi~.1

ltDt.2

n~* "A.,lj

l

J

Then, the receiver checks whether each ltD)', for 1 ::: i ::: A, is a codevvord in Note that this time C2 is used for error detection. H all ibif, are codewords in C2, then decoding is successful. The decoded words are either delivered to the user or saved for later delivery. If any !bi;' is not a codeword in C2, the receiver discards the stored erroneous data array i and saves the erroneous parity array P for later use, At the same time, the receiver a second retransmission, The second retransmission is the data array I (lbi1, [(])2, ... , !bi,.) itself. The for the second retransmission is the same as the first retransmission. Retransmissions and continue until the data array is successfully recovered. Retransmissions are alternate of the array P and the data array /, as shown in 22.24. Retransmissions can be carried out in any of the three modes: the and-wait, the go-back-N, and the H ARO with finite buffer is used, the retransmission protocol described in the section can be used. A special case for this hybrid ARO error control scheme is that n 1 = lq = 111. In this case no inner code is used; the outer code C2 is used for both error correction and detection. The half-rate invertible code is used for error correction 22.~.3

A '.'ii(Qlie«.:hruic '.'iiysii:iem

Next, a specific concatenated coded modulation ARO in [51] is used for illustration. Set m = 8 and >,. = 2. For this the outer code C2 is chosen as the shortened (224, 192) RS code over GF(2 8) obtained by deleting 31 information symbols from the NASA standard (255,223) RS code over GF(2 8 ). This code has 32 parity-check and is of correcting up to 16 symbol errors, The inner code C1 for the concatenation is chosen as the 8-JPSK BCM code of length 8, dimension 16 in Example 19.1. Using this inner

1196

Chapter 22

Auton1atic-Repeat-Request Strategies

FIGURE 22.24:

Data array

Original transmission

Parity array

First retransmission

Data array

Second retransmission

Parity array

Third retransmission

Alternating data-parity retransmission strategy.

16 information bits are coded into a sequence of eight 8-PSK signals (n1 = 8). This code has a spectral efficiency of 2 bits/signal (or effective rate = 1) and a minimum squared Euclidean distance of 4. H has a 4-state 8-section trellis and can be decoded with the soft-decision Viterbi decoding algorithm. In concatenation the outer code is interleaved by a degree of J.. = 2 before inner code encoding. The overall rate of

-

10-11

10-12

·~

10-u I-<

... 0

10-14

I-<
01)

ca

'o

10-10

0

u


"O

"' E

10-

16

4-<

0

:.0Cl

..D 0 I-<

i:,.,

10--21)" 10-21

5

5.5

6

6.5

7

7.5

8

8.5

EJN11 (dB)

FIGURE 22.25:

Upper bound on the probability of a decoding error.

Problems

'H917

0.9 0.8

so.

-"' bJ)

"'0

0.6

'"'

~

E-

0.5

0.3

6

7

8

9

10

11

EJN11 (dB)

FIGURE 22.26: Lower bound on the

)( ::-::: 1 )( the concatenated code is R = = 0.857 of 1.714 The half-rate invertible code retransmission is the shortened (64. RS code over the outer code C,. is capable of correcting up to 16 errors over a span of 64 and hence is very powerful. even in a very a transmitted data array should be recovered with at most one retransmission. The reliability and throughput of this have been analyzed in [51] and are shown in Figures 22.25 and 22.26. The performs extremely well for SNR Es/No than 7 dB channel biterror probability p :S 10- 2 ). For SNR E.1/No = 8 error-free communication is practically achieved and the efficiency is to the rate, 0.857. Other hybrid ARQ schemes coded modulation for error control can be found in [51, 53, and 64-66].

i"~Om.lEMS 22U_ In (22.5) we savv that the throughput of the go-back-JV ARQ depends on the channel block error rate P = 1 - (1 - p)", where 11 is the code block length, and p is the channel (BSC) transition probability. Let r be the data rate in bits per second. Let T be the round-trip delay time in seconds. Then, N = r · T / n. Suppose that p and k/11 are fixed. Determine the block length no that maximizes the throughput rJGBN· The block length 110 is called the optimal block length.

1198

222

Chapter 22

Automatic-Repeat-Request Strategies

Optimal block lengths for the three basic ARQ schemes were investigated by Morris [7]. Consider a continuous ARQ scheme that operates as follows. When the transmitter receives a NAK for a particular vector -v under the condition that the N - 1 vectors preceding v have been positively acknowledged, the transmitter stops sending new vectors and simply repeats the vector v continuously until an ACK for v is received. After receiving an ACK, the transmitter renews transmission of new vectors. At the receiver, when a received vector is detected in error under the condition that all the vectors preceding have been successfully received, the receiver rejects and all the N - 1 subsequent received vectors until the first repetition of v arrives. Then, the receiver checks the syndrome of and the following repetitions of v. An ACK is sent to the transmitter as soon as one repetition of v has been successfully received. ai. Derive the throughput of this scheme. !Ji. Compare the throughput of this scheme and the throughput of the conventional go-back-N ARQ. Suppose that we use the retransmission strategy described in Problem 22.2 but with a buffer of size N provided at the receiver. When a received vector is detected in error, the receiver stores the subsequent successfully received vectors. When a repetition of v is successfully received, the receiver releases and the error-free vectors held in the receiver buffer in consecutive order until the next erroneous vector is encountered. lll. Derive the throughput of this ARQ scheme. !Ji. Compare its throughput with that of the conventional go-back-N ARQ. We may shorten the (31, 16) BCH code to obtain a (30, 15) invertible code. Devise an inversion circuit for this code. In a stop-and-wait ARQ system, suppose that the forward channel is a BSC with transition probability p 1, and the feedback channel is a BSC with transition probability P2· Derive the throughput efficiency of this system. Repeat Problem 22.5 for the go-back-N ARQ system. Repeat Problem 22.5 for the ideal selective-repeat ARQ system. Design a type-II hybrid ARQ system using a rate-1/3 convolutional code similar to the system presented in Section 22.7. Let C be a half-rate invertible (2k, k) systematic linear block code. Let llil be an information sequence of k bits and f (lill) be its corresponding parity sequence. Prove that both (fill, f (1lll)) and (f (u), 1lll) are codewords in C. Consider the RS outer code C2 defined in Section 22.7. Prove that the parity word R[v(X)) given by (22.30) is also a codeword in C. Design a type-II hybrid ARQ system in which a RS code C2 over GF(2 111 ) is used for forward error correction, and a half-rate RS code C,. obtained by shortening C 2 is used for parity retransmission. This is simply the hybrid system presented in Section 22.8 without an inner code. The inner code C 1 of the hybrid system presented in Section 22.8 can be chosen as a binary (11, k) code designed for simultaneous error correction and detection. Design a concatenated hybrid ARQ system with C1 as the inner code.

v

v

v

v

22.3

v

v

22.4!

22.5

22.6 22.7 22.8 22.9

22.1@ 22.11

2'.:U2

BIBUOGRAPHY

JL R. J. Benice and A.H. Frey, Jr., "An Analysis of Retransmission Systems," IEEE Trans. Commun. Technol., COM-12: 135-45, December 1964.

Bibliography

n!Sl!Sl

2;, H. 0. Burton and D. D. "Errors and Error Control," Proc. 60(11): 1293-1310, November 1972.

:.t AR. K.

"Improving Automatic Repeat-Request(ARQ) Performance on Satellite Channels Under High Error Rate Conditions," IEEE Trans. Com-mun., COM-23: 436-39, 1975 .

.6,l,

J. A Lockitt, AG. Gatfield, and R.R. Dobyns, "A Selective-Repeat ARQ System," Proc. Third Int. Conf Digital Satellite Commun., Kyoto, ]\Tovember 1975.

!§,

J. J. Metzner," A of an Efficient Retransmission Strategy for Data NTC Rec., pp. 3B:1-l-3B:1-5, 1977.

«£, J.M. Morris, "On Another Go-Back-N ARQ Technique for High Error Rate Conditions," IEEE Trans. Commun., COM-26: 187-89, January 1978. 7,

Blocklengths for ARQ Error Control Schemes", IEEE Trans. Commun., Com-27: 488-93, February 1979.

§;,

D. Towsley, "The Stutter Go-Back-N ARQ Protocol," IEEE Trans. Commun., COM-27: 869-75, June 1979.

9\ P. S. Yu and S. Lin, "An Efficient Error Control Scheme for Satellite Communications," IEEE Trans. Commun., COM-28: 395-401, March 1980.

]Jt

, "An Efficient Selective ARQ Scheme for Satellite Channels," ICC'80, Conf Rec., pp. 4.6.1-4.6.4, Seattle, Wash., June 1980.

JL]_,

, "An Efficient Selective-Repeat ARQ Scheme for Satellite Channels and Hs Throughput Analysis," IEEE Trans. Commun., COM-29: 353-63, March 1981.

]_:Z,

M. J. Miller and S. "The Analysis of Some Selective-Repeat ARQ Schemes with Finite Receiver Buffer," IEEE Trans. Commun., COM-29: 1307-15, September 1981.

12t E. J. Weldon, "An Improved Selective-Repeat ARQ Strategy," IEEE Trans. Commun., COM-30: 480-86, March 1982.

:Mo S. D. J. Costello, Jr., and M. J. Miller, "Automatic-Repeat-Request Error Control Schemes,'' IEEE Commun. Magazine, 22(12): 5-17, December 1984. ]_(t

H. Bruneel and M. Moeneclay, "On the Throughput Performance of Some Continuous ARQ Strategies with Repeated Transmissions," IEEE Trans. Commun .., COM-34: 244-49, March 1986.

]_({£,

S. R. Chandran and S. Lin, "A Selective Repeat ARQ Scheme for Point-toMultipoint Communications and Hs Throughput Analysis," Comput. Commun. Rev., 16(3), 292-301, August 1986.

17, R. Fantacci, "Performance Evaluation of Efficient Continuous ARQ IEEE Trans. Commun., COM-38: 773-81, June 1990.

iu,,.,.,n.N·"

UOO

Chapter 22

Automatic-Repeat-Request Strategies

18, S. R. Chandran and S. Lin, "Selective-Repeat ARQ Schemes for Broadcast Links," IEEE Trans. Commun., COM-40: 12-19, January 1992. 19. G. Benelli, "Some ARQ Protocols with Finite Receiver Buffer," IEEE Trans. Commun., COM-41: 513-23, April 1993. 20,

, "A Selective ARQ Protocol with Finite-Length Buffer," IEEE Trans. Commun., COM-41: 1102-11, July 1993.

21. J. L. Wang and J. A. Silvester, "Optimal Adaptive Multireceiver ARQ Protocols," IEEE Trans. Commun., COM-41: 1816-29, December 1993. 22, G. M. Djuknic and D. L. Schilling, "Performance Analysis of an ARQ Transmission Scheme for Meteor Burst Communications," IEEE Trans. Commun., COM-42: 268-271, February/March/April 1994. 23, S. B. Wicker, Error Control Systems for Digital Communications and Storage, Prentice Hall, Englewood Cliffs, N.J., 1995. 24!. R. Cam and C. Leung, "Throughput Analysis of Some ARQ Protocols in the Presence of Feedback Errors," IEEE Trans. Commun., COM-45, pp. 35-44, January 1997. 25. F. Fantacci, "Performance Evaluation of a Finite Buffer Generalized Selective Repeat ARQ Scheme for Satellite Communications," IEEE Trans. Commun., COM-45: 140-48, February 1997. 26, R. Cam and C. Leung, "Multiplexed ARQ for Time-Varying Channels-Part I: System Model and Throughput Analysis," IEEE Trans. Commun., COM-46: 41-51, January 1998. 27.

, "Multiplexed ARQ for Time-Varying Channels-Part II: Postponed Retransmission Modification and Numerical Results," IEEE Trans. Commun., COM-46: 314-26, March 1998.

28. K. Brayer, "Error Control Techniques Using Binary Symbol Burst Codes," IEEE Trans. Commun., COM-16: 199-214, April 1968. 29. E. Y. Rocher and R. L. Pickholts, "An Analysis of the Effectiveness of Hybrid Transmission Scheme," IBM]. Res. Dev., 7, 426-33, July 1970. 30. D. M. Mandelbaum, "Adaptive-Feedback Coding Scheme Using Incremental Redundancy," IEEE Trans. Inform. Theory, IT-20: 388-89, May 1974. 31. A. R. K Sastry, "Performance of Hybrid Error Control Scheme on Satellite Channels," IEEE Trans. Commun., COM-23: 689-94, July 1975. 32. A. R. K. Sastry and L. N. Kanal, "Hybrid Error Control Using Retransmission and Generalized Burst-Trapping Codes," IEEE Trans. Commun., COM-24: 385-93, April 1976. 33. P. S. Sindhu, "Retransmission Error Control with Memory," IEEE Trans. Commun., COM-25: 473-79, May 1977.

2l4L S. Lin and J. S. Ma, "A Hybrid ARO System with Error '' IBM Res. , 7478(#32232),

Retransmission for

3:i J. J. in Block-Retransmission'--""'"'~'·""'"·" IEEE Trans. 1979. Commun., COM-27: 525-32, ".Ptoc.

37, S. Lin and P. S. Yu, "SPREC: An Effective , 7591(#32852) April 1979. 3:tt H. Yamamoto and K. Ho, "Viterbi

Codes with September 1980. 3,9),

"IEEE

IT-26:

S. Lin and P. S. Yu, "A Hybrid ARO Scheme with Retransmission for Error Control of Satellite Channels," IEEE Trans. Cormnun., COM-30: 1701-19, 1982.

&].([lo

ened TX, 1982.

and M. Miller, "A ARO Scheme Codes," Proc. IEEE GLOBECOM, pp. C 8.6.1-C.8.6.5,

,;:.].., Y. M. Wang and S.

"A Modified Type-H System and Hs Performance Analysis," IEEE Trans. Commun., COM-31: 593-607, May 1983.

6J.:t A Drukarev and D. J. Costello, Jr., tial Decoding," IEEE Trans.

ARO Error Control IT-29: 521 1983.

til2t D. Chase, P. D. and J. K. Wolf, of Code to a Selective-Repeat ARO " Proc. MILCOM'85, Rec., vol. L pp. 247-52, 1985. ,dJ,:i\,

D. Chase, "Code Combining: A Maximum-Likelihood for Combining an Number of Noisy "IEEE Trans. Commun., COM-33: 385-93, May 1985.

4\§,

H. Krishna and S. D. Morgera, "A New Error Control Scheme for Systems," IEEE Trans. Commun., COM-35: 981-90, October 1987.

ARO

,011£, J. Hagenauer, "Rate-Compatible Punctured Convolutional Codes (RCPC

Codes) and Their Applications," IEEE Trans. Commun., COM-36: April 1988. 61,7, S. Kallel and D. Haccoun, "Sequential Decoding with ARO and Code Combining: A Robust Hybrid FEC/ARO System," IEEE Trans. Commun., COM-36: 773-80, July 1988. 4llt S. D. Morgera and V. K. Oduol, "Soft-Decision Decoding to the Generalized Type-H Hybrid-ARO Scheme," IEEE Trans. Commun., COM-37: 393-96, April 1989.

1202

Chapter 22

Automatic-Repeat-Request Strategies

49. L. R. Lugand, D. J. Costello, and R.H. Deng, "Parity Retransmission Hybrid ARQ Using Rate 1/2 Convolutional Codes on a Nonstationary Channel," IEEE Trans. Commun., COM-37: 755-65, July 1989. 5(t S. B. Wicker, "High Reliability Data Transfer over the Land Mobile Radio Channel," IEEE Trans. Vehic. Technol., 39: 48-55, February 1990.

51. T. Takata, T. Fujiwara, T. Kasami, and S. Lin, "An Error Control System with Multiple-Stage Forward Error Corrections," IEEE Trans. Commun., COM-38: 1799-1809, October 1990. 52. S. Kalle! and D. Haccoun, "Generalized Type-U Hybrid-ARQ Schemes Using Punctured Convolutional Codes," IEEE Trans. Commun., COM-38: 1938-46, November 1990. 53. R.H. Deng, "A Hybrid ARQ Scheme Using TCM and Code Combining," Electron. Lett., 27: 866-68, May 1991. 54!. M. B. Pursley and S. D. Sandberg, "Variable-Rate Hybrid ARQ for MeteorBurst Communications," IEEE Trans. Commun., COM-40: 60-73, January 1992. 55. S. Kallel and C. Leung, "Efficient ARQ Schemes with Multiple Copy Decoding," IEEE Trans. Commun., COM-40: 642-50, March 1992. 56. S. B. Wicker, "Reed-Solomon Error Control Coding for Data Transmission over Rayleigh Fading Channels with Feedback," IEEE Trans. Vehic. Technol., 41: 124-33, May 1992. 57. V. K. Oduol and S. D. Morgera, "Performance Evaluation of the Generalized Type-H Hybrid ARQ Scheme with Noisy Feedback on Markov Channels," IEEE Trans. Commun., COM-41: 32-40, January 1993. 58. S. Kalle!, "Efficient Hybrid ARQ Protocols with Adaptive Forward Error Correction," IEEE Trans. Commun., COM-42: 281-89, February/March/April 1994. 59. M. D. Rice, "Application of Generalized Minimum Distance Decoding to Hybrid-ARQ Error Control," IEEE Trans. Commun., COM-42: 640-47, February/March/April 1994. 60. L. de Alfaro and A. R. Meo, "Codes for Second and Third Order GH-ARQ Schemes," IEEE Trans. Commun., COM-42: 899-910, February/March/April 1994. 61. M. D. Rice and S. B. Wicker, "Adaptive Error Control for Slowly Varying Channels," IEEE Trans. Commun., COM-42: 917-26, February/March/April 1994. 62. S. B. Wicker and M. J. Bartz, "The Design and Implementation of Type-I and Type-U Hybrid-ARQ Protocols Based on First-Order Reed-Muller Codes," IEEE Trans. Commun., COM-42: 979-87, February/March/April 1994.

16i2t

Protocols Using Punctured MDS Codes," , "Type-H IEEE Trans. Commun. . COM-42: 1994.

~,,rn,u,.crna,-,,"

6§0

S. K. Lee and M. C.

ARO Schemes Coded Modulation and IEEE Trans. Commun" COM-42: June 1994. "'An

6(6;, L K. Rasmussen and S. B. Vv'icker,

ARQProto-

cols Based on CRC 2569- 75, October 1995. rftlo K. Sakakibara and IV!. MDS Codes and GMD December 1995.

Vv'VUOU/o.,"

· A Multicast J!EEE Trans. Commun., COM-43: "IEEE Trans.

APPENDIX

A

Galois fields GF(2 111 ) for 3 ::: m ::: 10 are given. Each element in GF(2 111 ) is expressed as a power of some primitive element a and as a linear sum of a 0a, ... , a 111 - 1 . The minimal polynomial of a is used to generate the field. For every 111, the generating polynomial of the field is given. Each element is represented by the following notation: (ao,

Cll, ...• Clm-d

where the integer i represents ai, and the binary m-tuple (ao, ai = ClQ

a 1 , .... Clm- l)

+ a1a + ... + llm-l<Xm-1

Gmfoi§ Fi.eDi!ll§ ())JI' 01ndleir 2111

JL GF(2 3 ) generated by p(X) = 1 + X

0 1 2

2. GF(2 4 ) generated by p(X) 0 1 2 3 4 5 6 3,,

+ X3

000 100 010 001 = 1+X

3 4 5 6

110 011 111 101

7 8 9

1101 1010 0101 1110 0111 1111 1011 1001

+ X4

0000 1000 0100 0010 0001 1100 0110 0011

10

11 12 13 14

GF(2 5 ) generated by p(X) = 1 + x 2 + x 5 0 1 2 3 4 5

00000 10000 01000 00100 00010 00001 10100

6 7 8 9 10 11

12

01010 00101 10110 01011 10001 11100 01110

represents

Appendix A

13 14 15 16 17 18

19 20 21 4\" GF(2 6 ) generated

p(X)

0 1 2 3 4 5 6 7 8 9 10 11 12 13

14 15 16 17

18 19 20 21 22 23 24 25 26 27 28 29 30

00111 10111 11111 11011 11001 11000 01100 00110 00011

Tables of Galois Fields

22 23 24 25 26 27 28 29 30

10101 11110 01111 10011 11101 11010 01101 10010 01001

31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62

101001 100100 010010 001001 110100 011010 001101 110110 011011 111101 101110 010111 111011 101101 100110 010011 111001 101100 010110 001011 110101 101010 010101 111010 011101 111110 011111 111111 101111 100111 100011 100001

= 1 + X + X6 000000 100000 010000 001000 000100 000010 000001 110000 011000 001100 000110 000011 110001 101000 010100 001010 000101 110010 011001 111100 011110 001111 110111 101011 100101 100010 010001 111000 011100 001110 000111 110011

12(!]5

1206

Appendix /1,

Tables of Galois Fields

5. GF(2 7 ) generated by p(X) = 1 + x 3+ x7

14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42

0000000 1000000 0100000 0010000 0001000 0000100 0000010 0000001 1001000 0100100 0010010 0001001 1001100 0100110 0010011 1000001 1101000 0110100 0011010 0001101 1001110 0100111 1011011 1100101 1111010 0111101 1010110 0101011 1011101 1100110 0110011 1010001 1100000 0110000 0011000 0001100 0000110 0000011 1001001 1101100 0110110 0011011 1000101 1101010

43

VLlV1-V1-

0 1 2 3 4 5 6 7 8 9 10 11 12 13

A"1 1 £\1 A'I

44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87

1010010 0101001 1011100 0101110 0010111 1000011 1101001 1111100 0111110 0011111 1000111 1101011 1111101 1110110 0111011 1010101 1100010 0110001 1010000 0101000 0010100 0001010 0000101 1001010 0100101 1011010 0101101 1011110 0101111 1011111 1100111 1111011 1110101 1110010 0111001 1010100 0101010 0010101 1000010 0100001 1011000 0101100 0010110 0001011

88

1001101

71 72

Tables of Galois Fields

A

89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 (/j;o

GF(2 8 ) generated

1101110 0110111 1010011 1100001 1111000 0111100 0011110 0001111 1001111 1101111 1111111 1110111 1110011 1110001 1110000 0111000 0011100 0001110 0000111

p(X)

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

108 109 110 111 112 113 114, 115 116 117 118 119 120 121 122 123 124 125 126

+ v8

= 1 + x 2 + x3 +

00000000 10000000 01000000 00100000 00010000 00001000 00000100 00000010 00000001 10111000 01011100 00101110 00010111 10110011 11100001 11001000 01100100 00110010 00011001 10110100 01011010 00101101 10101110 01010111 10010011

1001011 1101101 1111110 0111111 1010111 1100011 1111001 1110100 0111010 0011101 1000110 0100011 1011001 1100100 0110010 0011001 1000100 0100010 0010001 A

24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

11110001 11000000 01100000 00110000 00011000 00001100 00000110 00000011 10111001 11100100 01110010 00111001 10100100 01010010 00101001 10101100 01010110 00101011 10101101 11101110 01110111 10000011 11111001 11000100 01100010

n01

1208

Appendix A

Tables of Gaiois Fieids

49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95

00110001 10100000 01010000 00101000 00010100 00001010 00000101 10111010 01011101 10010110 01001011 10011101 11110110 01111011 10000101 11111010 01111101 10000110 01000011 10011001 11110100 01111010 00111101 10100110 01010011 10010001 11110000 01111000 00111100 00011110 00001111 10111111 11100111 11001011 11011101 11010110 01101011 10001101 l 1111110 01111111 10000111 11111011 11000101 11011010 01101101 10001110 01000111

96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142

10011011 11110101 11000010 01100001 10001000 01000100 00100010 00010001 10110000 01011000 00101100 00010110 00001011 10111101 11100110 01110011 10000001 11111000 01111100 00111110 00011111 10110111 11100011 11001001 11011100 01101110 00110111 10100011 11101001 11001100 01100110 00110011 10100001 11101000 01110100 00111010 00011101 10110110 01011011 10010101 11110010 01111001 10000100 01000010 00100001 10101000 01010100

t\

143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189

00101010 00010101 10110010 01011001 10010100 01001010 00100101 10101010 01010101 10010010 01001001 10011100 01001110 00100111 10101011 11101101 11001110 01100111 10001011 11111101 11000110 01100011 10001001 11111100 01111110 00111111 10100111 11101011 11001101 11011110 01101111 10001111 11111111 11000111 11011011 11010101 11010010 01101001 10001100 01000110 00100011 10101001 11101100 01110110 00111011 10100101 11101010

190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236

Tables of Galois Fields

01110101 10000010 01000001 10011000 01001100 00100110 00010011 10110001 11100000 01110000 00111000 00011100 00001110 00000111 10111011 11100101 11001010 01100101 10001010 01000101 10011010 01001101 10011110 01001111 10011111 11110111 11000011 11011001 11010100 01101010 00110101 10100010 01010001 10010000 01001000 00100100 00010010 00001001 10111100 01011110 00101111 10101111 11101111 11001111 11011111 11010111 11010011

121())9

1210

Appendix A

Tables of Galois Fields

237 238 239 240 241 242 243 244 245

11010001 11010000 01101000 00110100 00011010 00001101 10111110 01011111 10010111

7. GF(29 ) generated by p(X) 0 1 2 3 4 5 6 7 8 9 10 11 12

13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33

= 1

246 247 248 249 250 251 252 253 254

11011000 01101100 00110110 00011011 10110101 11100010 01110001

34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68

101011010 010101101 101000110 010100011 101000001 110110000 011011000 001101100 000110110 000011011 100011101 110011110 011001111 101110111 110101011 111000101 111110010 011111001 101101100 010110110 001011011 100111101 110001110 011000111 101110011 110101001 111000100 011100010 001110001 100101000 010010100 001001010 000100101 100000010 010000001

11110011 1 1 (I()(){){) 1

.L_l_VVVVVJ..

+ x 4 + x9

000000000 100000000 010000000 001000000 000100000 000010000 000001000 000000100 000000010 000000001 100010000 010001000 001000100 000100010 000010001 100011000 010001100 001000110 000100011 100000001 110010000 011001000 001100100 000110010 000011001 100011100 010001110 001000111 100110011 110001001 111010100 011101010 001110101 100101010 010010101

Appendix A

69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95

96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 11,4

115

101010000 010101000 001010100 000101010 000010101 100011010 010001101 101010110 010101011 101000101 110110010 011011001 101111100 010111110 001011111 100111111 110001111 111010111 111111011 111101101 111100110 011110011 101101001 110100100 011010010 001101001 100100100 010010010 001001001 100110100 010011010 001001101 100110110 010011011 101011101 110111110 011011111 101111111 110101111 111000111 111110011 111101001 111100100 011110010 001111001 100101100 010010110

116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162

Tables of Galois Fields

001001011 100110101 110001010 011000101 101110010 010111001 101001100 010100110 001010011 100111001 110001100 011000110 001100011 100100001 110000000 011000000 001100000 000110000 000011000 000001100 000000110 000000011 100010001 110011000 011001100 001100110 000110011 100001001 110010100 011001010 001100101 100100010 010010001 101011000 010101100 001010110 000101011 100000101 110010010 011001001 101110100 010111010 001011101 100111110 010011111 101011111 110111111

'i2n

1212

Appendix A

Tables of Galois Fields

163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209

111001111 111110111 111101011 111100101 111100010 011110001 101101000 010110100 001011010 000101101 100000110 010000011 101010001 110111000 011011100 001101110 000110111 100001011 110010101 111011010 011101101 101100110 010110011 101001001 110110100 011011010 001101101 100100110 010010011 101011001 110111100 011011110 001101111 100100111 110000011 111010001 111111000 011111100 001111110 000111111 100001111 110010111 111011011 111111101 111101110 011110111 101101011

210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256

110100101 111000010 011100001 101100000 010110000 001011000 000101100 000010110 000001011 100010101 110011010 011001101 101110110 010111011 101001101 110110110 011011011 101111101 110101110 011010111 101111011 110101101 111000110 011100011 101100001 110100000 011010000 001101000 000110100 000011010 000001101 100010110 010001011 101010101 110111010 011011101 101111110 010111111 101001111 110110111 111001011 111110101 111101010 011110101 101101010 010110101 101001010

[;

~-

' I

p,

1j

l

257 258 259 260 261 262 263 26£!, 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303

010100101 101000010 010100001 101000000 010100000 001010000 000101000 000010100 000001010 000000101 100010010 010001001 101010100 010101010 001010101 100111010 010011101 101011110 010101111 101000111 110110011 111001001 111110100 011111010 001111101 100101110 010010111 101011011 110111101 111001110 011100111 101100011 110100001 111000000 011100000 001110000 000111000 000011100 000001110 000000111 100010011 110011001 111011100 011101110 001110111 100101011 110000101

304 305 306 307 308 309 310

311 312 313 3M 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350

Tables of Galois Fields

111010010 011101001 101100100 010110010 001011001 100111100 010011110 001001111 100110111 110001011 111010101 111111010 011111101 101101110 010110111 101001011 110110101 111001010 011100101 101100010 010110001 101001000 010100100 001010010 000101001 100000100 010000010 001000001 100110000 010011000 001001100 000100110 000010011 100011001 110011100 011001110 001100111 100100011 110000001 111010000 011101000 001110100 000111010 000011101 100011110 010001111 101010111

UB

1214

Appendix A

Tables of Galois Fields

351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397

110111011

111001101 111110110 011111011 101101101 110100110 011010011 101111001 110101100 011010110 001101011 100100101 110000010 011000001 101110000 010111000 001011100 000101110 000010111 1()()011()11

110011101 111011110 011101111 101100111 110100011 111000001 111110000 011111000 001111100 000111110 000011111 100011111 110011111 111011111 111111111 111101111 111100111 111100011 111100001 111100000 011110000 001111000 000111100 000011110 000001111 100010111 110011011

398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444

111011101 111111110

011111111 101101111 110100111 111000011 111110001 111101000 011110100 001111010 000111101 100001110 010000111 101010011 110111001 111001100 011100110 001110011 100101001 110000100 011000010 001100001 100100000 010010000 001001000 000100100 000010010 000001001 100010100 010001010 001000101 100110010 010011001 101011100 010101110 001010111 100111011 110001101 111010110 011101011 101100101 110100010 011010001 101111000 010111100 001011110 000101111

Appendix A

445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477

100000111 110010011 111011001 111111100 011111110 001111111 100101111 110000111 111010011 111111001 111101100 011110110 001111011 100101101 110000110 011000011 101110001 110101000 011010100 001101010 000110101 100001010 010000101 101010010 010101001 101000100 010100010 001010001 100111000 010011100 001001110 000100111 100000011

it GF(2 10 ) generated by p(X)

0 1 2 3 4 5 6 7 8 9

=

Tables of Galois Fields

478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510

110010001 111011000 011101100 001110110 000111011 100001101 110010110 011001011 101110101 110101010 011010101 101111010 010111101 101001110 010100111 101000011 110110001 111001000 011100100 001110010 000111001 100001100 010000110 001000011 100110001 110001000 011000100 001100010 000110001 100001000 010000100 001000010 000100001

10 11 12

1001000000 0100100000 0010010000 0001001000 0000100100 0000010010 0000001001 1001000100 0100100010 0010010001 1000001000

1 + x 3 + x 10

0000000000 1000000000 0100000000 0010000000 0001000000 0000100000 0000010000 0000001000 0000000100 0000000010 0000000001

13

14 15 16 17 18 19 20

'1215

1216

Appendix A

Tables of Galois Fields

21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67

0100000100 0010000010 0001000001 1001100000 0100110000 0010011000 0001001100 0000100110 0000010011 1001001001 1101100100 0110110010 0011011001 1000101100 0100010110 0010001011 1000000101 1101000010 0110100001 1010010000 0101001000 0010100100 0001010010 0000101001 1001010100 0100101010 0010010101 1000001010 0100000101 1011000010 0101100001 1011110000 0101111000 0010111100 0001011110 0000101111 1001010111 1101101011 1111110101 1110111010 0111011101 1010101110 0101010111 1011101011 1100110101 1111011010 0111101101

68 69 70

71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 00

00

89 90 91 92 93 94 95

96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114

1010110110 0101011011 1011101101 1100110110 0110011011 1010001101 1100000110 0110000011 1010000001 1100000000 0110000000 0011000000 0001100000 0000110000 0000011000 0000001100 0000000110 0000000011 1001000001 1101100000 0110110000 0011011000 0001101100 0000110110 0000011011 1001001101 1101100110 0110110011 1010011001 1100001100 0110000110 0011000011 1000100001 1101010000 0110101000 0011010100 0001101010 0000110101 1001011010 0100101101 1011010110 0101101011 1011110101 1100111010 0110011101 1010001110 0101000111

Appendix A

115 116 117 118 119 120 121 122 123 124. 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161

1011100011 1100110001 1111011000 0111101100 0011110110 0001111011 1001111101 1101111110 0110111111 1010011111 1100001111 1111000111 1110100011 1110010001 1110001000 0111000100 0011100010 0001110001 1001111000 0100111100 0010011110 0001001111 1001100111 1101110011 1111111001 1110111100 0111011110 0011101111 1000110111 1101011011 1111101101 1110110110 0111011011 1010101101 1100010110 0110001011 1010000101 1100000010 0110000001 1010000000 0101000000 0010100000 0001010000 0000101000 0000010100 0000001010 0000000101

162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208

Tables of Galois Fields

1001000010 0100100001 1011010000 0101101000 0010110100 0001011010 0000101101 1001010110 0100101011 1011010101 1100101010 0110010101 1010001010 0101000101 1011100010 0101110001 1011111000 0101111100 0010111110 0001011111 1001101111 1101110111 1111111011 1110111101 1110011110 0111001111 1010100111 1100010011 1111001001 1110100100 0111010010 0011101001 1000110100 0100011010 0010001101 1000000110 0100000011 1011000001 1100100000 0110010000 0011001000 0001100100 0000110010 0000011001 1001001100 0100100110 0010010011

nn

1218

Appendix A

Tables of Galois Fields

209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255

1000001001 1101000100 0110100010 0011010001 1000101000 0100010100 0010001010 0001000101 1001100010 0100110001 1011011000 0101101100 0010110110 0001011011 1001101101 1101110110 0110111011 1010011101 1100001110 0110000111 1010000011 1100000001 1111000000 0111100000 0011110000 0001111000 0000111100 0000011110 0000001111 1001000111 1101100011 1111110001 1110111000 0111011100 0011101110 0001110111 1001111011 1101111101 1111111110 0111111111 1010111111 1100011111 1111001111 1110100111 1110010011 1110001001 1110000100

256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302

0111000010 0011100001 1000110000 0100011000 0010001100 0001000110 0000100011 1001010001 1101101000 0110110100 0011011010 0001101101 1001110110 0100111011 1011011101 1100101110 0110010111 1010001011 1100000101 1111000010 0111100001 1010110000 0101011000 0010101100 0001010110 0000101011 1001010101 1101101010 0110110101 1010011010 0101001101 1011100110 0101110011 1011111001 1100111100 0110011110 0011001111 1000100111 1101010011 1111101001 1110110100 0111011010 0011101101 1000110110 0100011011 1011001101 1100100110

Tables of Galois Fields

303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349

0110010011 1010001001 1100000100 0110000010 0011000001 1000100000 0100010000 0010001000 0001000100 0000100010 0000010001 1001001000 0100100100 0010010010 0001001001 1001100100 0100110010 0010011001 1000001100 0100000110 0010000011 1000000001 1101000000 0110100000 0011010000 0001101000 0000110100 0000011010 0000001101 1001000110 0100100011 1011010001 1100101000 0110010100 0011001010 0001100101 1001110010 0100111001 1011011100 0101101110 0010110111 1000011011 1101001101 1111100110 0111110011 1010111001 1100011100

350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396

0110001110 0011000111 1000100011 1101010001 1111101000 0111110100 0011111010 0001111101 1001111110 0100111111 1011011111 1100101111 1111010111 1110101011 1110010101 1110001010 0111000101 1010100010 0101010001 1011101000 0101110100 0010111010 0001011101 1001101110 0100110111 1011011011 1100101101 1111010110 0111101011 1010110101 1100011010 0110001101 1010000110 0101000011 1011100001 1100110000 0110011000 0011001100 0001100110 0000110011 1001011001 1101101100 0110110110 0011011011 1000101101 1101010110 0110101011

U:191

1220

Appendix A

Tables of Galois Fields

397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443

1010010101 1100001010 0110000101 1010000010 0101000001 1011100000 0101110000 0010111000 0001011100 0000101110 0000010111 1001001011 1101100101 1111110010 0111111001 1010111100 0101011110 0010101111 1000010111 1101001011 1111100101 1110110010 0111011001 1010101100 0101010110 0010101011 1000010101 1101001010 0110100101 1010010010 0101001001 1011100100 0101110010 0010111001 1000011100 0100001110 0010000111 1000000011 1101000001 1111100000 0111110000 0011111000 0001111100 0000111110 0000011111 1001001111 1101100111

444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490

1111110011 1110111001 1110011100 0111001110 0011100111 1000110011 1101011001 1111101100 0111110110 0011111011 1000111101 1101011110 0110101111 1010010111 1100001011 1111000101 1110100010 0111010001 1010101000 0101010100 0010101010 0001010101 1001101010 0100110101 1011011010 0101101101 1011110110 0101111011 1011111101 1100111110 0110011111 1010001111 1100000111 1111000011 1110100001 1110010000 0111001000 0011100100 0001110010 0000111001 1001011100 0100101110 0010010111 1000001011 1101000101 1111100010 0111110001

Appendix A

491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537

1010111000 0101011100 0010101110 0001010111 1001101011 1101110101 1111111010 0111111101 1010111110 0101011111 1011101111 1100110111 1111011011 1110101101 1110010110 0111001011 1010100101 1100010010 0110001001 1010000100 0101000010 0010100001 1000010000 0100001000 0010000100 0001000010 0000100001 1001010000 0100101000 0010010100 0001001010 0000100101 1001010010 0100101001 1011010100 0101101010 0010110101 1000011010 0100001101 1011000110 0101100011 1011110001 1100111000 0110011100 0011001110 0001100111 1001110011

538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584

Tables of Galois Fields

1101111001 1111111100 0111111110 0011111111 1000111111 1101011111 1111101111 1110110111 1110011011 1110001101 1110000110 0111000011 1010100001 1100010000 0110001000 0011000100 0001100010 0000110001 1001011000 0100101100 0010010110 0001001011 1001100101 1101110010 0110111001 1010011100 0101001110 0010100111 1000010011 1101001001 1111100100 0111110010 0011111001 1000111100 0100011110 0010001111 1000000111 1101000011 1111100001 1110110000 0111011000 0011101100 0001110110 0000111011 1001011101 1101101110 0110110111

]221

1222

Appendix A

Tables of Galois Fields

585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631

1010011011 1100001101 1111000110 0111100011 1010110001 1100011000 0110001100 0011000110 0001100011 1001110001 1101111000 0110111100 0011011110 0001101111 1001110111 1101111011 1111111101 1110111110 0111011111 1010101111 1100010111 1111001011 1110100101 1110010010 0111001001 1010100100 0101010010 0010101001 1000010100 0100001010 0010000101 1000000010 0100000001 1011000000 0101100000 0010110000 0001011000 0000101100 0000010110 0000001011 1001000101 1101100010 0110110001 1010011000 0101001100 0010100110 0001010011

632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678

1001101001 1101110100 0110111010 0011011101 1000101110 0100010111 1011001011 11 nm nm n1

1111010010 0111101001 1010110100 0101011010 0010101101 1000010110 0100001011 1011000101 1100100010 0110010001 1010001000 0101000100 0010100010 0001010001 1001101000 0100110100 0010011010 0001001101 1001100110 0100110011 1011011001 1100101100 0110010110 0011001011 1000100101 1101010010 0110101001 1010010100 0101001010 0010100101 1000010010 0100001001 1011000100 0101100010 0010110001 1000011000 0100001100 0010000110 0001000011

A

679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725

1001100001 1101110000 0110111000 0011011100 0001101110 0000110111 1001011011 1101101101 1111110110 0111111011 1010111101 1100011110 0110001111 1010000111 1100000011 1111000001 1110100000 0111010000 0011101000 0001110100 0000111010 0000011101 1001001110 0100100111 1011010011 1100101001 1111010100 0111101010 0011110101 1000111010 0100011101 1011001110 0101100111 1011110011 1100111001 1111011100 0111101110 0011110111 1000111011 1101011101 1111101110 0111110111 1010111011 1100011101 1111001110 0111100111 1010110011

726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772

Tables of Galois Fields

1100011001 1111001100 0111100110 0011110011 1000111001 1101011100 0110101110 0011010111 1000101011 1101010101 1111101010 0111110101 1010111010 0101011101 1011101110 0101110111 1011111011 1100111101 1111011110 0111101111 1010110111 1100011011 1111001101 1110100110 0111010011 1010101001 1100010100 0110001010 0011000101 1000100010 0100010001 1011001000 0101100100 0010110010 0001011001 1001101100 0100110110 0010011011 1000001101 1101000110 0110100011 1010010001 1100001000 0110000100 0011000010 0001100001 1001110000

1223l

1224

Appendix A

Tables of Galois Fields

773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819

0100111000 0010011100 0001001110 0000100111 1001010011 1101101001 1111110100 0111111010 0011111101 1000111110 0100011111 1011001111 1100100111 1111010011 1110101001 1110010100 0111001010 0011100101 1000110010 0100011001 1011001100 0101100110 0010110011 1000011001 1101001100 0110100110 0011010011 1000101001 1101010100 0110101010 0011010101 1000101010 0100010101 1011001010 0101100101 1011110010 0101111001 1011111100 0101111110 0010111111 1000011111 1101001111 1111100111 1110110011 1110011001 1110001100 0111000110

820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866

0011100011 1000110001 1101011000 0110101100 0011010110 0001101011 1001110101 1101111010 0110111101 1010011110 0101001111 1011100111 1100110011 1111011001 1110101100 0111010110 0011101011 1000110101 1101011010 0110101101 1010010110 0101001011 1011100101 1100110010 0110011001 1010001100 0101000110 0010100011 1000010001 1101001000 0110100100 0011010010 0001101001 1001110100 0100111010 0010011101 1000001110 0100000111 1011000011 1100100001 1111010000 0111101000 0011110100 0001111010 0000111101 1001011110 0100101111

Tables of Galois Fields

867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913

1011010111 1100101011 1111010101 1110101010 0111010101 1010101010 0101010101 1011101010 0101110101 1011111010 0101111101 1011111110 0101111111 1011111111 1100111111 1111011111 1110101111 1110010111 1110001011 1110000101 1110000010 0111000001 1010100000 0101010000 0010101000 0001010100 0000101010 0000010101 1001001010 0100100101 1011010010 0101101001 1011110100 0101111010 0010111101 1000011110 0100001111 1011000111 1100100011 1111010001 1110101000 0111010100 0011101010 0001110101 1001111010 0100111101 1011011110

914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960

0101101111 101111011 1100111011 1111011101 1110101110 0111010111 1010101011 1100010101 1111001010 0111100101 1010110010 0101011001 1011101100 0101110110 0010111011 1000011101 1101001110 0110100111 1010010011 1100001001 1111000100 0111100010 0011110001 1000111000 0100011100 0010001110 0001000111 1001100011 1101110001 1111111000 0111111100 0011111110 0001111111 1001111111 1101111111 1111111111 1110111111 1110011111 1110001111 1110000111 1110000011 1110000001 1110000000 0111000000 0011100000 0001110000 0000111000

n2s

1226

Appendix A

Tables of Galois Fields

961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991

0000011100 0000001110 0000000111 1001000011 1101100001 1111110000 0111111000 0011111100 0001111110 0000111111 1001011111 1101101111 1111110111 1110111011 1110011101 1110001110 0111000111 1010100011 1100010001 1111001000 0111100100 0011110010 0001111001 1001111100 0100111110 0010011111 1000001111 1101000111 1111100011 1110110001 1110011000

992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022

0111001100 0011100110 0001110011 1001111001 1101111100 0110111110 0011011111 1000101111 1101010111 1111101011 1110110101 1110011010 0111001101 1010100110 0101010011 1011101001 1100110100 0110011010 0011001101 1000100110 0100010011 1011001001 1100100100 0110010010 0011001001 1000100100 0100010010 0010001001 1000000100 0100000010 0010000001

fj\f~ D[ID ~ UUU@ ~

~ \\f [n)@U11J D@ ~ ~ clJ

(im711 )

[E~®mrD®ITTJLt~ D[Ft)

@~

A list of all minimal of elements in ) with 111 ::: 10 is given. Each ) is as a power of a primitive element a in The minimal ; (X), of ai is given the powers of its nonzero terms. For let m = 6. The notation

5

(0.1.2,5.6)

represents the field element a 5 in GF(2 6 ) whose minimal ¢s(X)

= 1+X +

+ xs + x6.

For each m, the field ) is generated by the first minimal ¢1(X)].For GF(2 6 )is by ¢1 ai 111

and all its

=l+X+X 6 .

have the same minimal

for

=6, 3

(0. 1. 2. 4. 6)

that ¢3(X) = 1 + X +

+

+

is the minimal polynomial of

lL m = 2

Zo

2t

111

111

[i.e.,

1

(0. L

1

(0.1.3)

3

(0, 2, 3)

1 5

(0, 1, 4) (0, 1. 2)

3 7

(0,1,2,3,4) (0,3.4)

= 3

= 4

1228

Appendix B

Minimal Polynomials of Elements in G,C(2111 )

4. m =5 1 5 11

(0,2,5) (0, 1, 2, 4, 5) (0, 1, 3, 4, 5)

3 7 15

(0,2,3,4,5) (0, 1,2,3,5) (0,3,5)

1 5 9 13 21 27

(0, 1, 6) (0, 1, 2, 5, 6) (0,2,3) (0, 1, 3, 4, 6) (0, 1, 2) (0, 1, 3)

3 7 11 15 23 31

(0,1,2,4,6) (0, 3, 6) (0, 2, 3,-5,-6) (0, 2, 4, 5, 6) (0, 1, 4, 5, 6) (0, 5, 6)

5. m =6

6, m =7

7,

111

1 5 9 13 19 23 29 43 55

(0, 3, 7) (0,2,3,4, 7) (0, 1,2,3,4,5,7) (0, 1, 7) (0, 1, 2, 6, 7) (0, 6, 7) (0, 1,3,5, 7) (0, 1, 2, 5, 7) (0,2,3,4,5,6, 7)

3 7 11 15 21 27 31 47 63

(0,1,2,3, 7) (0, 1, 2, 4, 5, 6, 7) (0,2,4,6, 7) (0, 1,2,3,5,6, 7) (0, 2, 5, 6, 7) (0, 1, 4, 6, 7) (0, 4, 5, 6, 7) (0, 3, 4, 5, 7) (0, 4, 7)

1 5 9 13 17 21 25 29 37 43 47 53 59 63 87 95 119

(0, 2, 3, 4, 8) (0, 1, 4, 5, 6, 7, 8) (0,2,3,4,5, 7,8) (0, 1, 3, 5, 8) (0, 1, 4) (0, 1, 3, 7, 8) (O, 1, 3, 4, 8) (0, 2, 3, 7, 8) (0, l, 2, 3, 4, 6, 8) (0, 1, 6, 7, 8) (0, 3, 5, 7, 8) (0, 1, 2, 7, 8) (0,2,3,6,8) (0,2,3,4,6, 7,8) (0, 1, 5, 7, 8) (0, 1, 2, 3, 4, 7, 8) (0, 3, 4)

3 7 11 15 19 23 27 31 39 45 51 55 61 85 91 111 127

(0, 1,2,4,5,6,8) (0,3,5,6,8) (0, 1, 2, 5, 6, 7, 8) (0, 1, 2, 4, 6, 7, 8) (0, 2, 5, 6, 8) (0, 1, 5, 6, 8) (0, 1, 2, 3, 4, 5, 8) (0, 2, 3, 5, 8) (0, 3, 4, 5, 6, 7, 8) (0, 3, 4, 5, 8) (0, l, 2, 3, 4) (0,4,5, 7,8) (0, 1, 2, 3, 6, 7, 8) (0, 1, 2) (0,2,4,5,6, 7,8) (0, 1, 3, 4, 5, 6, 8) (0,4,5,6,8)

=8

B Q>

o

Minimal Polyno1oiials of Elements in

)

n2'BJ

m =9

1 5 9 13 17 21 25 29 35 39 43 47 53 57 61 73 77 83 87 93 103 109 117 123 127 175 187 219 239 'Y)o 111

(0, 4, 9) (0,4,5,8, 9) (0, 1, 4, 8, 9) L 2, 4, 5, 6, 9) (0, L 3, 4, 6, 7, (0, L 2, 4, 9) (0, L 5, 6, 7. 8, 9) 1~3~5,6,8" O 0\ o, _/ J

2,3,6,7,8,9) (0, L 3, 6, 7, 8, 9) 3,4,6,8,9) (0, 2, 4, 7, 9) (0,2,4.5,6, 7,9) (0, 1, 2, 3, 4, 6, 9) 1, -, c:. 0 9) (0 ".J,0,o~ 2, 4, 8, 9) (0, 2, 5. 7, 9) (0,3,4,5,6, 7,9) (0. L 2, 3, 5, 7, 9) 1,2,3,4,5,6,8,9) (0, 1, 2, 3, 6, 8, (0, L 2, 7, 9) 3,5,6,9) (0, 5, 7, 8, 9) (0,3,4,6, 7,8,9) (0, 2, (0,2,3,5,6,8,9)

85 91 95 107 111 119 125 171 183 191 223 255

(0, 3, 4, 6, 9) (0,3,4, 7,9) 2,3,5, 9) (0, 5, 6, 8, 9) 2, 7, 8, 9) 3~ s~ 6i s" 1..2,3,7,8,9) -1l, -2 J,, L1 -,,) 9) ' 1, 2, 3, 5, 6, (0 ' ]_, 4 '::J, r ? o O) u, o, 2.3,4,5,6, 9) (0, 2, 4. 6, 7, 8, (0, 2, 3, 4, 5, 7, 9) (0, 1, 2, 3, 6, 7, 9) 1 0 2 9) ~ '~, .J, 0, 1,3,4,5,6,7,8,9) 1,2,6.7,8,9) (0. L 2, 4, 6, 7, 9) (0, L 3, 6, 9) (0, 3, 4, 5, 7, 8, 9) (0, L 5, 7, 9) (0, L 2, 3, 4, 8, (0, 1, 9) 4, 6, 7, 9) (0,2,4,5,7,8,9) (0, L 3, 5, 8, 9) L 4, 5, 9) (0, L 5, 8, (0,5, 9)

3 7 11 15 19 23 27 31 35 39 43

(0, L 2, 3, 10) (0,3,4,5,6, 7,8,9, 10) (0, 2, 4, 5, 10) (0, L 3 . 5, 7, 8, 10) (0,1,3,4,5,6,7,8, 10) (0, 1, 3, 4, 10) (0, 1,3,4,5,6,8,9, 10) (0, 1, 5, 9, 10) (0, L 4, 9, 10) (0, L 2, 6, 10) (0, 3, 4, 8, 10)

3 7 11 15 19 23 27 31 37 41 45 51 55 59 ?;-~

v~

75

79

,r,

j

C'

,-

= 10 1 5

9 13 17 21 25 29 33 37 41

(0, (0, (0., (0,

3, 10) 2, 3, 8, 10) 1, 2, 3, 5, 7, 10) L 2, 3, 5, 6, 10) 2,3,6,8,9, 10) 1,3,5,6, 7,8,9, 10) (0, L 5, 8, 10) (0,4,5,8, 10) (0, 2, 3, 4, 5) (0, L 5, 6, 8, 9, 10) (0,2,5,6, 7,8, 10)

1230

Appendix B

45 49 53 57 61 69 73 77 83 87 91 95 101 105 109 115 119 123 127 149 155 159 167 173 179 183 189 205 213 219 223 235 239 247 253 341 347 363 375 383 447 495

Minimal Polynomials of Elements in GF(2m)

(0, 4, 5, 9, 10) (0,2,4,6,8,9, 10) (0, 1,2,3, 7,8,10) (0, 4, 6, 9, 10) (0, 1,4,5,6, 7,8,9,10) (0, 6, 7, 8, 10) (0, 1,2,6,8,9, 10) (0, 1, 3, 8, 10) (0,1,4, 7,8,9,10) (0, 3, 6, 7, 10) (0,2,4,5, 7,9,10) (0, 2, 5, 6, 10) (0, 2, 3, 5, 10) (0, 1,2, 7,8,9,10) (0, 1, 2, 5, 10) (O,l,2,4,5,6,7,8,lO) (0, 1,3,4,6,9, 10) (0, 4, 8, 9, 10) (0, 1,2,3,4,5,6, 7, 10) (0, 2, 4, 9, 10) (0, 3, 5, 7, 10) (O,l,2,4,5,6, 7,9,10) (0, 1, 4, 5, 6, 7, 10) (0, 1,2,3,4,6,7,9,lO) (0, 3, 7, 9, 10) (0, 1,2,3,8,9, 10) (0, 1, 5, 6, 10) (0, 1, 3, 7, 10) (0, 1, 3, 4, 7, 8, 10) (0,3,4,5, 7,8,10) (0, 2, 5, 9, 10) (0, 1,2,3,6,9, 10) (0, 1,2,4,6,8, 10) (0, 1,6,9, 10) (0, 5, 6, 8, 10) (0, 1, 2) (0, 1, 6, 8, 10) (0,2,5) (0, 2, 3, 4, 10) (0, 2, 7, 8, 10) (0,3,5, 7,8,9, 10) (0, 1, 2, 3, 5)

47 51 55 59 63 71 75 79 85 89 93 99 103 107 111 117 121 125 147 151 157 165 171 175 181 187 191 207 215 221 231 237 245 251 255 343 351 367 379 439 479 511

(O,l,2,3,4,5,6,9, 10) (0, 1, 2, 5, 6, 8, 10) (0, 1, 3, 5, 8, 9, 10) (0, 3, 4, 5, 8, 9, 10) (0, 2, 3, 5, 7, 9, 10) (0, 1, 4, 6, 7, 9, 10) (0, 1, 2, 3, 4, 8, 10) (0, 1, 2, 5, 6, 7, 10) (0, 1, 2, 6, 7, 8, 10) (0, 1, 2, 4, 6, 7, 10) (0,1,2,3,4,5,6,7,8,9,10) (0, 1, 2, 4, 5) (0,2,3,4,5,6,8,9, 10) (0, 3, 4, 5, 6, 9, 10) (0, 1, 4, 6, 10) (0, 3, 4, 7, 10) (0, 1, 2, 5, 7, 9, 10) (0, 6, 7, 9, 10) (0, 2, 3, 5, 6, 7, 10) (0. 5, 8, 9, 10) (0, 1, 3, 5, 6, 8, 10) (0,3,5) (0, 2, 3, 6, 7, 9, 10) (0, 2, 3, 7, 8, 10) (O,l,3,4,6,7,8,9. 10) (0,2, 7,9, 10) (0, 4, 5, 7, 8, 9, 10) (0, 2, 4, 5, 8, 9, 10) (0, 5, 7, 8, 10) (0, 3, 4, 6, 8, 9, 10) (0, 1,3,4,5) (0, 2, 6, 7, 8, 9, 10) (0, 2, 6, 7, 10) (0,2,3,4,5,6, 7,9,10) (0, 7, 8, 9, 10) (0,2,3,4,8,9, 10) (O,l,2,3,4,5, 7,9, 10) (0, 2, 3, 4, 5, 8, 10) (0, 1, 2, 4, 5, 9, 10) (0, 1, 2, 4, 8, 9, 10) (0, 1, 2, 4, 7, 8, 10) (0, 7, 10)

(G ®ulJ (~Hf@ ~(Cj [1 [f\QJ ~ © V ~ DulJ @J [11f [Jg) [f l [[lfO l @

c~

uu

l® 01J

Jl(}})

rt~

Jl

=

BCH codes of length up to in the representation is coded

as follovvs:

0 <~ 000 2 <---+ OJ O L! +---+ 100 6 +--> 11 Q 1 +-------+ 001 3 <------) 011 5 101 7 <-> 111

When the octal of a digits are the coefficients of the at the left. For consider the polynomial in octal form is

is in binary. the ·with the high-order coefficients BCH code in the table. Its generator

1701317 Expanding this

001 The generator g(X)

we obtain

111 000 001 011

001

111

is then

=xis+

+

Primitive ltd

the

in

+

+

+

BCH Codes of

7

4

11 7 5 26 21 16 11 6 57 51 45 39 36

-1- X 3 -1up to 210

+ X -1- 1 -

1

G2;n11,1?n·1:r[1JJil' 1011[J)Uynnl[J)mmi\21Il

k

15 15 15 31 31 31 31 31 63 63 63 63 63

+

1 1

13

2 3

721 2467 ~5 3551 107657 5423325 313365047 103 12471 1701317 166623567 1033500423

1 2 3 5 7 1 2 ·1 J

4

5

23

1232

Appendix C

Generator Polynomials of Binary Primitive BCH Codes

n

k

t

Generntor pollynomiaB

63 63 63 63 63 63 127 127 127 127 127 127 127 127 127 127 127 127 127 127 127 127 127 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255

30 24 18 16 10 7 120 113 106 99 92 85 78 71 64 57 50 43 36 29 22 15 8 247 239 231 223 215 207 199 191 187 179 171 163 155 147 139 131 123

6 7 10 11 13 15 1 2 3 4 5 6 7 9 10 11 13 14 15 21 23 27 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 18 19

255

115

21

157464165547 17323260404441 1363026512351725 6331141367235453 472622305527250155 5231045543503271737 211 41567 11554743 3447023271 624730022327 130704476322273 26230002166130115 6255010713253127753 1206534025570773100045 335265252505705053517721 54446512523314012421501421 17721772213651227521220574343 3146074666522075044764574721735 403114461367670603667530141176155 123376070404722522435445626637647043 22057042445604554770523013762217604353 7047264052751030651476224271567733130217 435 267543 156720665 75626641375 23157564726421 16176560567636227 7633031270420722341 2663470176115333714567 52755313540001322236351 22624710717340432416300455 15416214212342356077061630637 7500415510075602551574724514601 3757513005407665015722506464677633 1642130173537165525304165305441011711 461401732060175561570722730247453567445 2157133314715101512612502774421420241654 1206140522420660037172103265161412262725 06267 6052666557210024726363640460027635255631 3472737

C ca,

Generator Polynomials of Binary Primitive BCH Codes

k

1233

iGeilllennfoir Jll)([l)Il:Jlmlillil1lll21Il

255

107

22

255

99

23

255

91

25

255

87

26

255

79

27

255

71

29

255

63

30

255

5,:,:_,

31

255

47

42

255

45

43

255

37

45

255

29

,1-,

255

21

55

255

13

59

255

9

63

511 511 511 511 511 511 511 511 511 511 511 511 511

502 493 484 475 466 457 448 439 430 421 412 403 394

1 2 3 4

,I

s

6 7 8 9 10 11

12 13

2220577232206625631241730023534742017657 4750154441 1065666725347317422274141620157433225241 1076432303431 6750265030327444172723631724732511075550 762720724344561 1101367634147432364352316343071720462067 22545273311721317 6670003563765750002027034420736617 462J 01 5326711766541342355 2402471052064432151555417211233116320544 4250362557643221706035 1075447505516354432531521735770700366611 1726455267613656702543301 7315425203501100133015275306032054325414 326755010557044426035473617 2533542017062646563033041377406233175123 334145446045005066024552543173 1520205605523416113110134637642370156367 0024470762373033202157025051541 5136330255067007414177447245437530420735 706174323432347644354737403044003 3025715536673071465527064012361377115342 242324201174114060254657410403565037 1256215257060332656001773153607612103227 341405653074542521153121614466513473725 4641732005052564544426573714250066004330 67744547656140317467721357026134460500547 15726025217472463201031043255355134614162 367212044074545112766115547705561677516057 1021 1112711 1530225571 1630256304641 1112724662161763 1142677410335765707 1034122337164372224005 1561350064670543777423345 1727400306127620173461431627 1317711625267264610360644707513 1337530164410305712316173767147101 1573436303657311762726657724651203651 1102510344130333354270407474305341234033

1234

Appendix C

Generator Polynomials of Binary Primitive BCH Codes

k

t

511

385

14

511

376

15

511

367

17

511

358

18

511

349

19

511

340

20

511

331

21

511

322

22

511

313

23

511

304

25

511

295

26

511

286

27

511

277

28

511

268

29

511

259

30

511

250

31

511

241

36

511

238

37

511

229

38

n

Giell11iemfor polynomial

1775546025777712372455452107300530331444 031 1116744706521725532227162607146216210106 733203 1126657202505666323017001652245562614435 511600655 1574456154545450414733416176535156070037 760411373255 1455012234675753242074501555657377006165 521557376050313 1036075427062474623220662047122611677363 511364110105517777 1540644721106050570342772405117177453215 136663677461641457321 1153614675506111211374366675624670755236 523645062677061770735073 1037043005346411102774516447047070735602 327224637421536736251517437 1510736212272133353502352536320703410147 225273064337077160035254047351 1112630477530033170044524747727767532752 046612603077472052247671744467035 1317463403265645042064775326044775737416 714071756016714523650022734505401471 1111170752122547100341422773660302256230 317751245413717303607737426401526326045 1241160247151367165615372023170221264427 22643765316304350343631063142530173520 5601 1121314111162101532370722243711014463333 4772560250516566143547137606623504332146 46117 1007276607431444342624513277572752237465 2773105153763257577607352353353361571636 37347245 1142630652622415305026460545443230406701 2432776321666056566417103173036006026112 51467341721 1266115765104357454721732420217755547411 1377076177455240605755033452532207221321 266342055373 1441474125377630723466544074104363527262 5110602507742556745735375653737245201524 721767740003105

(

Generator

k

g

511

220

39

511

211

41

511

202

42

511

193

43

511

184

45

511

175

46

511

166

47

511

157

51

511

148

53

511

139

SL!

511

130

55

511

121

58

511

112

59

FE

of

Primitive BCH Cod,es

12:llSi

fG<eli1l<E:Jr2J~([j)Jl' l!]lOfi;}'illl([i)llTl1lll/Jlil

1330350650041246137404371541263763570720 1431575333227603262602752030077406436030 760454644342665055 1537262671145756636471271211466023057434 5664477051253665031617124007140032144713 400007755712030376003 1355074061631664750650654005233224126373 5501617417747326444652077100470767607227 335404174253126577344477 1151502313317462115717012400332054150716 2357615552060114767351204573257260045473 425675673673505075470425055 1320551776766374340772233460740727550552 2041306675275253552627101444370345136765 16425364-4172265100216633505671 1202260564674312433743724246412313100673 3703766567307723446714611616650062423137 214200762270577576135715734456503 1105764456740764631742144442303723134662 4415551173770646017645044577670522052736 107022447017356700151227221443336113 1710171171400604264434731140711521370753 6440012114236665475350701102415515624310 174161400004517373603747715514145565503 1414131546120371255541603030602601124642 7727350174267055345645560670707547107070 535577450405662075046364112446577412 302461 1775336143053070047771371572323076147655 0747705134625235605330765134143000233634 1267612204553536475125547007426342706405 51763 1126210420631670733262507445577025266400 0744725674510475154245554654156641313571 4050203537624232123030520531734566124772 13206321 1541271357655772525604106044116552170135 4434750270764625323125324327677742325121 2771735535703734317405374311756764357501 74233577257 1264236440177126545461720120275756614321 2522620050735005241425745450416057231261 2131152145444635107565754242126117753372 24522460653365

1236

Appendix C

Generator Polynomials of Binary Primitive BCH Codes

k

t

511

103

61

511

94

62

511

85

63

511

76

85

511

67

87

511

58

91

511

49

93

511

40

95

511

31

109

511

28

111

n

Gelrl.ernftor polynomfaR

1267502203437327636237202040115050677105 7315147230644142256367333406167475350432 5307201333612360340675162226660265473307 65357307372010437 1034624554522076136543376246276501412751 3662235012007006252201651305170664740707 5134565753153573011560335354474211375326 02636772420754543335 1311423305612705436042312400305355360613 5536677204454060166477652423477032137140 3157621106144232601520207000161502003022 44744512244665460344315 1266305771250774445612456156722760607324 4460173371073371716611544036044377532354 6736213462531106045203515637171342621252 52705066142632374105237105 1572123711765346653773251761251635002412 6410254565007065546756015043572302057573 4010547010044577303456773650454016441676 52645603000655246474332003041 1057521760004167050454306543214444676237 1110323251120106217732446025515645663300 5122223411465454201703211427513246445624 30356677543412533230060230330601 1400373125226245616434770021451434774715 0430652515175422234333765344626250514664 2652723355123643633615716343345655723373 56773023742177503076702013524172673 1065643260601413014144161647626313166 6646125563104544564702042267572604002367 7222071247202762236702640237554475163650 230542617172444002424037141774557402 76623 1004032013504641301465031022773733377 2312030141207564005615007277456377116231 5116355123767005745123620162301722500173 755126134777520656552342125576467672201 45745 15064242170161057713111265316713155370 0662271356244404074721040675344372405062 0404060603623075530602521050711015101113 460677710774225141620173656560126201213 35531

Appendix C

Generator Polynomials of

1k

t

511

19

119

511

10

127

1023 1023 1023 1023 1023 1023 1023 1023 1023 1023 1023 1023

1013 1003 993 983 973 963 953 943 933 923 913 903

1 2 3 4 5 6 7 8 9 10 11 12

1023

893

13

1023

883

14

1023

873

15

1023

863

16

1023

858

17

1023

848

18

1023

838

19

1023

828

20

1023

818

21

1023

808

22

I!/,

Primitive SCH Codes

U311

CGemHeirfil!foil' fcOJILDil;)'Iffim]nfail

1323204574570661164212403733264114023404 6117057401637365534154143137744473020704 476474132477654.0766245335511016032261140 7052420104522110303361617354750363145550 46713 1702172311027431733521206643077042605327 72520245744222"176':1-2034145450713500264.726 3477463246601133310064536566115023625306 7473677673467052234355271140607221533731 42701037 2011 4014167 12052210423 30135372217233 67441634100257771 155441273452021342255 321370747475547513070313 760744225715270200004506345 1323526661245521113217162255031 2023237633202230444160563331425623 7755535722250615754561135410204703015 1762347256322045077757166370232714216 7257 3040745710762554654627001731571615206417 2421 4773627447536521222606523723122700031460 4273713 1373673270027255300005046713741010077651 22140012433 3440272156360444433517306353401033533501 14020277143115 1322555466146551135464235057214741470461 7135735657240141 3542752065222573717143677007420570562622 0205536512744500263 5210617676532176677756466752642074636403 1144034344342361121165 1213222404544636466003710275014422027057 75313007201145723077423013 2214325742216310053650417376464144271657 51274313300617216324271557407 5552750420617757113515375401021454054142 43020552307423532731034607546117

1238

Appendix C

Generator Polynomials of Binary Primitive BCH Codes

k

t

1023

798

23

1023

788

24

1023

778

25

1023

768

26

1023

758

27

1023

748

28

1023

738

29

1023

728

30

1023

718

31

1023

708

34

1023

698

35

1023

688

36

1023

678

37

1023

668

38

1023

658

39

n

Giellllemfort )l_l)
1652724250022427704512246362640211603371 646303263710572663702515300240421537 2246066255654012757523064141151344242436 432467416546402446171130753671556534665 7403630677220307364435256070064531622417 2156747445532621743456011004302553363 21161 1420057410715701076671164654273645645222 551071265714762042013666351251553254760 6604067 3544663745163634242616250415043615446630 1242677700521404233564674545716503565046 015304675 5160035350014031172374207074650122660340 6725265542413700106041273652611227613754 613777751617 1761052465717574451310664156407472743161 2025501044536741540070065741445436720677 4215713707625277 2705040500317205454745436524651543310405 0017465005222512321066476651072114503137 3602071775011137127 6421001547326360050675717264533767251455 4755764143772111200712325604323701627400 3637473623253276573051 1221657751474373730117767342335416045271 4315315367757451645726466677117331525450 50454154042571133265734145 2234207617247627652572131446233344733763 6432733672331007455154713117134375547477 57210324464635077421557061245 6062331567761473445053533264026122102305 3201440006656062450307555544233405300271 40456114002742376254456531124177 1173340210112354267372717132007746772117 2430242075060060272435267356231765714270 070343261342147115315606445423035675 2726674363712255027500744177372151741454 5645433073762151476755130543142214506356 405532675477541702642035025443744706413 4526553271372400673644374714763070706341 1031171237616540024630303133511353737123 3160732343127163117047027565041001443 54105

C

117!

Genernto1·

k

of

Primitive SCH Codes

123~

cG·eiJillsir2llUIDit

1023

648

41

1023

638

42

1023

628

43

1023

618

44

1023

608

45

1023

598

46

1023

588

47

1023

578

49

1023

573

50

1023

563

51

1023

553

52

1053012656042512406741405367113520216337 333515136144505130172221.5370653734053400 0442526605347576637101417411054700433515 223273 3760125774555145676316166607000752402515 L!-40564137207'7333177067640613307077706604 566651067307226054-6441216265142152656464 533676375 6421516056355057002552057134724545361312 235702701641701222010201274246457 5330112 6724317410001044430017517040452131500121 350470437063 1436767275404102542246475266313531611600 5027177636020644031621570413456443331322 7260670220051221512570532647243752525477 5754664246713353 3205632662501303326372612360531506221660 1642671653635152412144657627370612401323 5612466273254637300536451672472606153046 2161570470103566541 5175076L74225613532026031146621612144714 3656373757601671144563673511345562033377 2363115242327614411151057011653316234767 24207144-06612507752125 1425717410513115113640172157603722701046 3011170047424744303123121316566534441253 56022275674253134547633Jl477014361615330 04527036262433614645152063 3177430361201520403652734206337256777357 1514665321603455147706422374540553332214 5746177477620442053330430305772517145376 36407425537275744133016141137 1355677017526536535146475314700031707456 3026607614213012666657500272520767133533 7304265646727506154132227032714337707023 2341637207202520573521417420615 277236150442,B216766531i!-1026604362002113 2553454106501612101536430475161545700052 0475027073570114161614606247142254455044 6255557776515151167236264235327161 6042334623077055525431324656261417515014 2274125133435506177502357332260547747255 3162040767437444775076764454714640345671 0027561254427647334263053360460303115

U40

Appendix C

Generator Polynomials of Binary Primitive SCH Codes

k

t

1023

543

53

1023

533

54

1023

523

55

1023

513

57

1023

503

58

1023

493

59

1023

483

60

1023

473

61

1023

463

62

n

Gel!llemfo1r

J!)OlymimmiaH

1055532301605012610131175414035550021456 2763276437435575056231126247124212102427 6704204717425615057004044214740647640450 06516621136261664711665773405527205053143 3011252271024003601256355614410110631241 4476043760457564721111127466077665562665 4217667430146671642613617516247055413742 6636741420630626224221121140152405654631 6753 6173100556046353561760536040447255721247 4310077565636320561601165471476057132554 2322473307203230122443324706157714547610 2204305664132377402106560521763116372354 2140361 1442056607323310761242261470600262276017 5101364127621557726777406532331057077546 1527721765575312220547404020757305657546 5457253141504105143444175702256556213675 47711533103 3526111340256374262000600572256610352600 0577617220337007505520131512037473367771 3400541207320422505347425362014026650010 0004257043231653414626355376624351306710 02311565250731 7535405755220775623403553444573506243560 7256600605234131432506166510202242405073 3356624720337475566117114661531204767172 3111426350630167625357243023251630010175 45226050161245001 1010227406455200076236416372024031175567 5534053006400654636246221506521030256417 4635764603621630460402150663270555030262 2471552611215057442745721707226536376112 325071626447667724133 3275255734777512716766130324236300561602 3266541073200606432722454332243070233312 1056616662644203305305353270024721414270 6252777265242372623345602661643601551362 360114712022071563522541 4257710770354177520700120310175131741452 2646316040516304114052431351313706134426 2044226555241044230276332312417700133134 5011373217527767452370777645631335625621 275401071525545617027552161

Appendix C

Generator Polynomials of Binary Primitive BCH Codes

k

fl

1023

453

63

1023

443

73

1023

433

74

1023

423

75

1023

413

77

1023

403

78

1023

393

79

1023

383

82

PE

Uii\1

Gerrneir~tll[))Ir [lll[))Ilyrrnl[))Il1Dlll!llll

1531237560135257466053045644313751237706 0514513604361025341247434430553661067700 1052325072756200027154257505607050746074 2347574503151051424660227041423302376426 0257036602570516146760636153261 3051462775160646577510534745450470242247 435306056026620343604133627 6663363011 il.62 7247152315630145650707511617214040605320 4277716062400570267044607301501125151777 3374535517744726746624051117614557 6551077013564477321042777730023704151601 7521127762157242770232322533724016627637 1251547755210341250226270224512023346531 5173725652335621405465526246704034012300 7073271127075457754155273245775745013 1371403545740325650734320662641445605572 0647077714165030001606304253676672714102 5356347426042441521146714214012236710237 4164612444130330470102450320103726066270 46047250466074142533050300015153772074227 3130514551013371502077510305732557321401 1760033161356760432560144747314167124256 5376173247642602517357010626427257776456 7734266446545120037332774475061657524550 1423614017265745456752500432104234734 3107567 6530503067625562124141564675424734756257 5145101333262333026444047152033731026412 7462440055716211016277744037230663055316 4172765324526335350412724676631061133755 7637416161530226355450072447426761051420 0676657 1653506701401422711244424506213255341202 7657471014506751046044362763202645334655 6625534505264745765143400727723477524642 3717565120014071373273475432524117405663 5606472676113552676751152426134176727371 57011323251 2101556350316274636721774002521346374755 0416046710713235213260000173163311010751 1524536177002664305516503446336030163303 2543716222246044772213606523671240711261 7657615110133267371365102061275103775727 57105036743057

1242

Appendix C

Generator Polynomials of Binary Primitive BCH Codes

k

t

Generntm poBynomiaH

1023

378

83

1023

368

85

1023

358

86

1023

348

87

1023

338

89

1023

328

90

1023

318

91

1271727232475673032645416341027644322054 4742747152233707376724600063056046415216 0615441501471172331343574553307577366042 6310001651125246345406555466665316732504 0134511004147527430357700216203600436576 3562314467231267 2667110244306717554116670535772352255702 0775135077721751514126771356024621053205 1314022567463732172361241171201412026160 3653333750471661055152633432534522430740 0302522242423267144130672542411721301423 3216317412603200011 7435121372564560416164423405512531131273 4223333111503244154771560003234404757442 6171255246153400574361641566444466317375 4160134125055131104230756443674102100730 5155725322757546453441553326405634665130 7314476261042544230245 1152150570436510265766676136141303474274 4526012535312616667574523032536722700050 0625207035201332737114107013243424325560 6164445352775311630144736763442471124475 7750150011616262637717177075102213217170 43446340305445612706534703 3435404744265677602320351555134172304633 0127646153113200255460143043276470736246 6125401015341073237073746771541102307733 4025666153620557042677053703237045526065 4431637224726547347532253255127643611341 66201344577712444057365034127 4376075566465464447725523310167055725051 3364752460175061537135014226525515352756 1554472635775452747645066645517076050542 7007214641140377770232661301737600047265 0644621575075623315742537655651745704130 72547436036455673176766776021557 1706570374315130744666215341745357673544 6506517074057307606757741716571770230670 4666124321175656663111343514112511526132 5767234612127537336563140520412006236673 0652025365545623610266465373576416357076 757337145232060114113270376256424571

n

Appendix C

Generator

k

t

1023

308

93

1023

298

94

1023

288

95

1023

278

102

1023

268

103

1023

258

106

1lP,

of

Primitive BCH Codes

12di].3J

Geillleir'@ilm' n»([J)Ilym:ri:imr.11n21Il

26421203200477176222311406331673263407 4046546227174626344411404424360521267164 3105715450254066734621777407051773363403 211544225553064525153700121577654-5577154 7627474632455711524715735601204030300362 516113715713635367435754766272556123 75247 7607316125761433642526304450363545626472 5611714372152073305634315715022554055231 6076751112611277717145131135317527677046 6732556644427514047164046110636365565566 4550340671546270704215331125127241135267 1670301124644115710644070316271466331 10673 1701604066633416774726444725261650551067 2124410746002141416267465201122102A55167 6755376270304423013326176347761574061225 2161051122456127543465052666631743005425 2266655317653460754531732325667620546756 4036613660503377260207743030117536367621 203555 2561112413204004672512333427015121540424 3237123071101755037216065252775302223547 4065127152454745331164320065161070571534 437124732460771702704.Jt-[4626140302447 6234 2751113123413544231643217420335257714150 0236421250632332705126052420172332450755 310424635 5654035737756644460334414133030111146310 0146152621621754746144205012055330527077 5303715417371641000727253627610744441051 5703446316265332602512753027230261252257 0230566667675120633312503010076171344377 7330061102755376503137425117533736240637 177561306317 1472716560654553504476114427330200150115 1173472111661146472710536457264074616730 6212707640513421202420267426242134553637 0427517704053463140321453024446377474011 2076150643076734714642657240376712557330 2203556104577153413672763550007757247113 0110646647103743

U44

Appendix C

Generator Polynomials of Binary Prirnitive BCH Codes

k

t

1023

248

107

1023

238

109

1023

228

110

1023

218

111

1023

208

115

1023

203

117

n

GemHell'alfoll' RJ)Ilyl!llrnnmnaH

3461477710376543241414375231636516574724 5065316230137456120461323574541525726041 2556573562616526111127656576320525763510 5644024312145652627132756063506201454560 0503657215441172100402476306531430623526 5426422633275763410271325014165632012203 2414745504624453615 6107055505572007262714470615665443601727 4773321535432655566365546074301200532116 2555655214623411066175714545116221410540 1214402533154265163574524537271755667277 4742133671045121217224375055532423613465 2547057726572740725403741750507323711445 5226722253133616554655 1645505272602350646313312172433135453243 4317660426124673504670142363510702632530 5465440350706502341177157432562253614237 0371356404544253565207222015007240167503 4540743040342334543505710424074123577104 0704601774071102570072722602051723521347 64771653006051570762272065 2532727377067146456122304005405563345245 7027263257630231533014541245300437451413 4422124564661526271012077252745371352727 5724274535530277606021744662377454417475 5417200576157107713300760220650710444522 5676334024215714747554147237530037305312 55325367612674645741101010215 7662425544271246134245033532655061777064 3113564744501532047502013441735363751643 4611742014356270417543440327560724415144 7613713075166362462555105072064606436604 7372533427761377771313714611060116225324 7273756021533523511473763263374430301043 41312067517077073657202575266431 2600721761002761741110660767163553757530 6525157537267407357336666131163647123236 6275645264765562251403631501114054010135 3224017277322017716776500255740570067411 7120457710072403775723015475413276153570 1153221436207760470741233214262705415704 6022354153273653525227756023726523

C

Genera'i:or

k

t

1023

193

118

1023

183

119

1023

173

122

1023

163

123

1023

153

125

1023

143

126

/!ii

Pdrnitive BCH Codes

12t!il5

Ge:irnert£ii1(I})rr

7053315727030267005076533662446626601361 20617771711501237035176023'23634330163122 0440504733611235242564514652123365275225 524A13410C153G7Ti72243!L1-4.t.1,L!.0563401766701 5257142421560325755421566155040177772450 567645207166] 107L!.7 l 162522137LJ.06175403173 3166505125521364032352227514 l L<S713341 1347473474775270044720557004055543253641 5414261074530641366160044747074270440647 6351726463337066164237425741310003010151 345337423247377204620333"1555337543714510 7054072166471561270i!.50770527154501716302 2332610013551710764724143151161146625775 6310407013234622100223? t'.lQ7201765701 721245 2361302433102534366303174067425000312611 5407714446113241270367016260545512127107 41460774212445352?5152145570435377754120 5555476532041352234100662235537577223261 44575535355404.03471102147 l 16764306331760 6032315053403217330370624061030766417314 6401136667040305764271011336376561455315 7553 411313103'.~2520143LJ2001147350344201677441 732521373766353065562456.';7 54314734523735 5425411305340760252512234263304567336743 677641661577462771756517576/J.750357722307 2217561502536205126601021474136200301335 545710314164354255715114L1254-001725320301 7436544220547403671156214574372425103751 6460607 1473037520540655305664534611401067646510 6566466240750272645247101532717001327355 1255250401446241777221035]54710077244137 3023726106475461742104112674543042527464 7451775407521677507671036342125536233317 6644217425306265727330713311536600712505 243204434170655144700M26647662422226300 32444750511 2705672534137533656134605524750654047625 4275423257512563102361416067617730642771 1460013664347574610040617626371543266057

1246

n

Appendix C

k

Generator Polynomials of Binary Primitive BCH Codes

t

1023

133

127

1023

123

170

1023

121

171

1023

111

173

1023

101

175

GemHerniQ}Ir J!»Oilyllllmll1lfall

3554365713174265772110167734047217705111 3733747446147746503062302477575037740315 1165171334262046730256770526522266602575 2346240531211431754131515171115505251076 23337477461525 4617241122502533137567475531722511566715 7425701644602354172410473477746724542114 5544443464632270001337721543511324751670 2616154503645720574203625447407506343636 6627145456220533275233402326143257055011 6320123731574052034715703301044271032476 7607251200167144164775070153251671274256 43765453321156665 1606122475314452440364114065506377666426 5343610702760377310024522705343600423320 2424333105146144027327126143431675314002 3036402653162631026523102264715527745315 0374162211376360615537273444653255610566 1614203104347443540633723112046663543521 1253420231107716033636204725255752061446 310672727204716406065 5222675662147725741314744210323377001541 2253272514721374170157274532253203570061 5555001733462474144045642452515062144017 1131414121534317141271717416540247672143 1364537676373322240234442776120501273102 5245611735266752042303471766360032052267 6522561717727552103333636452501626224763 172044444636551422213 1522057216652345524151261131756011401100 2417457353411556263253430644235445523722 3141414362643623756035136777741661732620 2500037150531207635215567034127213515066 4410720607604354463431406515431477631373 2040303636642717670427347615762362355036 7600521462074250472554267242343544575670 3446634044261764644465517 3545472772647733506027327007274644770353 7076474016112034433402242207212433412503 4212535626065370]42605376134717357361465 6417762011214767217653655554653572111747 4551152615662076355010304237412255115177 6272636505203446412116131004765425047506

C in

Generator Polynomials of Binary Primitive BCH Codes

k

1:247

tGeillle1t£1im1 fil)i[])IlymD!llllln£1Il

1023

91

181

1023

86

183

1023

76

187

1023

66

189

1023

56

191

2051240151667124466614627030314263307135 5576456772101677262472416421 4055177762504121412577610205332003777645 6673574005261545600620674271417162613012 15062561052425574236407643364-01204101741 7126672067376365721700727456241223231302 2756306103623641555001215465766126L!.J0341 7306703214450577041500501573212167321536 3434543304701503707164511142547260441612 1735616377403703643762331023117 2264761771543214471675702410753621637726 1160424142731011126327103065573247413075 4432257227721764404464742501644540205701 3267260210053106372523361731350152677206 0076334324656464215710544105353537351675 3364012464542355412231261607767455076427 5143702025133630600744625136760211406363 453422170361560266523166234073223 5504413226010767557023243061737064215533 4421615324723723124722403164751617036710 2332107021677452321433245440364276246075 0627166653450647114065506704152554300353 0021311306574650326702510673440306147402 0127524575753074316742263020463722341655 5043324233205722150256144415372750304032 050275143511316413151005306045676027 1335505434441741406157526754573170243027 3424173625471123363551344271423420240635 4431211746027242072506552240651527643442 5225735404126141635066623161414334637002 3253213327457376454172114672756045050327 1457414742355472656525107236667037135445 6044605352522350777052251520016127100460 0523064264130454576437037741704275570603 3533660424156746206407772444526075003401 3327044430111755764054213671261403637721 7365150066452035503620722031617013704506 1451154164441245647372600477550413342313 4332421423334727666016525177347306160053 4367723121360245640211624352467665254225 6204245534234357413620302403642764362211 35516672370421454305415541747131253251 73331

1248

Appendix C

Generator Polynomials of Binary Primitive SCH Codes

n

k

t

1023

46

219

1023

36

223

1023

26

239

1023

16

247

1023

11

255

Gel!1lernfoir poByll1lomiial

63015416256656230205213566027445455171175 52653111473567026514175574471346103313706 37137663547260635501352643104443676250677 3370422474424106014660455576033053653630 31370740762367221675415372060440327703604 22320146000111212425113737126570553131746 45762574576401050005016160506604517273022 6274076512210521522004216362565324376075 1152711551526005136004274504217435002475 0437221435664073005171237220151140064707 5441034055747331016655123301566414331316 3711704233561771130764134237244667223150 7613166546645026325127605716064161523643 7407144501652522375660250700303477777350 5457120771071520002217212553614046136364 3211657443425105061615376162061404232341 2027502143 3476655146625052723141431762032154723323 2124522103643311017642323320202416451725 3130645425737600246350051466006303723670 4734745250307710106433015376231566674173 2473210524736630110637274120466644500552 3530736347265641711514677043332674476035 7766776101312115727255145732774036010471 3655061034476144410065213573347702152554 6455751231633 6036262767731160101141105063701021363565 3762274052211432564657262162121174115766 0750767504220703662612124577754243717505 4060630501002657505104156311506734744442 5411743547305156755340240153001037104363 0513040170535057376012745423660541522550 5633323652623572617766360711322000255022 0251544067462673641704456634322534227212 4736702723114761 3435423242053413257500125205705563224605 1630065263153013075623353102124314210612 6137022171665514567226426351765550103471 0474154323571003071265736644050721570112 5641143017433246415076511474510565005614 4327137466737311303124750456366004341761 1165663371330203524572775110303560116122 5714476377111212075267515476733757766662 624063406704036177

ACK (positive acknowledgment), 15 ACSS (add-compare-select-subtract) process, 727 additional check-sum, 297 additive group, 27 -8 additive identity, 32 additive white Gaussian noise. See AWGN acid operation, 518 AEWE (average Euclidean weight enumerator), 959 affine permutation, 283 AIOWEF (average input output weight enumerating function), 992-8 algebra binary field arithmetic, 37-42 decodings, 395 fields, 31-7 Galois field GF (2"') arithmetic computations, 54-5 basic properties of, 47-54 construction of, 42-7 groups, 25-31 matrices, 61-3 vector spaces, 55-61 AM (amplitude modulation), 953-4 ANSI (American National Standards Institute) CRC code, 183 APP (a posteriori probability) decoding algorithm, 440 APPL-values, 564 ARQ (automatic-repeat-request), 14-15 concatenated coded modulation hybrid system codes, 1192-3 described, 1192 FEC subsystem and retransmission, 1193-5 specific system, 1195- 7 continuous, 15 described, 1156-63 half-rate invertible code class, 1178-81 hybrid systems, 1174-8, 1190-2

selective-repeat with finite receiver buffer 1167-9 blocked-state described, 1163-4 with mixed retransmission modes, 1171-4 normal-state operation, 1166-7 receiver's operation and error recovery, 1165-6 throughput efficiency, 1170-1 transmission and retransmission, 1164-5 type-U hybrid selective-repeat with finite receiver buffer blocked state receiver's operation, l 183-6 described, 1181 normal state receiver's operation, 1183 reliability, 1189-90 throughput efficiency, 1186-9 transmission and retransmission, 1181-3 associative binary operation, 25 associative polynomials, 40 asymptotic coding gain, 17, 532, 958 automatic repeat request. See ARQ average bit multiplicities, 787, 790 average codeword multiplicities, 787, 790 average Euclidean weight enumerator. See AEWE average input output weight enumerating function. See AIOWEF A WEF ( average weight enumerating function), 992 A WGN (additive white Gaussian noise), 5-6, 523-4 channel with finite output quantization, 10-11 Shannon limit, 20 backsearch limiting, 633 backward index. See BWX backward metric, 566

1250

Index

backward received sequences, 715 backward recursion, 566, 714-15 Bahl, Cocke, Jelinek, and Raviv algorithm. See BCJR algorithm bandwidth, 9-10, 21 bandwidth expansion, 10, 952 base code, 942 base matrix, 908 basis, vector space, 59 baud rate. See symbol transmission rate BCH (Bose, Chaudhuri, and Hocquengham) bound,205 BCH (Bose, Chaudhuri, and Hocquengham) codes, 236-7 BCJR (Bahl, Cocke, Jelinek, and Raviv) algorithm, 515, 563-82, 711 BCM (block coded modulation), 23 concatenated, 1081-8 distance concepts, 1063-4 multilevel, 1064-74, 1090-100 multistage decoding of multilevel, 1075-81 product coding technique with, 1088-90 belief propagation algorithm. See BP algorithm BER (bit-error rate), 15-16, 114,537 Berlekamp algorithm, 241-8 Berlekamp-Preparata burst -error-correcting convolutional codes, 1128-34 Berlekamp's algorithm, 194, 209, 242-3, 247 BF (bit-flipping) decoding, 872-4 Bhattacharyya parameter, 532 binary BCH (Bose, Chaudhuri, and Hocquengham) codes decoding, 205-9 described, 194 error correction computing error-location numbers and error correction, 225-7 finding error-location polynomial, 225 numbers, finding, 215-17 syndrome computations, 224-5 Galois field arithmetic, implementing, 217-24 Goppa codes, 230 iterative algorithm for finding error-location polynomial, 209-12 primitive, 194-205 simplified iterative algorithm for finding error-location polynomial, 212-15 weight distribution and error detection, 227-30 binary concatenation, 761-3

binary field, 33-5 arithmetic, 37-42 linear block codes from (See iinear block codes) binary operation, algebraic, 25 binary-phase-shift-keying. See BPSK binary symmetric channel. See BSC binary symmetric erasure channel. See BSEC bipartite graph, 856, 857 bit-error probability, 529-31, 807-13, 880-1 bit-error rate. See BER bit-flipping decoding. See BF decoding bit-level for binary linear block codes, 342-51 structural properties of, 354-60 bit metric, 517, 610-12 bit ordering. See optimum permutation BLER (block-error rate), 15-16 block coded modulation. See BCM block codes, 3, 453. See also linear block codes blocked state, 1165, 1167-9 block-error rate. See BLER block interleaving, 132 bootstrapping effect, 782 Bose, Chaudhuri, and Hocquengham. See BCH bound; BCH codes bounded distance decoding algorithm, 399 BP (belief propagation) algorithm, 445-7 BPSK (binary-phase-shift-keying), 5 coding gain, 16 Shannon limit, 20 branch parallel and composite, 693 trellis diagram, 339,370 branch metric, 567 branch probability, 718 BSC (binary symmetric channel), 8, 90-3, 522 BSEC (binary symmetric erasure channel), 9 buffer overflow, 1165 burst-error channels, 13 burst-error-correcting codes, 13 cyclic codes, decoding, 1105-7 defined, 1104-5 phased Burton codes, 1118-19 described, 1117 random and/or single or multiple concatenated, 1121 described, 1119-20 modified fire codes for simultaneous correction of burst and random, 1121-4

lnde)( from RS codes, 1120-1 single fire codes, 1107-11 interleaving, 1116-17 product construction, 1117-18 short efficient, 1114-16 burst-error-correcting convolutional codes Berlekamp-Preparata, 1128-34 capability, bounds on, 1127-8 described, 1127 interleaved, 1139-42 ctH:- ivrn,,,ev, 1135-9 with random errors burst-finding, 1 lL!-6-9 burst-trapping, 1149-52 described, 1H2 diffuse, 1142-6 burst errors encl-around, 154-5 random error channels with, 14 bursts, 132 Burton codes, 1118-19 BWX (backward index), 1168-9 Cartesian product, 382-90 catastrophic encoder, 484-5, 487-9, 509-10, 542 CCITT X-25 (Consultative Committee for International Telegraphy and Telephony, Recomrnendation 183 CE (cross-entropy), 840-2 CER (constellation expansion ratio), 1053 channel, 2 channel decocleL 2 channel encoder, 1 channel error sequence, 646 channel information matrix, 878 channel reliability factor, 560, 781 characteristic of the field, 35 Chase decoding algorithms, 408-9, 417 check-sums, 107-13, 273-4, 297,647 check-sum vertices, 857 circular trellis, 590-1 closest coset decoding, 1076 code-bit vertices, 857 coded communication system, 15-21 coded modulation, 21-3, 952 code polynomial, 137-9 code rate, 3, 4 codeword, 1, 3, 66, 455-6 codeword conditional weight enumerating function. See CWEF

T25i'i

codevvmcl

enumerating function. See IOvVEF codeword input recluadancy weight enumerating function. See IRV\/EF codeword enumerating function. See WEF channel, 3 coding gain, 15-18 modulation and, 5-10 threshold, 16 --17 colurnn distance fun.;:tion, S07

cc,lun1n 853 column splitting factm, 912-13 commutative group. 25- 7 commutative polynomials over cOJT1pare 518

39

compound channels, 14 computational cutoff rate of the channel, 629 distribution, 626 concatenaied coding binary concalenalion, 761-3 with convolutional inner codes, 760-1 described, 739 LDPC codes with turbo codes, 944-5 multilevel, 7L!J-8 single-level, 739-43 soft-decision decoding of, 748-50 conditional error probability of the decoder, 11 conjugates, 1.!.S, 53--1.J connecting trellis, 933

constellation expansion factor. 956 constellation ratio. See CER construction field, 942 Consultative Committee for International Telegraphy and Telephony, Recommendation X-25. See CCITTX-25 continuous ARQ, 15 controller canonical form, 459 convolutional codes block codes versus. 453 described, 3, 453-4 distance properties, 506-10 encoding, 454-86 Fano sequential decoding, 620-6 majority-logic decoding described, 645-69 performance characteristics, 670- 7 self-orthogonal and orthogonalizable codes, 677-85 sequential decoding code construction, 640-5

1252

Index

performance characteristics, 626-40 structural properties, 486-506 suboptimum decoding, 605-6 trellis structure, decoding based on BCJR algorithm, 563-82 code construction for use with Viterbi decoding, 538-44 described, 515 performance bounds, 525-37 punctured and tail-biting, 582-98 SOY A, 558-63 Viterbi algorithm, 516-25, 544-58 ZJ sequential decoding, 606-20 convolutional inner codes, 760-1 correctable error patterns, 84 correct path, 526 correlation, 397 correlation discrepancy. 398 coset code, 744-5 coset lender, 119-20 coset representative space, 120 cosets and cosel leaders, 83- 7 cost function f(s;), 426 cost function list (CFL), 428--9 CPMT (composite path metric table), 696-700 optimum sectionalization. 703-4 RMLD algorithm, 701-3 CRC (cyclic redundancy check) codes, 182-3 cross-entropy. See CE current input information bit, 348-9, 352 current time-out vector, 1164-5 CWEF (codeword conditional weight enumerating function), 495-6, 500 cyclic codes decoding, 155-62, 1105-7 described, 136-43 encoding, 146-50 error-trapping decoding, 166-73 generator and parity-check matrices, 143-6 Hamming, 162-6 improved error-trapping decoding, 173-5 product codes, 184-5 quasi-cyclic codes, J 85-8 shortened, 179-83 syndrome computation and error detection, 150-5 (23, 12) Golay code, J 75-9 cyclic redundancy check codes_ See CRC codes

data array, 1194-5 data loss. See erasure data rate. See information transmission rate DC (disjoint-crossing) constraint, 930 decoder memory, 545 decoding binary BCH codes, 205-9 cyclic codes, 155-62 nonbinary BCH and RS codes, 241-8 decoding error, 2, 11, 546-7 decoding rule, 11 decomposable codes, soft-decision decoding oL 748-50 decomposition, code, 124 decomposition into component codes described, 750-4 iterative multistage MLD algorithm, 754-60 degree distribution, 923 degree of a polynomiaL 38-9 demodulator, 2 designed distance, 203-4 detectable error patterns, 79 difference equation, 148 difference-set codes, 292-6 digital source, 2--3 digital sink, 3 dimension, vector space, 59 direct product, J29 direct-sum, 120 discrepancy p, 210 discrete memoryless channel. See DMC disjoint-crossing constraint. See DC constraint distance profile, 640 distance properties convolutional codes, 506-10 turbo coding, 783-807 distinct nonzero polynomials, 45 distributive polynomials over CF (2), 40 dividing part of the tree, 606 DMC (discrete memoryless channel), 7-8, 517 dot product, 60 double-error-correcting linear block codes, 102-5 double-error event enumerators, 792 doubly extended RS code, 240 DTI (doubly transitive invariant) property, 284 dual code of C, 70-1 111121 ..:p9rP r.f c;;:, hf\_ 1

dummy information bit, 348, 350, 352

Index edge trellis diagram, 370 effective decoding length, 650 effective minimum free distance, 816 elementary symmetric functions, 208 encoded sequence, 1 encoder state, 486 encoding convolutional codes, 454-86 cyclic codes, 146-50 energy per information bit, 532 erasure probability, 626, 629-30 erasures correcting with BCH codes, 217 nonbinary BCH and RS codes, 263-9 error coefficient, 398 error control strategies, 14-15 error correction binary BCH codes computing error-location numbers and error correction, 225-7 finding error-location polynomial, 225 syndrome computations, 224-5 linear block codes, 78-82 error floor. 772 error-location polynomial, 208 iterative algorithm for finding, 209-12, 225 simplified iterative algorithm for finding, 212-15 error positions, 561 error probability, 15 error probability amplification factor, 542 error probability of the decoder, 11 error propagation, 109, 748 errors correcting with binary BCH codes, 217 detecting, 72-6 nonbinary BCH and RS codes, 263-9 types of, 13-14 error-trapping decoding cyclic codes, 166-73 improved, 173-5 error-value evaluator, 247 ESE (extend-select-eliminate) algorithm, 930-2 estimated information sequence, 2 Euclidean algorithm, 40, 248-55 Euclidean distance, 22 Euclidean geometry LDPC codes, 860-6 majority-logic decoding, 304-8, 309-19 event-error probability, 529, 957 even-weight vectors, 288

'l253l

EXIT chart (extrinsic information transfer), 822-6 exponent, 236 extend-select-eliminate algorithm. See ESE algorithm extension field, 34 extrinsic information transfer chart. See EXIT chart Fano algorithm, 605-6, 620-6, 644 Fano metric, 609-10 FAST algorithm. 644 FEC (forward error correction), 14 ARQ strategies, 1193-5 ARQ versus, 15 fields, 31- 7 final node, 342 final state, 340 finite field, 32 finite geometry error performance, 880-5 LDPC codes, 9i7-20 finite graph, 856 finite-state machine model, 338-42 Fire codes, 1107-14, 1121-4 first constituent code, 769 first event error, 526-7 forward-backward algorithms, 571 forward error correction. See FEC forward index. See FWX forward metric, 566 forward path, 490-2 forward received sequences, 715 forward recursion, 566, 714 fractional rate loss, 486 frames, 619 frequency-domain decoding, 255-62 future subcodes of C, 356 FWX (forward index), 1164, 1166-7 Gallager codes, constructing, 902-6 Gallager function, 626 Galois fields, 34-5 arithmetic, 37-42, 54-5 basic properties of, 47-54 binary BCH codes, 217-24 construction of, 42-7 finite geometries over, 917-20 primitive polynomials, constructing with, 198-200 Gaussian approximation, 822-6 generalized Chase and GMD, 409-12 generalized minimum distance algorithm. See GMD algorithm

1254

Index

generalized stack algorithm, 640 generator, 68 generator matrices, 66, 143-6, 456 generator polynomials, 140,461 generatorsequences,455 geometric construction, LDPC codes, 858-60 geometric construction, low-density check parity-codes. See LDPC codes geometric uniformity, 979 girth, of graph, 856 GMD (generalized minimum distance) algorithm, 407 -8 go-back-N ARQ, 15 Golay code (24, 12), 99, 101 Golay code (23, 12), 125, 175-9 Goppa codes, 230 Gray mapping, 965-9 ground field of GF (2 111 ), 46 groups, 25-31 guard spaces, 1127-8 Hamming cyclic codes, 162-6 linear block codes, 100-2 Hamming codes, 99, 125 converting to SEC-DED codes, 102-3 decoding circuit, 181-2 error-trapping decoding, 172 L-step orthogonalizable, 298-9 majority-logic decodable codes, 291 Hamming distance, 12,116,522 Hamming weight and distance, 76-8, 80 hard-decision decoding, 9,395 h-error event enumerator, 798 Hsio's codes, 103 hybrid ARQ scheme, 1174-8 IBM-SDLC (IBM Synchronous Data Link Control), 183 IDBP (iterative decoding based on belief propagation), 858, 875-80 identity element, 25 identity matrix, 63 IEC TC57, 183 IEEE Standard 802-3, 83 incidence vector, 309, 930 information codeword, 1181 information error sequence, 646 information sequence, 1 information source, 1 information transmission rate, 10 initial node, 342 initial seed codeword, 427 inner codes, 740

inner product, 60 input queue, 1164 interleaver gain, 775,796,801,803, 1127 interleaving depth, linear block codes, 131-2 I-burst-error-correcting code constructed by, 1116-17 inversion process, 1180 IOWEF (codeword input-output weight enumerating function), 493-5, 497-500 irreducible polynomial, 41 IRWEF (codeword input redundancy weight enumerating function), 500, 503-4, 506 isometry, 963 iterative algorithm for finding error-location polynomial, 209-12 iterative belief-propagation-based decoding, 444-7 iterative decoding, 130, 826-44 iterative decoding based on belief propagation. See IDBP iterative MAP-based decoding, 442-4 iterative multistage MLD algorithm, 754-60 I wadare-Massey burst-error-correcting convolutional code, 1135-9 jth spectral component of V(X), 255

key equation, BCH decoding, 249 KNIH algorithm, 417-19 Kronecker product, 114-15 label, trellis diagram branch, 339 Lagrange's theorem, 31 I-burst-error-correcting code LCM (least common multiple), 195 LDPC (low-density parity-check) codes breaking cycles in Tmmer gr"phs, 892-8 concatenations with turbo codes, 944-5 construction based on balanced incomplete block designs, 935-8 construction based on finite geometries over GF(p'), 917-20 construction based on shortened RS codes with two information symbols, 938-44 construction by column and row splitting, 885-92 decoding BF decoding, 872-3 described, 871-2 error performance of finite-geometry, judging, 880-5

MLG and BF decoding, 873-4 MLG decoding, 872 SPA, 875-80 described, 851-5 Euclidean geometry, 860-6 Gallager codes, constructing, 902-6 geometric construction, 858-60 graph-theoretic, 929-35 irregular, 922-9 masked EG-Gallager codes, 906-12 projective geometry, 866-70 quasi-cyclic codes, constructing by circulant decomposition, 912-16 random, 920-2 shorted finite-geometry, 898-901 Tanner graphs for linear block codes, 855-8 least common multiple. See LCM likelihood ratio, 553 linear block codes. See also trellis diagrams cyclic codes decoding, 155-62 described, 136-43 encoding, 146-50 error-trapping decoding, 166-73 generator and parity-check matrices, 143-6 Hamming, 162-6 improved error-trapping decoding, 173-5 product codes, 184-5 quasi-cyclic codes, 185-8 shortened, 179-83 syndrome computation and error detection, 150-5 (23, 12) Golay code, 125-8, 175-9 described, 66- 72 error detecting and correcting, 78-82 Hamming, 100-2 interleaving depth, 131-2 minimum distance, 76-8 over BSC, probability of undetected error, 90-3 product coding technique, 128-31 reliability-based soft-decision decoding algorithms Chase, 408-9 described, 395-400 generalized Chase and GMD, 409.-12 GMD, 407-8 iterative belief-propagation-based decoding, 444-7 iterative MAP-based decoding, 442-4

_ _::(irnum likelihood decoding bai =====n iterative processing o± least eliable positions, 417-19 . ==c3sures and general decoding 1n,e-~ - - - - - - ~ chemes, 400-2 . ·c-t reliable and least reliable ba me-· -22-L! or~ :5red statistic decoding algoritl -----··· pr::. -:-~-====rity-first search decoding =================-:_lgorithm, 424-32 R: _ .SD, 419-22 su -===;_cient conditions on optimalit ,========ecoded codeword, 402- 7 sy.,..::Jrome-based ordered statistic ========-ecocling, 438-9 W}:======='.V, L!-13-17 we=· Jhted majority-logic decodin1 --=;9-42 . Reec:---------Ivi[uller, 105-19 single ________ 5rror-correcting and ------~=_ouble-error-detecting, 102-: single-------- -oarity-check, repetition, and ;H-dual, 94-5 = squar" __..eJg construction of, 119-25 stand -:cl array and syndrome deem -~~,-- _,·.,--.-.--':2-90 s y n d r - - - - - - - - ::-ne and error detection, 72:nation, 58 linear com-t.:====~c-= mz__________

---==---

linearity, 6f.·==== linearly de; 511dent, 58 linearly ind -~~ pendent, 59 linear sums 68 linear syste ~_'Jatic block code, 68-9 LLR (log-li ________ _<.elihood ratio) A VI/G ·J channel, 780-2 L-va!L. _______ 5, 559--60 MAP ~Jgorithm, 711-12, 722-4, 7: location nuc --ibers, 283 __::....,)cl function, 11 log-likeliho _____ log-likeliho ----- ·~ cl ratio. See LLR log-MAP aL gorithm (log-domain BCJR z ::_gorithm), 570, 572-3, 734 low-density :,arity-check codes. See LD --::ides lowest-degr ~e polynomial, BCH codes, ---"'-=--=--~--- ubtrellis low-weight subop·· _____ __::::: imum iterative decoding "°-------- }gorithm based on c o ~ , - - - - - - - - _-_ putational complexity, 708 '='ration of candidate coclew gen - - - - - - - - - Q5 / imp rovements, 710-11 mir~·_________ imum-weight trellis search,

=====

56

Index

optimality test and search region, 705-6 trellis representation, 704-11 trellis diagram, 380-2 ~P-reprocessing algorithms, 402 ;tep orthogonalizable, 297 -8 tcWilliams identity, 91-3, 227 jority-logic decoding, 107-9, 114, 605-6. See MLG decoding described, 273, 645-69 Euclidean geometry, 304-8 codes, 309-19 twofold, 319-25 multiple-step, 296-304 one-step, 273-82 difference-set codes, 292-6 maximum-length codes, 290-2 overall parity-check digit, 282-90 performance characteristics, 670-7 projective geometry and projective geometry codes, 325-32 self-orthogonal and orthogonalizable codes, 677 -85 ority-logic decoding rule, 647-8 ority-logic error-correcting capability, 275 .P (maximum a posteriori probability) decoding algorithm, 440 >ping by set partitioning, 22,953, 980-1 ry phase-shift-keying. See MPSK ked EG-Gallager codes, 906-12 ked matrix, 908-9 rices, 61-3 imum a posteriori probability decoding algorithm. See MAP decoding algorithm imum distance separable codes. See MDS codes imum-length codes, 290-2 imum likelihood decoder/decoding. See MLD imum likelihood path, 517-18, 560-1 mum logarithm (max-log) approximation, 726-8 -log-MAP algorithm, 570-1, 726-34 ; (maximum distance separable) codes, 238 5itt decoder, 156-9, 166, 185 ory, channel with, 9 oryless channels, 7, 524 ory order, 4, 459 age blocks, 66 .C, 517

MEWE (minimum Euclidean weight enumerator), 959 MFSE (minimum free squared Euclidean) distance, 957 minimal branch trellis diagram, 370 minimal polynomial, 49 minimum distance block code, 66 convolutional code, 508-9 linear block codes, 76-8 minimum distance region, 404 minimum Euclidean weight enumerator. See MEWE minimum free distance of a convolutional code, 506-7 minimum free Euclidean distance, 953 minimum free Hamming distance, 953,958 minimum free squared Euclidean distance. See MFSE distance minimum path weight, 380-1 minimum squared subsel distance. See MSSD minimum truncation length, 548-9 minimum-weight subtrellis of C, 382 mirror symmetry, 373-4 MLD (maximum likelihood decoder/decoding), 10-13, 338 coding gain, 16-18 for DMC, 517 hard outputs, 711 iterative processing of least reliable positions, 417-19 soft-decision. See soft-decision decoding MLG (majority-logic) decoding, 297-8, 872 modulation, coding and, 5-10 modulator, 2 modulo-2 addition and multiplication, 26, 33-4, 38 modulo-111 addition, 26-7 most reliable and least reliable bases, 422-4 MPSK (M-ary phase-shift-keying), 6-7, 955 MRIP-reprocessing decoding algorithm, 402 MSSD (minimum squared subset distance), 981-3 multilevel concatenated coding, 743-8 multiple stack algorithm, 636 multiple turbo code, 769 multiplicative group, 29 multiplicative identity, 32 NAK (negative acknowledgment), 15 NASA Tracking Data Relay Satellite System. SeeTDRSS natural mapping. 965-9 negative acknowledgment. See NAK

Index Newton's identities, 208-10, 213-14, 242 nodes, 856 node space, 425 noise disturbances, 2 AWGN,5-6 channel coding theorem, 12-13 channel encoder/decoder pair and. 3 nonbinary BCH (Bose, Chaudhuri, and Hocquengham) codes Berlekamp algorithm, decoding, 24 i-8 described, 234 errors and erasures, correcting. 263-9 Euclidean algorithm. decoding with, 248-55 frequency-domain decoding. 255-62 primitive binary, generalizing to. 236-7 nondecreasing function. 428 nonsystematic feedforward encoders. 454-64 nonsystematic recursive convolutional code. SeeNSRCC nontouching cycles, 490-1 normal state, 1165 normal-state operation, 1166-7 NSRCC (nonsystematic recursive convolutiona! code), 478-9 null space of S, 60-1 numbers, error location, 207 observer canonical form. 469-70 ODP (optimum distance profile), 640-1 one-level concatenation, 740 one-step majority-logic decoding. 275 optimality condition. 402 optimum distance profile. See ODP optimum permutation, 369- 70 optimum sectionalization. 693-4. 725--6, 732. 733 ordered differences, 292 ordered statistic decoding algorithm. See OSD algorithm origin node, 606 orthogonal check-sums, 647-9 orthogonalizable code, 650-1 orthogonal scalar products. 60 orthogonal vectors, 274-5 OSD ( ordered statistic decoding) algorithm. 432-8 outer codes, 740 overall constraint length v. 459 overall parity-check digit, 282-90 parallel bundle, 305, 906-7 parallel components. 700-1 parallel concatenation, 771

12Si7

parallel decomposition, 374-80 parallel processing, 554 parallel transition. 973-5 Pareto distribution, 627-8 digits, 68 parity-check equations. 69. 273 parity-sheclc matrices, 66. 70-1, 646 construction and state labeling based on, 360-6 of cyclic codes. 143-6 of SEC-DED codes, 103-4 sum, 273-4, 647 parity codeword. 1181. 1193 parity error sequence. 646 parity polynomial. 145. 1193 parity retransmission, 1177-8 parity 649-50 parity weight, 500 PAR (peak-to-average power ratio), 1053 past subcocle of C. 356 path branch and bit metrics. 517 composite, 695-6 correct versus incorrect, 526 in graph, 856 trellis diagram, 339-40 path gain, 488, 490 path memory, 545 peak-to-average power ratio. See PAR perfect codes, 99-10 l perfect difference set, 292 perfect simple difference set of order q, 292-3 PFS (priority-first search) decoding algorithm. 425-6 PM (phase modulation), 954 polynomials. 41-2, 135 positive acknowledgment, See ACK power spectral density. See PSD power-sum symmetric functions, 208 primitive codes, 194--205. 236-7 primitive element, 37, 53, 236 primitive polynomials, 41-2. 135 priority-first search decoding algorithm. See PFS decoding algorithm product codes BCM with, 1088-90 cyclic codes. 184-5 linear block codes, 128-31 product code, /-burst-error-correcting constructed by, 1117 -18 projective geometry, 325-32, 866-70 proper subgroup. 29 PSD (power spectral density). 6 punctured code of C. 357

'1258

index

punctured Viterbi algorithm, 582-98 puncturing, 554 QAM (quadrature amplitude modulation) signal sets, 954-5 q-ary linear block codes, 234-40 Q-function, 398 quadratic interleavers, 780 quadrature amplitude modulation signal sets. See QAM signal sets quasi-cyclic codes, 185-8, 912-16 quick-look-in encoders, 541-3 random coding, 12-13, 920-2 random-error channels, 13. See also compound channels random-error-correcting capability, 81-2 random-error correcting codes, 13 rational functions, 473 RCPC (rate-compatible punctured convolutional) codes, 584-5, 1192 reading unit. See demodulator received array, 130 received information sequence, 646 received parity sequence, 646 receiver quantization, 550-3 recursive convolutional code. See SRCC recursive MLD (maximum likelihood decoding algorithm), 691, 695- 704 reduced echelon form. See REF reduced list syndrome decoding algorithm. See RLSD algorithm Reed algorithm, 412 Reed-Muller codes. See RM codes Reed-Solomon codes, 237-40 REF (reduced echelon form), 400 reliability-based soft-decision decoding algorithms Chase, 408-9 described, 395-400 generalized Chase and GMD, 409-12 GMD, 407-8 iterative belief-propagation-based decoding, 444-7 iterative MAP-based decoding, 442-4 maximum likelihood decoding based on iterative processing of least reliable positions, 417-19 measures and general decoding schemes, 400-2 most reliable and least reliable bases, 422-4 ordered statistic decoding, 432-8 priority-first search algorithm, 424-32

RLSD, 419-22 sufficient conditions on optimc1lity of decoded codeword, 402-7 syndrome-based ordered statistic decoding, 438-9 WED,413-17 weighted majority-logic decoding, 439-42 reliability vector, 561 repetition linear block codes, 94-5 reverse trellis-oriented form, 373 RLSD (reduced list syndrome decoding) algorithm, 419-22 RM (Reed-Muller) codes, 99, 105-19 row splitting factor, 913-14 RS codes, 1120-1 scalar multiplication, 55 scalars, 55 second constituent code, 769 sectionalization, trellis diagram, 374-80 selective-repeat ARQ, 15, 1170-1 select operation, 518 self-dual linear block codes, 94-5 self-loop, graphing, 856 self-orthogonal code, 650, 677-85 semirandom interleavers, 822 sequential decoding, 605-6 code construction, 640-5 performance characteristics, 626-40 SE (squared Euclidean) distances, 958-9 Shannon limit, 18-21 Shannon product, 385-90 shell mapping. l 052 shifting constraint, 185 shortened finite-geometry LDPC (low-density parity-check) codes, 898-901 shortened cyclic codes, 179-83 side-loop, 381 signal flow graph. 490 signal points, inner and outer, 1051-2 signal set expansion. 953 single-error-correcting linear block codes, 102-5 single-error event enumerators, 792 single-level concatenated coding, 739-43 single-parity-check code. See SPC code SISO (soft-in/soft-out) decoding algorithm. 711,770 SNR threshold, 824-6 soft-decision coding algorithms, 9, 691 soft-in/soi"i-out decoding algorithm. See SlSO decoding algorithm

Index soft outputs, 711 source decoder, 2 source encoder, 1 SOY A (soft-output Viterbi algorithm), 558-63 span, vectorspace,59 SPA (sum-product algorithm), 871, 875-80 SPC (single-parity-check) code, 94-5 spectral component of V(X) jth, 255 spectral efficiency, 22, 952 spectral thinning, 775, 777-8 spectrum polynomial, 257 speed factor, decoder, 619 speed, information transmission, 9-10 squared Euclidean distances. See SE distances squaring construction, linear block codes, 119-25 S-random interleavers. See semirandom interleavers SRCC (systematic recursive convolutional code), 473 stack algorithm/decoding. See ZJ algorithm/decoding stack-bucket algorithm, 620 standard array, 66, 82-90 state defining information set, 346 state labeling, trellis diagram, 351-4 state space. See also node space complexity and dimension profile, 343 trellis diagram, 339 state transition, 339 stop-and-wait ARQ, 15 storage medium. See channel ST (stuttering), 1174 subfield, 36 subgroup, 29 submatrix, 63 suboptimum decoding, 605-6 sufficient conditions on optimality of decoded codeword, 402-7 sum-product algorithm. See SP A symbol-error probability, 956- 7 symbol location matrix, 939 symbol location vector, 939 symbols. See codeword symbol transmission rate, 9-10 synchronization, decoder, 550 syndrome, 66 computation and error detection, 150-5 computations, BCH codes, 224-5 linear block codes, 72-6 Newton's identities, 208-9

T2:591

OSD algorithm of order i. 439 registers, 1108-10 syndrome-based ordered statistic decoding algorithm, 438-9 syndrome decoder/decoding, 66, 82-90, 646, 647 syndrome sequence, 646 systematic encoders, 464 systematic feedback encoders, 470-1, 474-7 systematic feedforward encoders, 464-9 systematic output sequences. 464 systematic structure, 68 tail-biting Viterbi algorithm, 582-98 tail part of the tree, 606 Tanner graphs, 855-8, 892-8, 908-9, 922- 7, 931,932 TCM (trellis-coded modulation), 22-3 construction, 980-92 described, 952-80 multidimensional, 1015-56 performance analysis, 992-8 rotationally invariant, 998-1015 TDRSS (NASA Tracking Data Relay Satellite System), 760-1 terminal nodes, 606 test error pattern set, 409 threshold decoding, 605-6 threshold increment, 621 throughput efficiency, 1159-63, 1170-1 time-invariant code trellis, 341 lime-invariant convolutional encoders, 487. 774 time-out vector, 1164 time-varying code trellis, 341 TOF (trellis oriented form), 343 transform pair, 257 transition probabilities, 7-8 transpose, 63 trellis-coded modulation. See TCM trellis diagrams bit-level, 342-51, 354-60 Cartesian product, 382-90 complexity and symmetry, 367 - 74 construction and state labeling based on parity-check matrix, 360-6 decoding based on BCJR algorithm, 563-82 code construction for use with Viterbi decoding, 538-44 described, 515 performance bounds, 525-37 punctured and tail-biting, 582-98

U~ICl

Index

SOVA, 558-63 Viterbi algorithm, 516-25, 544-58 described, 338 finite-state machine model and, 338-42 low-weight subtrellises, 380-2, 704-11 MAP decoding algorithm, 711-18 based on sectionalized trellis, 718-26 Max-log-MAP algorithm, 726-34 recursive MLD algorithm, 695-704 sectionalization and parallel decomposition, 374-80 soft-decision decoding algorithms, 691 state labeling, 351-4 Viterbi algorithm, 516-20, 691-5 trellis oriented form. See TOF triple-error event IRWEF, 794 truncated code of C, 357 truncation distance, 549-50 turbo coding, 454 described, 766-83 distance properties, 783-807 iterative decoding, 826-44 performance analysis, 807 -13 turbo codes, design of, 819-26 twofold Euclidean geometry codes, 319-25 two-stage hybrid SPA/MLG decoding, 880 type-I DTI code, 288 type-I geometry-Q LDPC code, 859-60 UEP (unequal error protection) codes, 1090 UMP (uniformly most powerful) MAP-based decoding algorithm, 444 uncorrectable error patterns, 278 undetectable error patterns, 73, 79 undetected bit-error probbility, 626 unequal error protection codes. See UEP codes unequal error protection property, 586 uniform interleaver, 783

uniformly most powerful MAP-based decoding algorithm. See UMP MAP-based decoding algorithm uniformly sectionalized, 375 uniform mapping, 959-63, 975 union bound, 398 unit element, 32 ITJ11lllil + vi-construction, 117-18, 123 Vandermonde determinant, 203 VA (Viterbi algorithm), 399,453, 515 decoding based on, 516-25 implementation and performance, 544-58 SOVA, 558-63 trellis representation, 691-5 trellis structure to use with, 538-44 vector addition, 55 vector spaces, 55-61 w1-weight trellis search, 710-11 waterfall region, 824 WED (weighted-erasure decoding), 413-17 WEF (codeword weight enumerating function), 488 weight distribution, BCH codes, 227-30 weighted-erasure decoding. See WED weighted MLG (majority-logic) decoding, 439-42, 873-4 weight enumerators, 91 WER (word-error rate), 15-16, 537 writing unit. See modulator w1c-weight subtrellis, 380 word-error probability, 807-10 word-error rate. See WER zero element, 32 zero polynomial, 45 ZJ algorithm/decoding, 605-20