Digital Communications

Introduction to Digital Communication Systems Lecture Notes for ECE 361 Fall 1995

Dilip V. Sarwate Department of Electrical and Computer Engineering University of Illinois at Urbana-Champaign Urbana, Illinois 61801

© 1995 by Dilip V. Sarwate All rights reserved. No part of this manuscript may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise without the prior written permission of the author.

Introduction to Digital Communication Systems: Lecture Notes for ECE 361, Fall 1995

Note to ECE 361 students These optional lecture notes sold through the IEEE Student Branch are intended as a supplement to the required textbook for ECE 361: Introduction to Digital Communications by M. B. Pursley, (also sold through the IEEE Student Branch.) The lecture notes include the text and diagrams that will appear on the viewgraphs that I will use in teaching ECE 361 in Fall 1995. The lecture notes also include much additional material in the form of comments and remarks on the material presented on the viewgraphs, as well as several exercises. This additional material is intended to help you understand the concepts presented on the viewgraphs. In some instances, the comments will remind you of a key fact that will make it clear how Step 6 on the viewgraph follows from Step 5. The comments may have a few details that have been skipped on the viewgraph itself, or (horrors!) they may suggest that you work out the details of some derivation all by yourself. Many of these comments (usually in paraphrased form) will also be made orally during the lectures. Thus, the inclusion of the comments in the lecture notes saves you the trouble of scribbling down these pearls of wisdom. Of course, some of these comments will not be made orally, usually because of sheer forgetfulness on my part but occasionally because of a lack of time. Thus, the inclusion of the comments in these lecture notes ensures that what I forgot to say or did not have time to say is nonetheless available to you. Please keep in mind that, regardless of the reason for the omission, the remarks included in these lecture notes but not said out loud in class are just as important as the ones stated in a ringing voice, and you are urged not to omit such unstated facts when studying for the course. In these lecture notes, the material that will appear on the viewgraphs is printed with borders around it. The additional remarks and comments are distinguished by not having any borders. The lecture notes are also printed with wide outside margins to provide you with space for making your own notes or for doodling when the lectures are more boring than usual.

Dilip V. Sarwate Urbana, Illinois August 1995

iii

Introduction to Digital Communication Systems: Lecture Notes for ECE 361, Fall 1995

Contents Introduction to Digital Communication Systems............................................................................................. Analog vs Digital Communication Systems........................................................................................... Modern Digital Communicaton Theory.................................................................................................. Block Diagram of a Digital Communication System............................................................................. Probability Review .............................................................................................................................................. Notation and Axioms.............................................................................................................................. Conditional Probability ............................................................................................................ Bayes’ Formula ........................................................................................................................ Stochastic Independence .......................................................................................................... Random Variables .................................................................................................................................. Expectation............................................................................................................................... Gaussian Random Variables .................................................................................................... Many Random Variables ........................................................................................................................ Conditional Distributions ......................................................................................................... Jointly Gaussian Random Variables ........................................................................................ Limit Theorems ...................................................................................................................................... Markov’s and Chebyshev’s Inequalities; the Chernoff Bound ................................................ The Laws of Large Numbers.................................................................................................... The Central Limit Theorem .................................................................................................................... Random Processes............................................................................................................................................... Definition and Basic Properties .............................................................................................................. Distribution and Density Functions ........................................................................................................ Mean, Autocorrelation and Covariance.................................................................................................. Stationary and Wide-Sense-Stationary Random Processes.................................................................... Cross-Correlation and Cross-Covariance Functions .............................................................................. White Noise Processes............................................................................................................................ Random Processes in Linear Systems ............................................................................................................... Mean-Square Convergence vs Stochastic Mean-Square Convergence .................................................. Interchange of Expectation and Integration Operations ......................................................................... The Mean of the Output Process ............................................................................................................ Cross-correlation between Input and Output Processes.......................................................................... Autocorrelation Function of the Output Process .................................................................................... Spectral Analysis of Random Processes................................................................................................. Power Spectrum of a Random Digital Signal......................................................................................... Baseband Digital Communication Systems ...................................................................................................... Binary Communication System.............................................................................................................. Signal Output from Filter ......................................................................................................... Noise Output from Filter .......................................................................................................... Thresholds and Error Probabilities........................................................................................... Conditional Error Probabilities ................................................................................................ Two Criteria for Choosing the Threshold............................................................................................... Minimax Threshold Criterion................................................................................................................. Summary for Minimax Threshold Criterion ............................................................................ Minimum Average Error Probability (Bayes’) Criterion........................................................................ Properties of Bayes’ Threshold ................................................................................................ Properties of Bayes’ Error Probability..................................................................................... Summary for Minimax and Bayes’ Criteria............................................................................................ Matched Filters....................................................................................................................................... The Schwarz Inequality............................................................................................................ Proof......................................................................................................................................... Application to Matched Filter Design ...................................................................................... Error Probabilities for Orthogonal and Antipodal Signaling ................................................... Frequency-Domain Interpretation of the Matched Filter ......................................................... Transmission of a Sequence of Bits .........................................................................................

v

1 1 3 4 10 10 12 14 15 17 18 20 21 23 28 29 29 31 32 33 33 35 36 38 41 42 43 44 46 47 47 48 51 52 55 55 57 58 58 59 60 60 62 66 69 69 72 74 74 75 76 80 83 86

Introduction to Digital Communication Systems: Lecture Notes for ECE 361, Fall 1995

Correlation Receivers.............................................................................................................................. Comparison of Matched Filter and Correlation Receiver Outputs......................................................... Summary of Matched Filters and Correlation Receivers........................................................................ Signal Design.......................................................................................................................................... Pre-whitening Filters .............................................................................................................................. Signal Space Concepts ........................................................................................................................................ Orthogonal and Orthonormal Functions................................................................................................. Minimum Mean-Square Error Representations........................................................................ Complete Orthonormal Sets..................................................................................................... Gram-Schmidt Orthonormalization Procedure ........................................................................ Orthonormal Representations................................................................................................................. Signal Representation............................................................................................................... Noise Representation................................................................................................................ Vector Channels...................................................................................................................................... Decision Regions .................................................................................................................................... Binary Signaling..................................................................................................................................... Maximum-Likelihood Decision Regions................................................................................. Bayesian Decision Regions ...................................................................................................... Summary for Binary Signaling ................................................................................................ M-ary Signaling ...................................................................................................................................... Preference Regions and Decision Regions............................................................................... Symbol Error Probabilities....................................................................................................... Two Receiver Implementations................................................................................................ Union Bound on Error Probabilities......................................................................................... Nearest-Neighbor Approximation for Error Probabilities........................................................ Bit Error Probabilities .............................................................................................................. Relationship between Bit Error Probability and Symbol Error Probability............................. One-dimensional M-ary Signaling.......................................................................................................... Symbol and Bit Error Probabilities .......................................................................................... Signal Energy ........................................................................................................................... Asymptotic Behavior for Large M........................................................................................... Two-dimensional M-ary Signaling......................................................................................................... Symbol and Bit Error Probabilities .......................................................................................... Signal Energy ........................................................................................................................... Asymptotic Behavior for Large M........................................................................................... M-ary Phase-Shift Keying ...................................................................................................................... Symbol Error Probabilities....................................................................................................... Summary for One- and Two-Dimensional M-ary Signaling.................................................................. Multidimensional M-ary Signaling......................................................................................................... Symbol and Bit Error Probabilities .......................................................................................... Signal Energy ........................................................................................................................... Asymptotic Behavior for Large M........................................................................................... Bandwidth of a Digital Signal ................................................................................................................ The Landau-Pollak Theorem.................................................................................................... M-ary Orthogonal Signaling................................................................................................................... Probability of Correct Decision and Probability of Error ........................................................ Union Bound on the Probability of Error................................................................................. Probability of Bit Error ............................................................................................................ Asymptotic Behavior for Large M........................................................................................... Simplex or Trans-Orthogonal M-ary Signaling...................................................................................... Bi-Orthogonal M-ary Signaling.............................................................................................................. Union Bound of the Probability of Error.................................................................................. Bit Error Probability................................................................................................................. Generation of Orthogonal, Transorthogonal, and Bi-orthogonal Signals............................................... FSK, PPM, and Hadamard Signal Sets .................................................................................... Fast Walsh-Hadamard Transforms........................................................................................... Summary for Multidimensional M-ary Signaling .................................................................................. Optimality of Linear Receivers on Additive Gaussian Noise Channels.................................................

vi

89 91 94 95 101 104 104 105 106 107 108 109 112 113 114 115 116 120 123 125 125 130 132 134 136 138 140 143 144 145 146 147 149 151 152 153 154 155 156 156 157 158 159 160 161 162 163 164 166 170 170 173 174 177 177 178 180 183