Information Theory And Coding Elective-I

Course: M.Tech Code: 301 Branch: A Date: November 2023

Save as PDF

Opens your browser print dialog — select "Save as PDF" to download.

Total No. of Questions : 8 [Total No. of Printed Pages : 3 [2]
Roll No ..........................

MEDC-301(A)

M.E./M.Tech., III Semester

Examination, November 2023

Information Theory and Coding

(Elective-I)

Time : Three Hours Maximum Marks : 70

Note : i) Attempt any five questions.

        ii) All questions carry equal marks.

1. a)
Define uncertainty, information and entropy. State the various units of information and find the relation between them. Show that entropy of a binary system (two events) is maximum when both events are equiprobable.
b)
A discrete source emits one of five symbols once every milliseconds with probabilities 1/2, 1/4, 1/8, 1/16 and 1/32. Find the source entropy and information rate.
2. a)
Given xi = {X1, X2, X3, X4, X5} with probabilities P(xi) = {0.3, 0.25, 0.2, 0.12, 0.08, 0.05}. Make Huffman code. Find efficiency of this code.
b)
A very noisy symmetric binary channel is able to transmit 1000 bits per second, but has a 25% probability of introducing error into each bit transmitted. What is the channel capacity in bits per second?
3. a)
What is mutual information? Show that the mutual information I(X; Y) ≥ 0. Under what condition does the equality hold?
b)
A BSC has the error probability p = 0.2 and the input to the channel consists of 4 equiprobable messages x1=000; x2=001; x3=011; x4=111. Calculate:
  1. p(0) and p(1) at the input
  2. Efficiency of the code
  3. Channel capacity
4. a)
Consider a (6, 3) linear block code whose generator matrix is given by 
[1 0 0 | 1 0 1]
[0 1 0 | 1 1 0]
[0 0 1 | 0 1 1]
  1. Find the parity check matrix.
  2. Find the minimum distance of the code.
  3. Draw the encoder and syndrome computation circuit.
b)
Define Hamming weight, Hamming distance and minimum distance of linear block codes with suitable example.
5. a)
How will you prove that GF(23) is at least an integral domain? How will you prove it is a finite field?
  1. A (7, 4) cyclic code has a generator polynomial:
    g(X) = X3 + X + 1.
  2. Draw the block diagram of encoder and syndrome calculator.
  3. Find generator and parity check matrices in systematic form.
MEDC-301(A) PTO Contd...

[3]

6. a)
Using the generator polynomial g(x) = 1 + x2 + x3, generate the systematic and nonsystematic cyclic code words for the message vectors 1011 and 1001.
b)
Define G and H matrix and show that CHT = 0.
7. a)
What are BCH codes? Discuss the steps for decoding BCH codes with an example.
b)
Consider (3,1,2) Convolutional code with g(1) = (110), g(2) = (101) and g(3) = (111):
  1. Draw the encoder block diagram.
  2. Find the generator matrix.
  3. Find the code word corresponding to the information sequence (11101) using time domain approach.
8.
Write a short notes on any two:
  1. Viterbi Algorithm
  2. Lempel-Ziv Coding
  3. Bandwidth signal to Noise Trade off
  4. Hamming code and their applications
MEDC-301(A)