ABSTRACT
The project focuses on error control and concealment for mobile video transmission using feedback techniques. The transmission of high quality real time interactive video over lossy networks due to data loss in the network causes video quality to degrade. The issue of error control and concealment in video transmission has become a very important aspect in video delivery over unreliable networks such as internet and wireless networks. Three techniques which are effective in controlling error during video transmission based on the role of the encoder and decoder was employed. In Retransmission, repair packets were retransmitted only when some packets are lost. This conventional data transmission retransmits the error packets and add at least one round trip time to the display time of a frame after its initial transmission. The packets that arrive after their display time was used to reduce error propagation. With Intra Update, one of the frames in the previous frame was used as a reference frame and encode the current frame in intra mode when a message was received from the decoder rather than using older frames as a reference frame. Under Reference Picture selection (RPS) which operates in both positive acknowledgement (ACK) and negative acknowledgement (NACK), the encoder selects one of the several previous frames as a reference frame for predictive encoding of subsequent frames. The Batch Video Quality Metric (BVQM) tool was used to determine the impact of the three feedback techniques. High quality videos with a wide variety of scene complexity and motion vector magnitude were selected and encoded. The results obtained reveal that at a higher round trip time of 240ms, video quality decreases for videos using Retransmission for repair but increases if the round trip time decreases at 60ms. With Intra Update, when the packet loss increases at 0.363, 0.521, 1.483, video quality degrades but a perfect video quality was achieved when the round trip time decreases at 0.731, 0.659 0.647. Under high packet loss of 1.111, 1.310, 2.169 at 20% and under high round trip time of 1.412, 2.419, 2.510 at 240ms, the quality of video degrades for RPS NACK but increases at a lower round trip time of 4.501, 4.471, 4.086 at 60ms and lower packet loss of 4.628, 4.456, 3.867 at 1%. With RPS ACK, video quality increases when the round trip time of 1.883, 1.722, 1.624 and packet loss of 45.323, 43.147, 42.529 was high but degrades when packet loss of 2.031, 2.568, 3.035 and round trip time of 1.121, 1.205, 1.283 decreases. When the results were compared, the loss rate was less than 1.482, RPS NACK performed better than RPS ACK but when it was higher than 1.482, RPS ACK performed better than RPS NACK at high round trip time and packet loss.
TABLE OF CONTENTS
Title Page i
Declaration ii
Certification iii
Dedication iv
Acknowledgments v
List of Tables ix
List of Figures x
List of Plates xii
Abstract xiv
CHAPTER 1: INTRODUCTION
1.1 Background of Study 1
1.2 Problem Statement 4
1.3 Aim and Objectives of the Study 5
1.4 Significance of Study 5
1.5 Scope and Limitations 5
1.6 Work Organisation 6
CHAPTER 2: LITERATURE REVIEW
2.1 Historical Background 7 2.2 Forward Error Correction (FEC) 8
2.3 Joint Source Channel Coding 9
2.4 Layered Coding 9
2.5 Error Concealment 9
2.5.1 Recovery of texture information 11
2.5.2 Recovery of coding mode 11
2.5.3 Recovery of motion vector 11
2.6 Feedback Based Error Control for Video Transmission 12
2.6.1 Retransmission 13
2.6.2 Reference picture selection 13
2.6.3 Intra update 14
2.7 H.264 Video Compression Standard 14
2.7.1 H.264 data structure 14
2.8 Review of Related Works 17
2.9 Research Gap 28
CHAPTER 3: MATERIALS AND METHODS
3.1 Materials 29
3.1.1 Video quality measurement tool 29
3.1.2 Selection of video sequences 32
3.1.3 Encoding video sequences using H.264 33
3.2 Methods 34
3.2.1 Block diagram of error control and concealment 34
3.3 Feedback Based Error Control Techniques 35
3.3.1 Retransmission 35
3.3.2 Reference picture selection 37
3.3.3 Intra update 38
3.4 Mathematical Modelling of Feedback Based Error Control 40
3.4.1 Mathematical modelling of retransmission 40
3.4.2 Mathematical modelling of RPS NACK 42
3.4.3 Mathematical modelling of RPS ACK 43
3.4.4 Mathematical modelling of intra update 44
CHAPTER 4: RESULTS AND DISCUSSION
4.1 Results 47
4.1.1 Retransmission 47
4.1.2 Reference picture selection 52
4.1.2.1 Negative acknowledgement 52
4.1.2.2 Positive acknowledgement 55
4.1.3 Intra update 58
4.2 Comparison of Feedback Based Error Control Techniques 61
CHAPTER 5: CONCLUSION AND RECOMMENDATIONS
5.1 Conclusion 66
5.2 Recommendation 67
5.3 Contribution to Knowledge 68
5.4 Research Findings 68
References 69
Appendices
LIST OF TABLES
3.1: Video sequences used with motion vector magnitude and their description 33
4.1: Results of group of picture length on video quality at 60% concealed 48
4.2: Results of group of picture length on video quality at 80% concealed 48
4.3: Results of retransmitted packet loss on video quality 49
4.4: Results of retransmitted round trip time on video quality 49
4.5: Results of RPS NACK packet loss on video quality 53
4.6: Results of RPS NACK round trip time on video quality 54
4.7: Results of RPS ACK packet loss on video quality 56
4.8: Results of RPS ACK round trip time on video quality 56
4.9: Results of Intra update packet loss on video quality 59
4.10 : Results of Intra update round trip time on video quality 59
4.11: Comparison of the four error repair techniques 62
4.12: Comparison of RPS NACK and Intra update 63
4.13: Comparison of RPS ACK and RPS NACK 64
LIST OF FIGURES
3.1: Block diagram of error control and concealment for mobile video transmission 34
3.2: Frames recovery using retransmission 36
3.3: Encoding using RPS NACK 37
3.4: Encoding using RPS ACK Mode 38
3.5: Encoding frames using Intra Update 39
4.1: VQM against group of picture length for 60% concealed frame 50
4.2: VQM against group of picture length 80% concealed frame 50
4.3: VQM against packet loss for Retransmission 51
4.4: VQM against round trip time for Retransmission 52
4.5: VQM against packet loss for RPS NACK 54
4.6: VQM against round trip time RPS NACK 55
4.7: VQM against packet loss RPS ACK 57
4.8: VQM against round trip time RPS ACK 57
4.9: VQM against packet loss for Intra Update 60
4.10: VQM against round trip time for Intra Update 61
4.11: VQM against packet loss 62
4.12: VQM against packet loss 63
4.13: VQM against packet loss 65
LIST OF PLATES
2.1: A sequence with I, B and P frames 15
LIST OF ACRONYMS
ARQ Automatic Repeat Request JSCC Joint Source and Channel Coding FEC Forward Error Correction RPS Reference Picture Selection NACK Negative Acknowledgment ACK Positive Acknowledgment GOP Group of Pictures GOB Group of Blocks BS Base Station
FEC/ARQ Forward Error Correction/Automatic Repeat Request
BSC Binary Symmetric Channel
VQM Video Quality Metric MB Macro-block
GUI Graphical User Interface HARQ Hybrid Automatic Repeat Request AMC Adaptive Modulation and Coding KRPS Keyframe Reference Picture Selection AVC Advanced Video Coding MPEG Moving Picture Experts Group HEVC High Efficiency Video Coding MVC Motion Vector Concatenation ADVS Activity Dominant Vector Selection FDVS Forward Dominant Vector Selection RDO Rate Distortion Optimization EC Error Concealment FMO Flexible Macro-block Ordering PHY Physical MSE Mean Square Error APP Apply DL Data Link PSNR Peak Signal to Noise Ratio Cb Blue Chrominance Component Cr Red Chrominance Component Y Luminance Component QoS Quality of Service PDA Personal Data Assistant ISD Independent Segment Decoding ISO International Organization for Standardization ITU International Telecommunication Union
IEC International Electro-technical Commission
VCEG Video Coding Experts Group CIF Common Intermediate Format ANSI American National Standards Institute BVQM Batch Video Quality Metric HRC Hypothetical Reference Circuit ITS Institute for Telecommunication Sciences RP Retransmission Packet
NTIA National Telecommunications and Information
MCTP Motion Compensated Temporal Prediction
RTT Round Trip Time DCT Discrete Cosine Transform CSV Comma Separated Values
CHAPTER 1
INTRODUCTION
1.1 BACKGROUND OF STUDY
During fading periods, mobile networks cannot guarantee quality of service because of high bit error rates that occur which results in video degradation. Errors transmitted in a mobile network can lead to loss of the connection where there is no error correction scheme such as forward error correction. These widely varying error rates limit the effectiveness of forward error correction (FEC) since a worst case design would lead to a restrictive amount of lost (Girod and Färber, 1999).
The increasing demand for mobile transmission in work environment has spring in the large use of wireless communication in outdoor as well as indoor environments. In spite of the surge in network connection, video data packets still loss network connections. Lost packets are harmful to video data due to contingency between video frames during encoding where lost video packet can result in error generation to many other video frames. To cover up the loss, video users that encounter loss can use error concealment. The ability to satisfactory repair video without response to the video server is limited (Wang et al., 2007).
Error control is an integral technique which dictates and repair data frames that have been damaged during transmission. The number of error caused during data transmission over a wireless channel causes a degradation on image and video quality. The errors introduced during data transmissions are not entirely eliminated, even with a strong error repair scheme such as Forward Error Correction, unknown error may still exist in the transmission channel which leads to a severe degradation on video quality. In such cases, a strong error correction scheme is needed to reduce the optical effect of transmission errors (Ye et al., 2011).
Error repair techniques such as Forward Error Correction add redundant bits to a bit stream, which allow the decoder detect and repair transmitted error packets. There will be a reduction in error propagation but it requires additional data to be added to the video stream. Encoding and decoding of these techniques can be somewhat intriguing due to extra data redundancy added to the bit stream which can incur a drop in video quality (Chun and Klaus, 2014).
In digital communications, error concealment aims to minimize the deterioration of signals caused by missing packet, efficient video transmission over unreliable channels may encounter huge issues because of unpreventable packet loss. At the decoder side, Error concealment are deployed to repair the damaged regions by utilizing spatial redundant data without altering the encoder structure (Ziguan et al., 2012).
Feedback based error repair techniques have interaction between the encoder and decoder and uses the acknowledgement from the decoder to adapt the source coder to the channel conditions. The feedback data provided by the decoder shows the spot of damaged parts of the video stream, On receiving the feedback data, the encoder can locate the affected areas and treat them individually (Houqiang and Stochhammer, 2012).
This thesis is centered on error control and concealment for mobile video transmission using feedback techniques, which includes Reference Picture Selection (RPS), Retransmission and Intra Update.
In Reference Picture Selection (RPS), the encoder picks one of the several previously received frames rather than switching the damaged frame at the encoder to stop inter frame error propagation at the decoder. The coder encode the current frame with reference to a frame that has been decoded successfully. This Reference Picture Selection can reduce the surplus bitrate by using the previously received frame called Reference Picture Selection Negative Acknowledgment (RPS NACK) or wait for the organized frame as the older reference frame called positive acknowledgment (RPS ACK).
In RPS NACK mode, when negative acknowledged frames are received by the encoder, the action of the encoder is not altered during error free transmission but uses the previous frame as a reference frame to encode the damaged frame. The video quality with RPS NACK mode drops when a transmission error occurs for a period of one round trip time.
In the RPS ACK mode, frames received correctly are recognized and the encoder uses only recognized frames as a reference frame to encode the damage frame. Since the encoder uses the older frame for prediction with a rise in round trip time, the coding performance drops (Girod and Farber, 1999).
Retransmission base error repair has been considered unsuitable for packet recovery since it requires one additional round trip delay to repair lost packets. Despite its drawback, retransmission base error repair is still an alluring solution due to its adequate bandwidth and cost processing. Retransmission is still applied in various high speed protocols due to its cost processing.
Retransmission is widely suited for most unicast stored media and various interactive applications where the retransmitted packet is low, thereby improving the error repair for video transmission without significantly increasing the delay (Papadapoulos and Parulkar, 1997).
With Intra Update error repair technique, intra prediction of video frame is coded in intra mode. During error free transmission, one of the frames in the previous frame is used as a predicting frame based on the feedback information sent by the decoder, the encoder knows the parts in a video frame that are bad and intra code those damaged parts in Intra mode (Andreas et al., 2006).
1.2 PROBLEM STATEMENT
Beyond the available bitrate, mobile video transmission offers a number of technical challenges since the mobile networks cannot provide quality of service due to high bit error rates which occurs during fading period which can lead to transmission errors in a mobile channel. These challenges are as follows:
1) Retransmission of corrupted data frames which can incur a delay and might be unacceptable for real time conversational services. One of the approaches to address these limitation is with the use of forward error correction (FEC) which was carried out for error repair and is increasingly being applied to eradicate errors in the system.
2) Transmitting high quality real time interactive video over lossy networks is challenging since data loss due to transmission errors can severely degrade video quality. In addressing this issue, the cipher and decipher reduces errors generated by adding additional data to the video stream (Ababneh and Almomani, 2015).
3) The coding/decoding and conceal error are managed in an independent manner with the use of feedback modes where there is an exchange among the cipher and decipher. Distorted video is being reclaimed by the decoder by calculating the distorted portion based on the data sent to the encoder. The encoder adds redundancy to the transmitted data in an orderly fashion so that the decoder can base on the encoding form to repair errors (Farber et al., 2009).
1.3 AIM AND OBJECTIVES
The aim of this research work is to investigate error control and concealment for mobile video transmission using feedback techniques.
The main objectives include the following:
• To study the existing literatures of related work.
• To investigate different techniques of error control scheme.
• To develop a model for feedback error control using Retransmission, Reference Picture Selection and Intra Update.
• To validate the results obtained with the existing methods.
1.4 SIGNIFICANCE OF STUDY
The significance of this thesis when implemented is to use Feedback based error repair techniques to eradicate errors in a video stream during transmission. This scheme ensures that the information received by the receiver is exactly the information transmitted by the sender. The receiver will detect the error and also determines the location of the error in the data.
To stop error generation without incurring excess loss of coding efficiency.
To improve the visual quality for end users.
1.5 SCOPE AND LIMITATIONS
The scope of this thesis will be limited to the application of feedback technique to the development of models for the feedback error control and concealment that improves the video quality during transmission.
The model will be implemented by developing Video Quality Metric to evaluate video quality performance and to indicate the model that provided lower classification of errors.
1.6 WORK ORGANIZATION
This thesis is organised into five chapters. Chapter 1 presents the introduction and background of the study. The project problem statement, the significance of study, the project aims and objectives are also presented in this chapter. The final part of this chapter looks at the scope and organisation of the project.
Chapter 2 presents the literature review of error control and concealment using feedback techniques. Related research works based on error control and concealment using feedback techniques are also discussed.
In chapter 3, the methodology and the processes leading to the elimination of error in the system based on the analytical models were presented.
Chapter 4 presents the results and discussion of error control and concealment with the use of Batch Video Quality Metric. Finally, chapter 5 is conclusion and recommendation for future work.
Login To Comment