Ericsson Technology Review: Versatile Video Coding explained – the future of video in a 5G world
•
0 gostou•2,029 visualizações
Continuous innovation in 5G networks is creating new opportunities for video-enabled services for both consumers and industries, particularly in areas such as the Internet of Things and the automotive sector. These new services are expected to rely on continued video evolution toward 8K resolutions and beyond, and on new strict requirements such as low end-to-end latency for video delivery. The latest Ericsson Technology Review article explores recent developments in video compression technology and introduces Versatile Video Coding (VVC) – a significant improvement on existing video codecs that we think deserves to be widely deployed in the market. VVC has the potential both to enhance the user experience for existing video services and offer an appropriate performance level for new media services over 5G networks.
![Many new and emerging 5G use cases will soon require video
compression efficiency and functionality that are beyond the capabilities
of today’s leading video codecs. Versatile Video Coding – the new video
compression coding standard recently approved by the Moving Picture
Experts Group and the International Telecommunication Union – includes
both improved compression efficiency and new features to enhance
support for immersive video and low-delay video coding.
RICKARD SJÖBERG,
JACOB STRÖM,
ŁUKASZ LITWIC,
KENNETH ANDERSSON
The latest Ericsson Mobility Report estimates
that video accounted for 63 percent of the
traffic in mobile networks during 2019 and
that this share will increase to 76 percent by
2025 [1]. Enabled by continuously improving
network performance, this growth is driven
by the ever-increasing availability of video
content in streaming services and online
apps, and by changing consumer behavior
toward consumption of the content on mobile
devices. The evolution toward large screens
with high resolutions beyond HD on mobile
devices further raises consumer expectations
regarding the quality of delivered video
content. Video codecs play a critical role
in coping with these expectations.
■ While the quality of video content depends on
several characteristics such as high pixel bit depth,
high frame rate, wide color gamut (WCG) and high
dynamic range (HDR), it is the resolution (the
number of pixels in a video picture) that is most
directly associated with the bandwidth required
for transmission. Other key factors determining
the required bandwidth are related to the type
of the video content and the latency with which
the content is delivered to the end user.
Atthesametime,innovationsin5Gnetworks
offernewopportunitiesforvideo-enabledservices
forbothconsumers(remotelyrenderedvirtual/
extendedrealityandcloudgaming,forexample)
andindustries,particularlywithrespecttothe
InternetofThings(IoT)andtheautomotivesector.
VersatileVideo
Codingexplained
– THE FUTURE OF VIDEO IN A 5G WORLD
✱ VERSATILE VIDEO CODING
2 ERICSSON TECHNOLOGY REVIEW ✱ OCTOBER 14, 2020](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Ericsson Technology Review: Versatile Video Coding explained – the future of video in a 5G world
Semelhante a Ericsson Technology Review: Versatile Video Coding explained – the future of video in a 5G world (20)
Mais de Ericsson
Mais de Ericsson (17)
Último
Último (20)
Ericsson Technology Review: Versatile Video Coding explained – the future of video in a 5G world
- 1. Machine-type detection Observed KPI values Expected KPI values M-QoE prediction Gaps (M-QoE quantification) Induced subscriber/vertical KPI gaps Device traffic Vertical KPI Subscriber KPI Application-type inference Machine learning ERICSSON TECHNOLOGY C H A R T I N G T H E F U T U R E O F I N N O V A T I O N I # 1 0 ∙ 2 0 2 0 VERSATILE VIDEO CODING EXPLAINED
- 2. Many new and emerging 5G use cases will soon require video compression efficiency and functionality that are beyond the capabilities of today’s leading video codecs. Versatile Video Coding – the new video compression coding standard recently approved by the Moving Picture Experts Group and the International Telecommunication Union – includes both improved compression efficiency and new features to enhance support for immersive video and low-delay video coding. RICKARD SJÖBERG, JACOB STRÖM, ŁUKASZ LITWIC, KENNETH ANDERSSON The latest Ericsson Mobility Report estimates that video accounted for 63 percent of the traffic in mobile networks during 2019 and that this share will increase to 76 percent by 2025 [1]. Enabled by continuously improving network performance, this growth is driven by the ever-increasing availability of video content in streaming services and online apps, and by changing consumer behavior toward consumption of the content on mobile devices. The evolution toward large screens with high resolutions beyond HD on mobile devices further raises consumer expectations regarding the quality of delivered video content. Video codecs play a critical role in coping with these expectations. ■ While the quality of video content depends on several characteristics such as high pixel bit depth, high frame rate, wide color gamut (WCG) and high dynamic range (HDR), it is the resolution (the number of pixels in a video picture) that is most directly associated with the bandwidth required for transmission. Other key factors determining the required bandwidth are related to the type of the video content and the latency with which the content is delivered to the end user. Atthesametime,innovationsin5Gnetworks offernewopportunitiesforvideo-enabledservices forbothconsumers(remotelyrenderedvirtual/ extendedrealityandcloudgaming,forexample) andindustries,particularlywithrespecttothe InternetofThings(IoT)andtheautomotivesector. VersatileVideo Codingexplained – THE FUTURE OF VIDEO IN A 5G WORLD ✱ VERSATILE VIDEO CODING 2 ERICSSON TECHNOLOGY REVIEW ✱ OCTOBER 14, 2020
- 3. Thesenewservicesareexpectedtorelyon continuedvideoevolutiontoward8Kresolutions andbeyond,andonnewstrictrequirementssuchas lowend-to-endlatencyforvideodelivery.Sincethe datarateofsuchvideo-enabledservicesisextremely high,thecostwouldbeprohibitive–eveninthemost modernnetworks–unlessthevideowascarried inacompressedformat,withthehelpofanext generationvideocodec. VersatileVideoCoding(VVC),thenewvideo compressioncodingstandardthatwillbepublished asISO/IEC23090-3andITU-Trecommendation H.266,offersthehighestcompressionefficiency availabletodayandisthereforethecodecthatis bestsuitedtoofferasuitableperformancelevelfor newmediaservicesover5Gnetworks.Itcanalso enhancetheuserexperienceforexistingvideo servicesbydeliveringsubstantiallyhigherquality atthesamebitrate. Alternatively,itcanbeusedtoreducethebitrate (roughlybyhalf)whilemaintainingthesamequality. Forsomelegacyapplications,thecostofreplacingan oldercodecsuchasH.264withaneweronesuchas HEVChasnotbeeneconomicallybeneficial, sincethereductioninbitrateof40percenthasnot compensatedforthecostofreplacingalargenumber ofsettopboxes.Eveninsomeofthesecases,the introductionofVVCmaychangethiscalculation sincethebitratereductioncomparedtotheolder codecissomuchgreater.Figure1providesan overviewofpotentialapplicationareas. Videocompressionoptionsavailabletoday Avideocodec,whichcanbeimplementedin hardwareorsoftware,encodesand/ordecodes digitalvideo.Modernvideocodecscanreducethe bitrateofuncompressedvideotolessthanone percentoftheoriginalratewithoutanynoticeable visualqualitydegradations.Themajorityofthese videocodecshavebeenstandardizedandpublished byastandardsdevelopmentorganization(SDO). Themostpopularandwidelydeployedcodecs includetheMPEG(MovingPictureExpertsGroup) seriesfromISO/IEC(InternationalOrganization forStandardization/InternationalElectrotechnical Commission)andthecorrespondingH.26xseries Figure 1 VVC application areas Versatile Video Coding High-quality HD and 4K video streaming Enhancing existing services Enabling new services Telepresence and screen sharing 8K HDR video streaming Low-latency cloud gaming Immersive XR and telepresence IoT and automative 360-video VR streaming VERSATILE VIDEO CODING ✱ OCTOBER 14, 2020 ✱ ERICSSON TECHNOLOGY REVIEW 3
- 5. operators,broadcasters,chipsetvendorsand academics.Throughouttheprocess,all documentationincludingtechnicalcontributions, draftspecificationtext,referencesoftwareand conformancebitstreamsweremadeavailable publicly. Ericssonhasbeenanactiveparticipantinvideo standardizationformorethan20yearsandwas closelyinvolvedinthedevelopmentoftheVVC standard.Throughouttheprocess,weledseveral ofthecoreexperiments,chairedad-hocworking groupsandmadesignificantcontributionstothe developmentofthetechnologybehindthevideo codec,mostnotablyintheareasofdeblocking filtering,referencepicturemanagement,low-delay videocodingandoptimizedencoderconfigurations. Wealsoparticipatedineffortsthatmadeanimpact inotherareasofthevideocodecsuchasintra andinterpredictionandadaptiveloopfiltering. Keybenefitsofversatilevideocoding Asthelatestandmostsophisticatedvideocodecto date,VVCoffersthehighestcompressionefficiency ofallvideocodecs,anditisparticularlyappropriate forhigherresolutionvideostreamsduetoits codingtools,whichcanoperateonblocksizes ofupto128x128pixelsandwith64x64samplesize transforms.VVCcanachieveareductioninbitrate ofaround40percentforexistingHDand4K servicesdeployedwithHEVC,atthesame visualquality. Figure2providesaperformancecomparison betweenVVCandfourothervideocodecs. ThecomparisonwasdonebyEricssonResearch usingvarioussourcesofinformationincluding in-housetesting[3,4].Thefigureshowsthe approximaterelativebitraterequirementforeach videocodecinordertoreachthesamevideopicture qualityforHDand4Kcontent,benchmarked againsttheHEVCvideocodec,whichisnormalized to100percent.VVCperformssignificantlybetter, requiringonly60percentoftherelativebitrate comparedwith bothHEVCandAV1. Ontopofitssuperiorcompressionperformance, theversatilityofVVCalsomakesitanattractive choicebeyondmainstream2Dvideoservices. VVChasbeendesignedtohandlebothtraditional camera-capturedcontentaswellastheincreasingly VERSATILE VIDEO CODING ✱ OCTOBER 14, 2020 ✱ ERICSSON TECHNOLOGY REVIEW 5 Figure 2 Relative bitrates for the same video quality 180% 160% 140% 120% 100% 80% 60% 40% 20% 0% 170% Relative bitrates for the same video quality (lower is better) H.264 VP9 HEVC AV1 VVC 120% 100% 100% 60%
- 7. oftuninginatGDRpositions.However,during standardizationofVVC,anEricssonproposalto addaGDRpicture-typeindicatortothestandard wasadopted.ThismakesGDRsupportmandatory inVVC,whichensuresdecoderinteroperability forsuchlow-delayapplications. BesidestheGDRfeature,VVCalsoinherited decodingunit(DU)baseddecoderoperationfrom itspredecessor,HEVC.DUsenableanencoderto outputafirstpartofapicture,inaspecified controlledmanner,withoutrequiringtheentire picturetofirstbeencoded.DUsandmandatory GDRsupportmakeVVCtheprimevideocodec choiceforlow-delayvideoapplications. HowVVCworks Itisimportanttonotethatthesuperiorcoding efficiencyofVVCisnotduetoanysingle compressiontool.Rather,itistheresultofcombining manytools,eachcontributingwithasmall compressionimprovement.Themostsignificant ofthesetoolsareblockpartitioning,advancedinter prediction,dependentquantization,adaptive loopfilteringandimproveddeblockingfiltering. Theadditionofsophisticatednewcodingtools hasalsohadanimpactoncomputationalcomplexity. Asaresultofthesedevelopments,itisexpectedthat VVCdecodercomplexitywillbearoundtwicethat ofHEVC. Blockpartitioning Similartoitspredecessors,VVCusesablock-based hybridcodingarchitecture.Avideocodecbasedon suchanarchitecturedoesnotcodepixelvalues directlybutinsteadpredictsandcodesonlyerror informationtocompensatefortheinaccurate prediction.Theprocessrunsonblocksofpixels ratherthantheentirepicture,sincethismakesit possibletoadjusttolocalvideopicture characteristics,minimizingthepredictionerror. Forvideocodecsbasedonsuchanarchitecture, anefficientpicturepartitioningschemeispivotal toachievinghighcompressionefficiency. InVVC,eachpictureissplitintonon-overlapping squarescalledcoding-treeunits(CTUs).Thelargest CTUsizeallowedinVVCis128x128pixels,larger thanthe64x64maximumsizeallowedinHEVC. Largeblocksimprovetheefficiencyofcoding flatareassuchasbackgrounds,especiallyfor high-resolutionvideossuchasHDand4K. Inordertoefficientlyrepresenthighlydetailed areassuchastexturesandedges,VVCemploys aflexiblepartitioningschemethatcanpartition 128x128-sizedCTUsdowntocodingunits(CUs) assmallas4x4pixels. Theschemeisbasedontwoparts.Thefirstisthe quaternarytree(quadtree)splitthatisalsoavailable inHEVC,whichcanrecursivelysplitCTUinto squaredCUsdownto4x4pixels,smallerthanthe 8x8minimumCUsizeinHEVC.Thesecondpart consistsofbinary-treeandternary-treesplitsthat partitionablockintotwoandthreerectangles respectively.Bothbinaryandternarytreesplits canoperateineitherhorizontalorvertical directions,berecursivelyappliedandmixed togetherinanestedmulti-typetree. Terms and abbreviations 3DOF – Three Degrees of Freedom | ALF – Adaptive Loop Filter | CTU – Coding-Tree Unit | CU – Coding Unit | DU – Decoding Unit | DVB – Digital Video Broadcasting | GDR – Gradual Decoding Refresh | HDR – High Dynamic Range | HEVC – High Efficiency Video Coding | IEC – International Electrotechnical Commission | IETF – Internet Engineering Task Force | ISO – International Organization for Standardization | ITU– International Telecommunication Union | ITU-T – ITU Telecommunication Standardization Sector | MPEG – Moving Picture Experts Group | SDO – Standards Development Organization | UHD – Ultra High Definition | VR – Virtual Reality | VVC – Versatile Video Coding | XR – Extended Reality VERSATILE VIDEO CODING ✱ OCTOBER 14, 2020 ✱ ERICSSON TECHNOLOGY REVIEW 7
- 8. Figure3providesanexampleofthepartitioning ofaclose-upsectionofapicture.HEVCblock partitioning(attopright)usesquaternary-treesplit withcodingblocksupto64x64pixels.VVCblock partitioning(atbottomright)usesquaternary-tree splitwithcodingblocksupto128x128pixelsand nestedmulti-typetreesplitemployingbinaryand ternarytreesplits. TheblockpartitioninginVVCishighlyflexible andprovidesabout8percentbitratereductionover HEVC[5].However,thisflexibilitycomesata computationalcost,especiallyontheencoderside, wheremanymorepermutationsneedtobe evaluatedtoselecttheoptimalpartition. Advancedinterprediction Oneofthemostefficientbit-savingtechniquesin videocompressionisinter-pictureprediction(more commonlyreferredtoasinterprediction),which simplymeanscopyingsamplevaluesfrompreviously codedpictures.Theinformationaboutwhereto copyfrom–thehorizontalandverticaldisplacement forablock–isstoredinwhatiscalledamotion vector.Techniquesthatimproveinterpredictionare responsibleforasubstantialpartofthebitrate reductionbetweenVVCandHEVC. Whilemanytechniqueshavebeenusedtoachieve theadvancedinterpredictioninVVC,fiveofthem areparticularlynoteworthy:theaffinemotion model,adaptivemotionvectorresolution, bi-directionalopticalflow,decoderside-motion vectorrefinementandgeometricpartitioningmode. Theprecisionofthemotionvectorshasalsobeen increasedto1/16pixelcomparedtothequarterpixel resolutionofHEVC. Inpreviousvideocodingstandards,ithasbeen possibletocompensatefortranslationalmotion; thatis,thedecodercanbeinstructedtofetchsample valuesnotfromthesameplaceinapreviousimage butratherfromanotherposition(1.75pixelstothe left,forexample).InVVC,theaffinemotionmodel toolmakesitpossibletospecifynotonlydistance 180% 160% 140% 120% 100% 80% 60% 40% 20% 0% 170% Relative bitrates for the same video quality (lower is better) H.264 VP9 HEVC AV1 VVC 120% 100% 100% 60% Figure 3 Comparison of the partitioning of a close-up section of a video picture in HEVC and VVC ✱ VERSATILE VIDEO CODING 8 ERICSSON TECHNOLOGY REVIEW ✱ OCTOBER 14, 2020
- 11. Further reading ❭ Ericsson blog, Cutting the bitrate with Versatile Video Coding, available at: https://www.ericsson.com/en/ blog/2019/7/mpeg-cut-the-bitrate-versatile-video-coding ❭ Ericsson blog, Industry Forum to promote Versatile Video Coding, available at: https://www.ericsson.com/ en/blog/2018/9/industry-forum-to-promote-versatile-video-coding References 1. Ericsson Mobility Report, June 2020, available at: https://www.ericsson.com/en/mobility-report/reports/ june-2020 2. Ericsson Technology Review, Next generation video compression, April 24, 2013, Fröjdh, P; Norkin, A; Sjöberg, R, available at: https://www.ericsson.com/en/reports-and-papers/ericsson-technology-review/ articles/next-generation-video-compression 3. 2019 Picture Coding Symposium (PCS), Ningbo, China, 2019, pp. 1-5, Compression Performance of the Versatile Video Coding: HD and UHD Visual Quality Monitoring, Sidaty, N. et al., available at: https://ieeexplore.ieee.org/document/8954562/authors#authors 4. ResearchGate, SMPTE Motion Imaging Journal, vol. 128, no. 10, pp. 14-24, Nov.-Dec. 2019, Analysis of Emerging Video Codecs: Coding Tools, Compression Efficiency and Complexity, J. L. Tanou and M. Blestel, available at: https://www.researchgate.net/publication/332073033_Analysis_of_Emerging_Video_ Codecs_Coding_Tools_Compression_Efficiency_and_Complexity 5. IEEE Transactions on Circuits and Systems for Video Technology, vol. 30, issue 5, pp. 1311-1325, May 2020, doi: 10.1109/TCSVT.2019.2945048, A VVC Proposal With Quaternary Tree Plus Binary-Ternary Tree Coding Block Structure and Advanced Coding Techniques, Y. Huang et al., available at: https://ieeexplore.ieee.org/document/8859290 VERSATILE VIDEO CODING ✱ OCTOBER 14, 2020 ✱ ERICSSON TECHNOLOGY REVIEW 11
- 12. theauthors ✱ VERSATILE VIDEO CODING 12 ERICSSON TECHNOLOGY REVIEW ✱ OCTOBER 14, 2020 Rickard Sjöberg ◆ is an expert in video compression at Ericsson Research where he currently works as a technical lead in video coding research. He joined Ericsson in 1996 and has contributed several hundred proposals for the ITU-T and MPEG video-coding standards. In addition, he has worked in product development related to video coding at Ericsson, including six months at Ericsson Television in Southampton in the UK. Sjöberg holds an M.S. in computer science from KTH Royal Institute of Technology in Stockholm, Sweden. Jacob Ström ◆ is a principal researcher at Ericsson Research with a focus on video compression. He joined Ericsson in 2001 and has contributed to standardization in the area of high dynamic range video as well as to the standardization of both HEVC and VVC. He is coauthor of more than 120 granted patents and has a similar number of patents pending. Ström holds a Ph.D. in image coding from Linköping University, Sweden, and has been a visiting Ph.D. student at the University of California San Diego and the Massachusetts Institute of Technology (MIT) in the US. Łukasz Litwic ◆ is a research leader at Ericsson Research. He joined Ericsson Television in 2007, where he worked on various aspects of image processing and video compression research, which formed the foundation of Ericsson real-time broadcast encoding products. In 2017, he joined Ericsson Research in Stockholm, Sweden, where he leads the Visual Technology team. He holds an M.S. from Gdansk University of Technology, Poland, and a Ph.D. from the University of Surrey, in Guildford in the UK. Kenneth Andersson ◆ is a senior specialist in video coding at Ericsson Research. He joined Ericsson in 1994 to work on speech coding and since 2005 has been active in video coding standardization in ITU-T and ISO/IEC for development of HEVC and VVC. He holds an M.Sc. in computer science and engineering from Luleå University in Sweden and a Ph.D. from Linköping University.
- 13. ISSN 0014-0171 284 23-3349 | Uen © Ericsson AB 2020 Ericsson SE-164 83 Stockholm, Sweden Phone: +46 10 719 0000