当前位置：搜档网 › Understanding the Characteristics of Internet Short Video Sharing YouTube as a Case Study

Understanding the Characteristics of Internet Short Video Sharing YouTube as a Case Study

a r X i v :0707.3670v 1 [c s .N I ] 25 J u l 2007

Understanding the Characteristics of Internet Short

Video Sharing:YouTube as a Case Study

Xu Cheng

School of Computing Science Simon Fraser University Burnaby,BC,Canada Email:xuc@cs.sfu.ca

Cameron Dale

School of Computing Science Simon Fraser University Burnaby,BC,Canada Email:camerond@cs.sfu.ca

Jiangchuan Liu

School of Computing Science Simon Fraser University Burnaby,BC,Canada Email:jcliu@cs.sfu.ca

Abstract —Established in 2005,YouTube has become the most successful Internet site providing a new generation of short video sharing service.Today,YouTube alone comprises approximately 20%of all HTTP traf?c,or nearly 10%of all traf?c on the Internet.Understanding the features of YouTube and similar video sharing sites is thus crucial to their sustainable development and to network traf?c engineering.

In this paper,using traces crawled in a 3-month period,we present an in-depth and systematic measurement study on the characteristics of YouTube videos.We ?nd that YouTube videos have noticeably different statistics compared to traditional streaming videos,ranging from length and access pattern,to their active life span,ratings,and comments.The series of datasets also allows us to identify the growth trend of this fast evolving Internet site in various aspects,which has seldom been explored before.

We also look closely at the social networking aspect of YouTube,as this is a key driving force toward its success.In particular,we ?nd that the links to related videos generated by uploaders’choices form a small-world network.This suggests that the videos have strong correlations with each other,and creates opportunities for developing novel caching or peer-to-peer distribution schemes to ef?ciently deliver videos to end users.

I.I NTRODUCTION

The recent two years have witnessed an explosion of net-worked video sharing as a new killer Internet application.The most successful site,YouTube,now features over 40million videos and enjoys 20million visitors each month [1].The success of similar sites like GoogleVideo,YahooVideo,MySpace,ClipShack,and VSocial,and the recent expensive acquisition of YouTube by Google,further con?rm the mass market interest.Their great achievement lies in the combi-nation of the content-rich videos and,equally or even more importantly,the establishment of a social network.These sites have created a video village on the web,where anyone can be a star,from lip-synching teenage girls to skateboarding dogs.With no doubt,they are changing the content distribution landscape and even the popular culture [2].

Established in 2005,YouTube is one of the fastest-growing websites,and has become the 4th most accessed site in the Internet.It has a signi?cant impact on the Internet traf?c distribution,and itself is suffering from severe scalability constraints.Understanding the features of YouTube and similar video sharing sites is crucial to network traf?c engineering and to sustainable development of this new generation of service.

In this paper,we present an in-depth and systematic mea-surement study on the characteristics of YouTube videos.We have crawled the YouTube site for a 3-month period in early 2007,and have obtained 27datasets totaling 2,676,388videos.This constitutes a signi?cant portion of the entire YouTube video repository,and because most of these videos are accessible from the YouTube homepage in less than 10clicks,they are generally active and thus representative for measuring the https://www.sodocs.net/doc/1118292249.html,ing this collection of datasets,we ?nd that YouTube videos have noticeably different statistics from traditional streaming videos,in aspects from video length and access pattern,to life span.There are also new features that have not been examined by previous measurement studies,for example,the ratings and comments.In addition,the series of datasets also allows us to identify the growth trend of this fast evolving Internet site in various aspects,which has seldom been explored before.

We also look closely at the social networking aspect of YouTube,as this is a key driving force toward the success of YouTube and similar sites.In particular,we ?nd that the links to related videos generated by uploader’s choices form a small-world network.This suggests that the videos have strong correlations with each other,and creates opportunities for developing novel caching or peer-to-peer distribution schemes to ef?ciently deliver videos to end users.

The rest of the paper is organized as follows.Section II presents some background information and other related work.Section III describes our method of gathering information about YouTube videos,which is analyzed generally in Section IV,while the social networking aspects are analyzed separately in Section V.Section VI discusses the implications of the results,and suggests ways that the YouTube service could be improved.Finally,Section VII concludes the paper.

II.B ACKGROUND

AND

R ELATED W ORK

A.Internet Video Sharing

Online videos existed long before YouTube entered the scene.However,uploading videos,managing,sharing and watching them was very cumbersome due to a lack of an easy-to-use integrated platform.More importantly,the videos distributed by traditional media servers and peer-to-peer ?le downloads like BitTorrent were standalone units of content.

Each single video was not connected to other related video clips,for example other episodes of a show that the user had just watched.Also,there was very little in the way of content reviews or ratings.

The new generation of video sharing sites,YouTube and its competitors,overcame these problems.They allow content suppliers to upload video effortlessly,automatically converting from many different formats,and to tag uploaded videos with https://www.sodocs.net/doc/1118292249.html,ers can easily share videos by mailing links to them,or embedding them on web pages or in https://www.sodocs.net/doc/1118292249.html,ers can also rate and comment on videos,bringing new social aspects to the viewing of videos.Consequently,popular videos can rise to the top in a very organic fashion.

The social network existing in YouTube further enables communities and groups.Videos are no longer independent from each other,and neither are users.This has substantially contributed to the success of YouTube and similar sites.

B.Workload Measurement of Traditional Media Servers There has been a signi?cant research effort into understand-ing the workloads of traditional media servers,looking at, for example,the video popularity and access locality[3]–[6]. The different aspects of media and web objects,and those of live and stored video streams have also been compared[7], [8].We have found that,while sharing similar features,many of the video statistics of these traditional media servers are quite different from YouTube;for example,the video length distribution and life span.More importantly,these traditional studies lack a social network among the videos.

The most similar work to ours is the very recent study by Huang et.al.[9].They analyzed a9-month trace of MSN Video,Microsoft’s V oD service,examining the user behavior and popularity distribution of videos.This analysis led to a peer-assisted VoD design for reducing the server’s bandwidth costs.The difference to our work is that MSN Video is a more traditional video service,with much fewer videos,most of which are longer than all YouTube videos.MSN Video also has no listings of related videos or user information,and thus no social networking aspect.

III.M ETHODOLOGY OF M EASUREMENT

In this paper,we focus on the access patterns and social networks present in YouTube.To this end,we have crawled the YouTube site for a3-month period and obtained information on its videos through a combination of the YouTube API and scrapes of YouTube video web pages.The results offer a series of representative partial snapshots of the YouTube video repository as well as its changing trends.

A.Video Format and Meta-data

YouTube’s video playback technology is based on Macro-media’s Flash Player and uses the Sorenson Spark H.263video codec with pixel dimensions of320by240and25frames per second.This technology allows YouTube to display videos with quality comparable to more established video playback technologies(such as Windows Media Player,Realplayer or

Uploader

Added Date

Categories

Fig.4.Video Bit-rate

Parameter Peak2The Rest

16583

σ58172

48.6% 2.7%

Fig.5.Date Added

Fig.6.Rank of Views

Fig.7.Rank of Ratings and Comments

to be found by our crawler.Nevertheless,as those videos become popular or get linked to by others,our crawler may ?nd them and get their https://www.sodocs.net/doc/1118292249.html,paring the entire dataset to the?rst and largest dataset,which was crawled on February22nd,we also see the same trend.

E.Views,Ratings–User Access Pattern

The number of views a video has had is the most important characteristic we measured,as it re?ects the popularity and access patterns of the videos.Because this property is chang-ing over time,we cannot use the entire dataset that combines all the data together.Therefore we use a single dataset from April3rd,2007,containing more than100thousand videos, which is considered to be relatively static.

Figure6shows the number of views as a function of the rank of the video by its number of views.Though the plot has a long tail on the linear scale,it does NOT follow a Zipf distribution,which should be a straight line on a log-log scale. This is consistent with some previous observations[3]–[6]that also found that video accesses on a media server does not follow Zipf’s law.We can see in the?gure,the beginning of the curve is linear on a log-log scale,but the tail(after the 2×103video)decreases tremendously,indicating there are not so many less popular videos as Zipf’s law predicts.This result seems consistent with some results[6],but differs from others[3]–[5]in which the curve is skewed from linear from beginning to end.Their results indicate that the popular videos are also not as popular as Zipf’s law predicts,which is not the case in our experiment.

To?t the skewed curve,some use a generalized Zipf-like distribution[3],while others use a concatenation of two Zipf-like distributions[5].Because our curve is different,we attempted to use three different distributions:Weibull,Gamma and Zipf.We?nd that Weibull and Gamma distributions both ?t better than Zipf,due to the drop-off in the tail(in log-log scale)that they have.

Figure7plots the number of ratings against the rank of the video by the number of ratings,and similarly for the number of comments.The two both have the same distribution,and are very similar to the plot of the number of views in Figure 6,yet the tails of the two do not drop so quickly compared to that of number of views.F.Growth Trend of Number of Views and Active Life Span Comparing the popularity of YouTube videos,we?nd that some are very popular(their number of views grows very fast), while others are not.Also,after a certain period of time,some videos are almost never watched.

Starting on March5th,2007,we updated the number of views statistic of relatively new videos(uploaded after February15th,2007)every week for seven weeks.To be sure the growth trend will be properly modelled,we eliminate any videos that have been removed and so do not have the full seven data points,resulting in a dataset size totaling approximately43thousand videos.

We have found that the growth trend can be modeled better by a power law than a linear?t.Therefore,a video’s growth trend can be increasing(if the power is greater than1), growing relatively constantly(power near1),or slowing in growth(power less than1).The trend depends on the exponent factor used in the power law,which we call the growth trend factor p.We de?ne the views count after x weeks as

v(x)=v0×(x+μ)p

Fig.8.Growth Trend Factor Fig.9.Estimated Active Life Span(t=10%)

Fig.10.Rank of User Friends

comparison,since we are only concerned with the shape of the curve instead of the scale.

For each video that has a growth trend factor p less than1, we can compute its active life span l from

v(l)

√

Fig.11.Dataset Size

Fig.12.Clustering Coef?cient Fig.13.Characteristic Path Length

de?ne a small-world graph as one in which the clustering coef?cient is still large,as in regular graphs,but the measure of the average distance between nodes(the characteristic path length)is small,as in random graphs.

Given the network as a graph G=(V,E),the clustering coef?cient C i of a node i∈V is the proportion of all the possible edges between neighbors of the node that actually exist in the graph.The clustering coef?cient of the graph C(G) is then the average of the clustering coef?cients of all nodes in the graph.The characteristic path length d i of a node i∈V is the average of the minimum number of hops it takes to reach all other nodes in V from node i.The characteristic path length of the graph D(G)is then the average of the characteristic path lengths of all nodes in the graph.

C.The Small-World in YouTube

We measured the graph topology for all the YouTube data gathered,by using the related links in YouTube pages to form directed edges in a video graph for each dataset.Videos that have no outgoing or no incoming links are removed from the analysis.In addition,a combined dataset consisting of all the crawled data integrated into one set is also created. Since not all of YouTube is crawled,the resulting graphs are not strongly connected,making it dif?cult to calculate the characteristic path length.Therefore,we also use the Largest strongly Connected Component(LCC)of each graph for the measurements.Every crawled dataset therefore results in2 graphs,plus2more graphs for the combined dataset.

For comparison,we also generate random graphs that are strongly connected.Each of the random graphs has the same number of nodes and average node degree of the strongly connected component of the crawled dataset,and is also limited to a maximum node out-degree of20,similar to the crawled datasets.The only exception is the combined dataset of all the crawled data,which was too large to generate a comparable random graph for.

Some graphs use the dataset size for the x-axis values,so that we can see trends as the dataset size increases.This is very informative,as we are not mapping the entire YouTube website,but only a portion of it.Therefore,some extrapolation as the dataset size increases will be needed to draw insights into the graph formed by all of the YouTube videos.

Figure11shows the dataset sizes and the date they were created on.It also has the strongly connected component size and the random graph size,both of which are very close to the total dataset size for the larger datasets.The combined dataset is also shown,and is given the most recent date.By far the largest crawled dataset is the?rst one,crawled on Feb22. Figure12shows the average clustering coef?cient for the entire graph,as a function of the size of the dataset.The clustering coef?cient is quite high in most cases,especially in comparison to the random graphs.There is a noticeable drop in the clustering coef?cient for the largest datasets,showing that there is some inverse dependence on the size of the graph, which is common for some small-world networks[18]. Figure13shows the characteristic path length for each of the datasets’graphs.There are two factors in?uencing the shape of the graph.As the dataset size increases,the maximum possible diameter increases,which is seen in the smallest datasets.Once the dataset reaches a size of a few thousand nodes,the diameter starts to decrease as the small-world nature of the graph becomes evident.For the largest datasets,the average diameter is only slightly larger than the diameter of a random graph,which is quite good considering the still large clustering coef?cient of these datasets.

The network formed by YouTube’s related videos list has de?nite small-world characteristics.The clustering coef?cient is very large compared to a similar sized random graph, while the characteristic path length of the larger datasets are approaching the short path lengths measured in the random graphs.This?nding is expected,due to the user-generated nature of the tags,title and description of the videos that is used by YouTube to?nd related ones.

These results are similar to other real-world user-generated graphs that exist,yet their parameters can be quite different. For example,the graph formed by URL links in the world wide web exhibits a much longer characteristic path length of 18.59[13].This could possibly be due to the larger number of nodes(8×108in the web),but it may also indicate that the YouTube network of videos is a much closer group.

VI.F URTHER D ISCUSSIONS

A very recent study shows that YouTube alone has com-prised approximately20%of all HTTP traf?c,or nearly10%

Fig.14.Cache Size to Hit-ratio

Fig.15.Accuracy to Update Threshold

Fig.16.Views to Neighbor Views Correlation

of all traf?c on the Internet,with a nearly20%growth rate per month[19],[20].Assuming the network traf?c cost is $10/Mbps,the estimated YouTube transit expenses is currently more than$2million per month.This high and rising expense for network traf?c is probably one of the reasons YouTube was sold to Google.

According to Alexa[21],the current speed of YouTube has become“Very Slow”and is considered slower than81% of the surveyed sites.This situation is only getting worse. Scalability is no doubt the biggest challenge that YouTube faces,particularly considering that websites such as YouTube survive by attracting more users.In this section,we brie?y discuss the implications of our measurement results toward improving the scalability of YouTube.

A.Implications on Proxy Caching and Storage Management Caching frequently used data at proxies close to clients is an effective way to save backbone bandwidth and prevent users from experiencing excessive access delays.Numerous algorithms have been developed for caching web objects or streaming videos.While we believe that YouTube will bene?t from proxy caching[22],three distinct features call for novel cache designs.First,the number of YouTube videos (42.5million[3])is orders of magnitude higher than that of traditional video streams(e.g.HPC:2999,HPL:412[11]).The size of YouTube videos is also much smaller than a traditional video(98.8%are less than30MB in YouTube versus a typical MPEG-1movie of700MB).Finally,the view frequencies of YouTube videos do not well?t a Zipf distribution,which has important implications on web caching[23].

Considering these factors,full-object caching for web or segment caching for streaming video are not practical solutions for YouTube.Pre?x caching[24]is probably the best choice. Assume for each video,the proxy will cache a5second initial clip,i.e.about200KB of the video.Given the Gamma distribution of view frequency suggested by our measurements, we plot the hit-ratio as a function of the cache size in Figure 14,assuming that the cache space is devoted only to the most popular videos.To achieve a60%hit-ratio,the proxy would require about1GByte of disk space for the current YouTube video repository,and nearly8GByte for a95%hit-ratio.Such demand on disk space is acceptable for today’s proxy servers.

Given the constant evolution of YouTube’s video repository, a remaining critical issue is when to release the space for a cached pre?x.We found in Section IV-F that the active life span of YouTube videos follows a Pareto distribution,implying that most videos are popular during a relatively short span of time.Therefore,a predictor can be developed to forecast the active life span of a video.With the predictor,the proxy can decide which videos have already passed their life span,and replace it if the cache space is insuf?cient.

The life span predictor can also facilitate disk space management on the YouTube server.Currently,videos on a YouTube server will not be removed by the operator unless they violate the terms of service.With a daily65,000new videos introduced,the server storage will soon become a problem.A hierarchical storage structure can be built with videos passing their active life span being moved to slower and cheaper storage media.From our30thousand videos dataset (Section IV-F),we calculate the predictor accuracy from the number of videos that have an active life span(according to equation2)less than an update threshold divided by the total number of videos,which is plotted in Figure15.This result facilitates the determination of an update threshold for the predictor with a given accuracy.

The cache ef?ciency can be further improved by exploring the small-world characteristic of the related video links(see Section V-C).That is,if a group of videos have a tight relation, then a user is likely to watch another video in the group after?nishing the?rst one.This expectation is con?rmed by Figure16,which shows a clear correlation between the number of views and the average of the neighbors’number of views.Once a video is played and cached,the pre?xes of its directly related videos can also be prefetched and cached,if the cache space allows.We have evaluated the effectiveness of this prefetching strategy,which shows that the resultant hit-ratio is almost the same as that of always caching the most popular videos,and yet its communication overhead is signi?cantly lower because it does not have to keep track of the most popular videos list.

B.Can Peer-to-Peer Save YouTube?

Short video sharing and peer-to-peer streaming have been widely cited as two key driving forces to Internet video

distribution,yet their development remains largely separated. The peer-to-peer technology has been quite successful in sup-porting large-scale live video streaming(https://www.sodocs.net/doc/1118292249.html, programs like PPLive and CoolStreaming)and even on-demand streaming (e.g.GridMedia).Since each peer contributes its bandwidth to serve others,a peer-to-peer overlay scales extremely well with larger user bases.YouTube and similar sites still use the traditional client-server architecture,restricting their scalabil-ity.

Unfortunately,our YouTube measurement results suggest that using peer-to-peer delivery for YouTube could be quite challenging.In particular,the length of a YouTube video is quite short(many are shorter than the typical connection time in a peer-to-peer overlay),and a user often quickly loads another video when?nishing a previous one,so the overlay will suffer from an extremely high churn rate.Moreover,there are a huge number of videos,so the peer-to-peer overlays will appear very small.1

Our social network?nding again could be exploited by considering a group of related videos as a single large video, with each video in the group being a portion of the large one. Therefore the overlay would be much larger and more stable. Although a user may only watch one video from the group, it can download the other portions of the large video from the server when there is enough bandwidth and space,and upload those downloaded portions to other clients who are interested in them.This behavior can signi?cantly reduce the bandwidth consumption from the server and greatly increase the scalability of the system.

Finally,note that another bene?t of using a peer-to-peer model is to avoid single-point of failures and enhance data availability.While this is in general attractive,it is worth noting that timely removing of videos that violate the terms of use(e.g.,copyright-protected or illegal content,referred to by the“Removed”category in Section IV-A)have constantly been one of the most annoying issues for YouTube and similar sites.Peer-to-peer delivery will clearly make the situation even worse,which must be well-addressed before we shift such sites to the peer-to-peer communication paradigm.

VII.C ONCLUSION

This paper has presented a detailed investigation of the characteristics of YouTube,the most popular Internet short video sharing site to date.Through examining the massive amounts of data collected in a3-month period,we have demonstrated that,while sharing certain similar features with traditional video repositories,YouTube exhibits many unique characteristics,especially in length distribution,access pattern, and growth trend.These characteristics introduce novel chal-lenges and opportunities for optimizing the performance of short video sharing services.

We have also investigated the social network among YouTube videos,which is probably its most unique and 1A very recent study on MSN Video[9]has suggested a peer-assisted VoD. We notice however that the statistics for MSN Video are quite different from YouTube,and the technique has yet to be substantially revised for YouTube.interesting aspect,and has substantially contributed to the success of this new generation of service.We have found that the networks of related videos,which are chosen based on user-generated content,have both small-world characteristics of a short characteristic path length linking any two videos,and a large clustering coef?cient indicating the grouping of videos. We have suggested that these features can be exploited to facilitate the design of novel caching or peer-to-peer strategies for short video sharing.

R EFERENCES

[1]YouTube serves up100million videos a day online.[Online].Available:

https://www.sodocs.net/doc/1118292249.html,/tech/news/2006-07-16-youtube-views

video-sharin [3]W.Tang,Y.Fu,L.Cherkasova,and A.Vahdat,“Long-term Streaming

Media Server Workload Analysis and Modeling,”HP Labs,Tech.Rep.,

2003.

[4]J.M.Almeida,J.Krueger,D.L.Eager,and M.K.Vernon,“Analysis

of Educational Media Server Workloads,”in Proc.NOSSDAV’01.

[5]S.Acharya,B.Smith,and P.Parnes,“Characterizing User Access To

Videos On The World Wide Web,”in Proc.SPIE’00.

[6]H.Yu,D.Zheng,B.Y.Zhao,and W.Zheng,“Understanding User

Behavior in Large-Scale Video-on-Demand Systems,”in EuroSys’06. [7]M.Chesire,A.Wolman,G.M.V oelker,and H.M.Levy,“Measurement

and Analysis of a Streaming-Media Workload,”in https://www.sodocs.net/doc/1118292249.html,ITS’01. [8] E.Veloso,V.Almeida,W.Meira,A.Bestavros,and S.Jin,“A Hierar-

chical Characterization of a Live Streaming Media Workload,”in Proc.

SIGCOMM’02.

[9] C.Huang,J.Li,and K.W.Ross,“Can Internet Video-on-Demand be

Pro?table?”in SIGCOMM’07,2007.

[10]YouTube:Video Format(from Wikipedia).[Online].Available:

https://www.sodocs.net/doc/1118292249.html,/wiki/Youtube#Video

Understanding the Characteristics of Internet Short Video Sharing YouTube as a Case Study

相关文档

最新文档