In the Beginning...

QoE sits at a confluence point between several disciplines, and one of those is QoS. Back when I started working on these topics, "QoE" was not a thing, we (networking people) would talk about "QoS from the user's perspective", or similar kludgy terms, for lack of a proper one. The underlying concept, however, was clear: how does QoS in the network affect the user's perception of media quality.

Back then, media quality had an implicit narrower meaning, too. VoIP was a new thing, and doing real-time video was mostly a research topic. Times change, technology advances, and QoE has taken a life of its own, evolving into a wider, much more inclusive concept spanning a much wider range of application domains.

For some applications of QoE, however, the link to QoS, and more generally, performance, remains strong. In particular, when dealing with monitoring and management, we are most often interested in understanding how the performance of the network and other resources affect the users' QoE.

We typically see QoS as a lower-level construct, which in practice provides one of the foundations of QoE; hence the plethora of so-called QoS to QoE models in the literature. These map stuff we can measure (e.g., network QoS), and stuff we know (encoding parameters, buffer sizes, etc.) into an approximation of how an ideal, average user (or if we did our homework and put some effort, a population of users) might experience the quality of a given service. It seems clear that, in a sense, QoE sits atop QoS, but is it really so?

A Different Perspective

Over the last few years I have often worked on QoE management ideas, and over the past year in particular, I have focused on quality monitoring for WebRTC platforms. One common idea that comes up in this context, is that of "service-level QoE", or "the QoE of the system", or similar terms. Like in the old times, we seem to lack a proper term to describe what we are after, despite it being a rather intuitive thing.

On the other hand, based on the definition of QoE, it seems rather absurd to speak of "the system's QoE", since QoE is all about the user. A system (or a service) cannot experience anything, it has neither expectations, nor an emotional state, nor a sensory system, for that matter.

So when we speak about "service-level QoE", we are, at best abusing the language, or at worst, severely confused about what QoE is. Since most of the people I know who work on these topics are pretty knowledgeable about QoE, I'll go with the first explanation. We are abusing the language, because we lack a suitable term for what we are working on.

Let us illustrate this a bit with WebRTC. At callstats.io we provide a WebRTC monitoring service; our users often have several WebRTC applications, and each application often sees hundreds, or thousands of calls a day. Calls can have many users, and each of those users will experience a certain quality while in the call. So far, so good, we have users, and each user's QoE can be estimated to some degree (remember kids, WebRTC is a hard problem, QoE-wise!).

But our goal is to provide a call QoE value, and furthermore, an application QoE value to our users (who are not the actual users on the call, but the service providers). While the terms themselves are rather nonsensical when going by the QoE definition, their intent seems quite clear: we want to know if a call was good or bad for the users, and we want to know this for all calls across a given WebRTC service, to see if the service itself is performing as expected (i.e., providing its users with a good experience!).

Troubles ahead

This pattern repeats itself in most, if not all of the QoE management literature, where the goal is to understand, and improve the performance of a service, and said performance is dependent on the quality experienced by the service's users.

Behind this seemingly innocent idea, there lies a mountain of complications, unfortunately. Our understanding of this "Service-level QoE" can take different forms, depending on the type of service, our goals, and a host of other factors.

The main idea boils down to figuring out how to create quality aggregates that allow us to build a notion of service quality on top of the individual users' QoE. When digging a bit deeper, we soon encounter temporal issues (pooling QoE over varying time scales), modeling issues (MOS vs. quantiles/distributions), finding trends in distribution variations, etc.

What's next?

I suspect (and hope) we will see more work on this topic over the coming years, eventually converging towards a proper definition for this idea of Quality of Service (nice term, but too loaded to use, isn't it?).

Working on WebRTC has yielded some insight on aggregating quality, but it's not yet mature enough, nor general enough to put it out there as a solution candidate for this problem.

For the time being, service-level QoE aggregates might be a good first-order approximation to putting a name to this idea, though a bit lacking in the marketing department.