ON HIGH FRAMERATE CAPTURE/EXHIBITION
February 10, 2020
[In response to an email inquiry asking whether cinema may need to go to high frame rate capture/exhibition in the future because the younger generation has been conditioned to dislike the 24fps look that we (the older generation) experience as cinematic and to prefer instead a smoother look that we experience as a junky "video" look. And additionally asking if any studies have or should be done on this subject.]
Hi, ----
I don’t have scientific data or know of any clinical studies on your direct question, but I do have some hard data about some underlying issues that can help in thinking about this topic. And I also have some non-data-driven speculation on your direct question. Here are some thoughts:
1.
It is known that 24fps is not enough to create the illusion of perfect smoothness in the human visual system and is on the edge of acceptability for persistence of vision to create realistic (non-stuttery) motion. This is certain, because if this weren’t true, we wouldn’t be able to tell the difference between 24fps and, say, 60fps, since they’d both look indistinguishable to us from perfectly smooth (non-quantized) motion. 24 fps was created as a “good enough” standard.
2.
This item is speculation:
On the one hand, it is likely true that we (“we,” the older generation(s) including myself) have been conditioned since childhood. We recognize that all the artful/meaningful/big-time cinema has the 24fps look while soap operas and home videos and talk shows and news have a different look to their motion. On the other hand, the fact that we’ve been conditioned differently than younger generations, doesn’t *necessarily* mean (though it might) that it’s a pure accident of circumstance. It may be true that imperfection and distancing from reality would be perceptually aligned with artfulness in a way that clinical perfection is not whether or not we’d been conditioned by conventions of the TV and cinema industries.
3.
Even if it's true that there’s is a generational conditioning that forces some or all young people to dislike the very look that to me gives images weight and artfulness, it’s still not clear that it follows that we need higher frame rate capture and exhibition to please them. I personally can’t tell the difference between motion smoothing and actual high frame-rate capture/exhibition. They both look the same and equally terrible to me, so maybe they both look the same and equally great to a younger person who doesn’t like the 24fps look. I’d be less interested in a clinical study of 24fps versus 60fps (or whatever high framerate) than I would be in high-framerate versus motion smoothing. The former is too fuzzy and aesthetic and subjective to be able to define the scope of a fair scientific study (what would we even test that could be meaningfully quantified?); the latter is a very clear test that could be done to a high degree of certainty in a clinical double-blind situation.
4.
There are a LOT of practical reasons not to go to higher framerate capture/exhibition whether or not we throw over the 24fps look for a smoother look using motion smoothing instead of actual high frame rate. These include but are not limited to:
4.a. There is always desire to shoot slow motion. Sometimes *very slow* slow motion. And we already struggle with cameras not going as fast as we wish. If the playback framerate is high, the slow motion is not as slow. For example, something like 200fps is the highest framerate we can currently get out of a high quality cinema camera (as opposed to a specialized camera that’s expensive to rent, cumbersome to use, and kind of crappy for image quality). If our project framerate is 24fps, then 200fps is an 8.3-times slowdown, which is a lot! But if our project is, say, 120fps like Ang Lee’s last movie, then 200fps is is only 1.6-times slow down, which isn’t much. So, that’s a huge difference. Even if cameras get faster, then the problem isn't solved, because the goal post has moved: the higher framerate project still has more restrictive slow-mo than the lower framerate one.
4.b. Data rate. We already struggle with data rate to the end user: whether it’s ISP bandwidth when watching Netflix at home or the limited data rate of DCP for theatrical release, bandwidth is a real bottleneck. Viewers are already seeing horrible macro blocking at home (and are right on the ragged edge of not seeing compression artifacts even in the theater). If we increase the frame rate, it’ll have to be compressed even more. Since the internet provider bandwidth is already the bottleneck, data rate can’t increase proportionally with frame rate. So, for example, 60fps content will be 2.5 times more compressed than the already-too-compressed 24fps content, creating even worse compression artifacts than the unacceptable amount we already have.
4.c. There’s another compound problem: because bandwidths to, say, smartphones on cell networks is even more constrained than for, say, streaming services like Netflix when viewed at home, there usually needs to be even smaller files (than the already-too-small files) ready for delivery to these even more constrained viewing systems. If you take a master that starts at 2k or 4k and make a version that has smaller pixel dimensions for these low-bandwidth situations, they look “perfect” in the sense that, for example, a 720-pixel-width image made from a 4K source looks at least as good as what it would look like if the source was 720. That is: although downsampling is itself a degradation, it’s not a compound degradation in that case. This is very much not the case with framerate. If you downsample high framerate movies to 24fps by either dropping frames or doing simple frame blending, it looks noticeably messed up, because the motion blur is wrong and (if doing the frame blending) you also have a bizarre-looking feathered multi-image. So, the only way to even sort of get away with this type of downsampling is to use a really complicated, adaptive algorithm (not a straight-forward algorithm like the various resizing algorithms) that may or may not look good on various content. In other words: color and spatial downsampling is of course a degradation in that it’s downsampling, but it doesn’t have the *additional* degradation of being worse than if the master was already the lower sample size. Framerate downsampling *does* have this compound degradation.
4.d. Combining item 4.c above with item 3 above: We see that bandwidth limits always demand downsampling and compression of spatial fidelity, of color fidelity, and of temporal fidelity. So, it seems like a bad idea to inflate the master data-size way up in a new manner we haven’t previously been doing that will cause visible problems when it’s necessarily sampled back down for most real-world use-cases. Especially if we can start at a reasonable (non-bloated) sampling rate and then use an algorithm to take it the other direction. That is: start with a high quality, but non-temporally-bloated master, then inflate it temporally when it arrives to the end user. Instead of starting with something that’s absurdly bloated and making a doubly-degrading deflation before transmitting to all but a small minority of privileged end users.
5.
All of these concerns about unnecessarily bloated data sizes also have an environmental impact. Data centers are now a major contributor climate change. It seems irresponsible to inflate the data size by a factor of somewhere between 2 and 5 without gaining any intra-frame image quality (in fact degrading intra-frame quality in many use cases) while gaining nothing but a subtle change to the feel of the inter-frame motion. And doubly so when that change of feel makes the image actually look worse to a huge swath of the population and, for the rest of the population, the desired effect can be achieved on the display device rather than in the source file.
Also, this isn’t an additional point, but just a way to really hammer home the point I already made that motion smoothing can be used instead of high frame rate: content is almost ubiquitously captured and disseminated at 24fps. So, if it’s true (is it?) that the younger generation has been conditioned to prefer “smooth” motion in a way that’s contrary to my own personal conditioning, it’s a certainty and not mere speculation that in the case of photographed imagery as opposed to video games (which is our very topic here) motion smoothing can achieve that look that they prefer and that we don’t need high frame rates… because the very thing they’ve been conditioned with is motion smoothing and not high frame rate.
-Steve