I made 60fps Flipbooks using A.I.
Film & Animation
Introduction
Have you ever wondered what your flip books would look like if you could enhance them to a smoother frame rate? In this experiment, I decided to take a few of my flip books, originally designed to flip at about 12 frames per second, and run them through AI frame interpolation software, aiming for a polished 60 frames per second. While the concept is intriguing, the results are worth examining closely.
Understanding Frame Interpolation
Frame interpolation technology allows us to artificially increase the frame rate of footage or animation. For example, we can take initial material filmed at 24 frames per second and elevate it to 60 frames per second. Although 60 FPS appears to be smoother, it doesn't always guarantee a better viewing experience. Personally, I find that artificial motion smoothing on TVs can often look unnatural and is a feature I usually disable. However, I was curious to see how this process would apply to my flip book animations.
The leap from 12 FPS (the original frame rate of my flip book) to 60 FPS is significant—this transformation requires interpolating five times the number of frames. To visualize it: for every real frame, the AI must generate four artificial frames to fill the gaps. This means that a whopping 80% of the final animation would be comprised of synthetic frames, so the outcome could be quite fascinating (or bizarre).
The Experiment Begins
I started with my pencil flip book, utilizing a tool called RIF for frame interpolation. The initial setup was more complex than anticipated, and as I navigated the various settings, it became apparent that achieving my goal was more challenging than expected. Here were some factors to consider:
- The video of the flip book was recorded at 24 frames per second, leading to redundant frames.
- The movement of flipping pages created blur in some frames, complicating the process.
To address these issues, I decided to create a new, clean video of the flip book. I filmed it at a slow flipping speed using a high shutter speed to ensure each frame was sharp. Next, I imported the new footage into Final Cut Pro, segmented it into individual frames, and carefully deleted any frames that contained blur or duplicates. With a clean video at 12 frames per second, free of imperfect images, I was all set for the AI process once again.
However, there was a caveat: the interpolation software required a robust video card, or the rendering speed was agonizingly slow. But after a long wait, I finally had my results! Watching the output for the first time revealed some quirks; the title exhibited some odd "jiggling," and there were strange frames throughout. While there was a replacement of motion and smoothness, I noticed that a sense of crispness inherent in the original animation seemed compromised.
Continuing the Experiment
I decided to try the process again with a couple more flip books, this time starting with clean, flat stop-motion videos. Once again, I ran the footage through the AI to generate 60 FPS versions. The output was generally intriguing, though still contained unnecessary anomalies. In particular, I found that while the animation felt smoother, it lacked the punch and intention of the original work, with overlays creating a wobbly effect resembling Jell-O.
In one final test, I ran my extensive "Return of Grumpy Cloud" flip book through the interpolation software. Frame by frame, it was fascinating to analyze the output. Despite some imperfections, it was remarkable to recognize that an overwhelming majority (80%) of what I was observing was entirely generated by the AI—nothing that was originally present.
As a traditionalist, I prefer the original animations. AI-generated smoothing falls short of replicating the animator's original intent, resulting in losses in timing and overall impact. While the balloon flip book yielded the most satisfactory results, the technology remained an interesting exercise rather than a viable substitute.
Conclusion
This experiment has been an eye-opening exploration into the intersections of art, technology, and personal taste in animation. It’s essential to appreciate the craftsmanship behind original pieces instead of relying solely on technology to redefine our perspectives.
Keywords
- Flip books
- AI frame interpolation
- 60 frames per second
- Motion smoothing
- Animation
FAQ
Q: What is frame interpolation?
A: Frame interpolation is a technology that artificially increases the frame rate of video or animation by generating additional frames between existing ones.
Q: Why did you dislike AI motion smoothing?
A: I find AI motion smoothing can appear unnatural and detracts from the original animator's intent, diminishing the impact and timing of movements.
Q: How did you improve the footage quality for processing?
A: To obtain cleaner footage, I filmed the flip book at a slow speed and utilized high shutter speeds to eliminate motion blur, then removed redundant frames.
Q: What is the final output of the project?
A: The final output consisted of 60 FPS versions of the flip books, showcasing both the enhancements and the quirks introduced by AI interpolation technology.
Q: What was the most successful flip book in the experiment?
A: The balloon flip book turned out to be the cleanest and most satisfactory version after going through the AI frame interpolation process.
