Smosh Ultimate

Sure, playing video games is fun, but what if we could have even more fun with it?

I’m using MaxMSP/Jitter to modify the output of the Nintendo Switch. More specifically, I’m messing with the video output of the console while playing Super Smash Bros. Ultimate.

The goal of this project is to modify the video output in fun ways á la data moshing and audio-visual feedback.

Check out some more smosh videos on Instagram.

So far I’ve managed to create a nice system for capturing the switch video, mess with it’s chroma- and alpha- levels, and sync them up to some brutish sound detection triggers.

The Max ‘control panel’ of the video manipulation looks like this:

You may be able to see there are a few setup steps, then some controls for switching color planes, selecting colors for chroma-blending, as well as music controls.

The video can be triggered by sound by detecting when the amplitude of a song passes a certain threshold as defined by the yellow bar above it:

I’ve now also modified the patcher to use the beloved-by-community jit.world object in an attempt to reduce lag and create an opportunity to use shaders & graphics-intensive video effects.

Overall, the whole setup now looks like this:

As you may be able to see, even with jit.world, the modulated output is not very high-res, though it is real (or near-real) time and I had fun playing with the live effects running on their own.

I think this issue may be due to the video capture card I am using, or how I am getting the signal into Max specifically.

More technical details below. Stay tuned for more updates as I progress with this project!

Technical Setup

Here’s what the full setup looks like:

…and here’s a diagram of what exactly is going on there:

If you want to try this out for yourself, you can start with project’s Max patches from this GitHub folder. The current best file to use is the smosh-world file.

Any and all feedback is welcome, contact me here.

Previous Iterations:

First viable iteration (handsfree-with-sound patch in the repo)

control panel (before I rediscovered my love for weird fonts)
A/V setup: Before using jit.world I couldn’t make the output jit.window both large AND realtime, so I was only using two monitors and reviewing the output in that small pwindow at the very top.

Credit where credit is due

None of this would be possible without the resources available on the web and through my education. Specifically:

Other resources that helped me and may help you:

Leave a Reply

Your email address will not be published. Required fields are marked *