Categories
001. Caleb's Favorite Posts 201. AV Mashups 991. Live Image Processing 994. Sound and Video

I’m a VJ now

Over the past few weeks I have been working on my first ‘patch’ – an interface for controlling an audio-visual performance. Here is a still-shot of something made by the patch:

A patch runs in program called Max – also known as Max/MSP/Jitter. It is my first use of a new tool to link sight and sound, as I have done with p5, processing, and animation in the past.

UPDATE: I performed using this patcher

Note: patch code is at the bottom of this post. You can just copy it and paste it into your own version of Max like so.

The A/V manipulation is done with a control panel specifically designed to create some chromatic and fade effects between two videos. It looks like this:

How it works:

This patcher takes in one audio source and two video sources to create a mashed-up result with sound influencing part of the video.

The sound signal is played aloud and also converted into a matrix of data using the jit.grab object. The series of operations that happen after that are designed to imitate the data flow below:

using FFT to create a frequency selector

The steering wheel is intended to select a set of frequencies to be processed (i.e. 0-to-128, or 896-to-1024, or even something like 193-to-321. That selection of frequencies is converted into a set of RGB values. These values are used as a background in the video.

The video layering creates a lot of trichromatic effects such as these by using two sets of jit.chromakey manipulations.

Patch details:

While the control panel looks nice, here is what it looks like “under the hood”:

under the hood of this Max patcher

The patch also uses gates to route the flows of the (three) matrices to the appropriate place. As the control panel shows, this patch is designed to have 5 “stages”

[Intro] One video, plain and simple, with music in the background

[Act 1] One video with the sound matrix chromakeyed behind it

[Intermission] back to one video, plain and simple

[Act 2] One video with another video chromakeyed behind it

[Act 3] The same as stage act 2 above, but with the second video chromakeyed like the first is in act 1

Max code:

<pre><code> ----------begin_max5_patcher---------- 6462.3oc6cs0jaaik9Y6eEbTk211xD2Imp1phimdSRsdrcY6L6lM6VcwVhca FSIpgjpsSlZxu8E23M0jhfjfRs8XmXYIRId.9vA3bEG7Od7iVbcxmByV37mc 9EmG8n+wiezijWRbgGo+7iVrI3SqhCxjesEaC+Xx0+5hKT2JO7S4xKmEFGtJ 2AUbis62DsMNLW9if5KtKHe06i1d6Uo7uqhnLBcouqqKDegCvErz0Cg.nKbH vktW3.4u57+U8HS1me3yLZsj971zS7nETW88x+scgJpr35fs2t3BmEKJeZY4 +Vr7tKVHtv+7wOV7xEFhBqR1rIba98fgWmFlwudPdTxVmMIqCchxC2jU78hi 1FtJY+1758fFPEnanBfHtKcATl.qfTWA.4IeEg5DlbuOLwJGjlIL3MW9125b 4ae9yd8k+o+Tws2UCYtpUbnw2nFGhunK56sjv6trkPx86uFBe.2kDM7gckOU .vW7Xgt8he0ab0ITcbkUGJVkDmjpnr6RlGFA3j0cI.Sb88DuyEhHPedanFsm mAj2lGbanyyS1lmlDGGlt33.NFyDXCuEqfHZaPzCDHGu3DfbO8+c66h1DJXZ O5D4IxhSvx07nsNkd33MxmVCuMXIBSva37h2u3Ge8q6g6ziHELPIKQHknBe+ I.Xt3xkTK.LH0v0TM.vnkhjtgO8KK52kXFjSz4ku8x28S8AjL7RfXwTtPExC bPz8r.huH7lbm+cmK2dqXQf+Zv18AwNu5tvzzn0yzBAJlaDlVwb6YIIdywxA X+4c4.Ml+lu+6dlQqJvQrkX.kBok.2CS4VXu4E39aQqcfNO+8oIaB5A4P.xR Fuiw0tTqm0CXXCM+vFvLXCvUEEwHtfOWfNKI4dSXVFe4v6Ac2rONNaUZX3Vm uATe45sAaTT6YoQAwC07PewLZs4gd.oXFfXYQOoJpvtWEqEH.vZSLBm.tcY0 nsrVLO41aiCWLPdh58clRqPrewql2sQkpfrKHkOXjGldU31fqU8mN66QBAV1 o62gKC14DlsZfrJtFBWHopzJEpGFWBxanbIReKzFTUusFlpAHMBwe7QwgbkI xZNm8QKB1sq1keTseh.V+UkQkrKJuTzV0kPkWJM7tnheOt7pAobDKmCW6SUs uOU5uDwiIYcX518Qxlh5h7A3GW7.K.YnbkNpjMjhU7jd0VfhyJbabxpODtt1 nEGy1EtMZ6gKSUd60g2DrON+pFHNbYq2+lfUgc9iakC5QKtkqwVxVQinARKt bA43KAK6ZhWq5Lxuw1fcs7i47JbToialw6j6ytNHULPnmoAKtYdRRbyaU96h 4pbpu8tnsaO.EyS108MSit88G42dcB+laN1yVdmrq1uUc2q3SeyuJK3tlncd Pbrd9byG+mB1FsIHOLORMD.cKuoZ0l2yEOjDG2n+ptycsbm0bd3UgeLZc96k DpNy.+qGsqfIZQ4n75naCyxadMt86YMux8lexuz9q0yQuJObytXdun4WngSR qOgr9JeMt9Aq.pVynb1VGBA5ZkMnGV4LT0+PJdsjOsSMCtmGRq2DtORHtZgt QGd8nsqC+TsUGzRAzKTLRbogjg64PYGHq9s5dBdWZTzCfBPZGcIAT2iBnv1A T3gMvVV9xLGR2lb14Gk+P3uMRH10LHFHkQPtmlrs.w31gXvXgXgFLWz1+L.H uRTfvL9NVCP18kdGr0whrj8oqJZTZlFml.AeMs7nskBF+kxYqhumQbBCsM.L rM.6rMTWjOWWlXoCuup.+4hNCVGUYjUyuk7UPg2wwBGgK8NNlBYXh7c9dtnl dGukGAr3Q.8cA9TwOD4x+kX063WBz2iXROA9jXsvcVMIT0+F6RS1kjVpwyRj +iqqa0Bg.10WoTl3pf77znq2mqXzpqr3fD5w0.65fXsHsxoyGSBXkXxGWMJa Gs7+0n7kejK7H4iNe6MwIb9qs2xMx9aEyhcPPWGH1cnVF4A.KgT9eDVBhUxC UKzLr.FBHmr.F1A57u4.FbDSckV0fbUhsTqrBGjgvfEmOq93r.qENQvYvi5T BcIfOn6w3cXojaVeAItsNOEsXts2uKCdSC4FJr14tn0gICs2CfH7RhuuOiuH Ek3V5nSsqPFFHPXedBB.n2RFU4lroCBzOOAAWd+zGgPDafAjOKwf5N6gRvKY Xeeg5BiCCveVhA0kBNcLX9WUrKWGCGpDPOeuxAeDFv0oh+G7HkEB8Oac7gK5 m4auNNx8bw0qyUL7jF2Atkc+wjqX.D7bq6mFFbmBW.zmMIX.d10.91f7PGxv UBlnhKtubwNJnmtNoMNfiH56hC96I.CfSZ0.ENfwiYl.633vCw9tO2HHqz28 Nm8cgswqjQatxObFCA.rOpxf.2Z5ABFCPzsXPdq7pMA4oQe5AOjzvFIf2zfD L3KAHogESSFRHOLfjTN4Cap07zy0f55SypYWEaLdVB2Zh+U4x54c8FguWStk iR65KWe48YhHiST5NH4I7fiHAVp6bFVY.Ut+CqEkRMH6U.X1nRhgJ775aa5s WOBSkP1PBeT1W9N.ChXclP1Esp7vMETnL+C0CuEZYbWP79Feg0QYh15ZM4un 1+q7s9.GeuM6iQ7gf9xQIkqQIPU35QdRwjztFcoE8vs612Tqo1Fu0JdwGeKl mzAAF6Ht+DGwe3sVbcE1vT4jjQsJLAdhWEtq7k64+vad0e8Y+mW9yNO+Uu7c u4Uu3sC2qlDhVAdhdm+.HzCiGa+6oE5LuQ.9a+3e4xW47hm8yW9le7keuU1N OL9nNobi7.b8F9N4gBl2d8y1yEizmDDEiLQt+bTIFznx.RrGRk6yc+bFWlOV 4p6YKoQCSprfoqzrEAjFrNYbBgYxD3193D3TfSfdvIHWSDOcd0NYrBpAc6CV UJBOuAICLzET4JwrTnCKCxwO.oRI1QDpLL9r4ZzhMJmXGyQndTj3M.FEy3pm 05xu3d26E3dzVx2cIEpynawhZZ4Q5rbrS9ui4kBJ6.gas+vFoJRsYgAafYyr 9JkYqQMRcfJyhbgv0SoxL12EJ2Ci5L2ntJy5e1PSMCixhjijDIl27lXNjL4T Ho2LHomDHosmx97j5Cg7GkUmLRwbtWF+wRkcQwNZk5x7vy1jQH2zEKOYD3hv y3rQ3ob1nZm4V+ky6zui1d957sANeSRCmuA37MPmuA47MCNPY9Tozd.23bg5 O.cpBMHg+PvYKn3AefCnk++.cvG9HvhJBY.hqBV.nQr8PfmqMQTGn09s6BV8 gwhVfdUdA.7JW802qG.C0BfQFJfIyRPg6KN3Mmn7RCTl04FCUDt10TsyOIba NJ8wNY3y6Pms4cUYl2RhcJXI.W4dKudccAqbyAYPPB9rCIC1NL.RsEzp22Gi MXn4Oub5rHB7pe5k+Emm8xm8he9+4x2L34DPo06Z+6AHng6TOn2hGlYhJ.nl uSwS2HaHb1GfyhiV2aUegx0xFI1H3HWVYGRn4fXPzqdHQJp+BR+zNgEJ8ONc Fsl3zyZfC5rZR7re5kO+G5cPPoZBlNQOewkuJGKsseu.34cajK1Ik01.4lpv q14w.WlpP5LBecM6yB2yYK12aoWBUUrqDROOrenplXpFV11jOBWtAemPEoKD uHtB5dWAbuqfabkcgo27j6PK4eufOsSowk5F6yexlvmj+9z8KCht4l68Kf26 Wnug6g23pj30s9b2rCs33b2Bas1kraew1xr+XHvzLC8E2dj4789idOzerLHq h92D8IcEOYE+Gkj8dme3u7me5OkEll8zUAwgWeSX5s6yR19zuOIgaA6SVmFc W3SyV89jj3m7wjzO7zW9y+zO9tW+TnKv6IfmJpjROUsEfxdpbXA7TaNQMMLX 8HlnpidQwDUvHxT4Sf.yiD72MI2EE57s2kDOb0FJ579xbtvmLhP+BQm3P+Zx pVEkPlutr03V1RyWLCKaA.z+UcgKSXbqWDe9Jy63Xdgv4RlKnZ6Y9Uotmsvh h.phQkJE6mR7W.bsyJyDUrJsCsW7WveMbneMbneoFNTDCaoIgPh6LNIrZWD7 0nf90oYm4nf1PfCviTle5iJdn0BKzWLADsIBQPUHzXBMJf9uJgFsIto10DiI 1nfJ2o9.I3nclIv843ZeeUzCfLuxZWrtRkMF+Wy0coJPAz9djiyW19T63K6q 2mmyIRO.DFH8QhmNauok+SYGQzKj5DLhCNfVw0q4OtOLpGWaNoPshfp1BfKd c5pL3CVXVg0b56fGKNe365Y3lqyMWvLPFbKnGvFSIvhzgt3zQgAlkoDdy71n 366ySh9jk97NIVr018vxk.jP33gMjdmhSX883FIjMy0j52zWByigkYJokfL. Arjy6pxg74B1rT8n1nXaCcKissugw1lL3Xa2Ho.78mmXay.m0faaDZKbmsmf iDNu.NjvVBDZr5Oi.N8A41PrJujVtXva6LodHLIFAPtC2RHp2CiLVgHJ4eXk LVy4yFdRqfA9kBamM9Lh2YkOyrMTbCDu1gThFIPcpQJxLzkh58YNDzsoqipC 19eF.1MVF0Rfc8kLOYfM7y.vtRCAKgz00FvxHcsXdvNqP6MwI7l9QE.cSR5l f75UWfikmvbilJz0jLpx4p63viKNxg3fMOiTBVGNhbloDdpoc+nxbF77K31Z rDEoOsHSHKL5cT0gE3CZVh7j3wvQTfNdGtW1FJGg6YK6xe8ad0+8O671Kewk O+cu5MNN+G7Wd66tTVeEb9u9gKu7EVYOHfXROVgTkHQcZnOrpr.blOkydinB nkbipbFkYzAcltd8zwID0CiSbJHZVyd+m5.WN3BvBTeTm4O1RmMr6ILV0O7F Y.jlWPeNspMkC2hUO3AaySAR4RqalX+N7k+zUQBjL.FE7YUKlNyvUwzxuArb 3EgNu5bYrQjL5vyVgoNSj1SNOwcITLF53JbLyQSogVqfTsZlmJ5hDtxKXFUZ Ds78xEjoCSrk+WbvCRXXlsfGuu7fGWf8fm4eG0INGO1GMx3MK9wpv9GGT.YG YdEVUMfT4GjJTzFD19VNI0ZXXoa8w1qJOFt3RbvPeeezYJP+1AXkMrir.NWk QtR0xfPU.r5SWjICrHV6.qvxWnqKj9vJCJj63A8dJXfNMUkaYLICIYLa0A2S 7Ncnooq8tKIIUNBtcCSGoku02hjTbeDoELzDmMiePXTbGLc4NW6D6rJNLHc3 IliTmdrpRTR7MLYzqOAs+5gN+upF2CpE0VGs4HqooOd20YouOzNql02Qkp3j .s9i6bj3bYghirRm6Bh45iH+uECT4iFNM2GWE+G3Xb4x4S6LUQStRqBGAocp w2XZZCp04PmCS39lkgaawcnY0d+Sk3sns2L3yeE84FoRDGaDk5CF8rUe8F41 2TuYelx12Dg9bc6aVz4mv12DQefT4dWkr8tvzbmrna2xWYTr6mbRu8ZGUJ1Y EGuRox7tBRIK0dCgLTOu5OuUYyMgAaG7jdfRwV0je7HJeyd84BwKNEGCehIC Y6hhiGK.nV8GiFA..Nmm4FiYPGopDsSXPm48Y8fdA.L9Acl+mcC5tvoNnS97 dPWC.SXP28bNn+6wNYwQqBEI80fqSEp9NBIkxOlS.Dh649jEJaUTVVRZly2l l7wLwYtKW599MaybPijU.oJkPfdOgAaQkerop9brOcxrGfiVxonHn331DO75 +VAho1pY5ShigwAgbe.c9CI5g+wHAAsIRDugiAzS8QcQdxshsFoYalGfuL7n sucXLCgT06Irs2UMjwW5DlWlo+Nusjl3TYE3zO2lP0MKSWryFzFWjL3Mt3Iv KnY2rKN329igez3UTKzT3ACLbefB5dRmxdQYMMo7ccBFC6Ht+5frnU462FId ZbQHX2hSm9aRhiS9n5TsOObytjZ7vZupuMeURp.CJ1wxE2cSxZUqQ93KOv6S RitMh29iC2dqJBLhs8BlPfKQPOfvv0E4Qq9PVYGq52T1DjN.T7GuhuxNQpl1 QK4uuOHNJ+2ZowjGsILKOMTjlppc2l5PgRwIM3DdMHZ6eXVclCnTuQWWzGS8 DfpTNjg594LxPTLp0upXJarRF+Cgo6jcn5jaZyaC+80Aq5En8TwODoxIDhTd n30whz5rJQA0HrMRTKKc93rKXa3vswPuOykE.KwYHWQnuv9D46IzAjnds0SJ RLI85.kwMosJffnXFTVAUvHeHPVWEXEEPkhRoR8j2V7afC72b+pqvoBu8U0e YUAGCqziGvvC5PmBYSTVT6hbkHFgCcDlbuI2R0qnALa5Ox93bma+cU30jytK zGgLprxTjYm5rxTj0Kxr5vq7e5YX55jz0goiaKeavZxFLx6drQ9C1B2msYKG cTTbziHTbRmKMD7WZCiThcGFQxBPlnAQKK0T9hKcdmMRTGso5wQOvWbCiL6N LRKZOxssjZ7jBYXx.FEkeKYbizELwhAUI3Jtdyg1rj8oqJTdqnZP3TQr0b8h 4pbWPseorX7V66rIZ8tDtFdZJhkpXQk6xJre4GppxFhwDITzJ2mUanBqvq8k JIMY9IMFX.HAW56CnDYA+wGIN7iU0Erhqe+Vt672xk0rTyFeEYMqaWiunSQS kXPSUJNAhJ+v8ap3SPS0HLkIF14Rj4Ke23J2uIClVSl2Z.8wAun6SGYSoiG0 .nAZCBYxXfHxuSmRtlLZ6aCJYBgrQWh4a.kXd1fRF0mrBknFgd1f2iQLgTHa PIij7vTStmFkLgKuniOIJQMg4SrG1mbehZByGzFK6QgFPIjMl5ZBWNwFcIhu QxiEPLdpjxjwIhuE3HHl.eGLX1pnaHjoNniA5SfHwGssPaiZsxTbq2lKvqtW QO3ih84dkFSxOYas2Mqq.QlzUXEo2F79eT2Uz5o1QWYZ5TSLY0+C5u1hgvHA OFv9BXt0LWS9IqCSXiW+fL0Y0PiIEZpjxDwjXpMV+0D4wyBSlgDFXaNFLy3g Q3DAW4HDrOqG.VXXTRo9r7h4aKJ06.mMzAAiMUdcmKCIDXJ1gGxJdnnHqqKx UXnmbq.JK+HkW25bZXSUA0nN.UDUWcG.RoKokE7IRmcfoMGEalyRrgdfX2Sk Uo3AHbaRDB4exPOjIp1hrg7BD6TMNgLRWNlMnjIxhP1vTQIk.lPInMnDzDJM UcUPFMghXEVBjIjBakIulnrGzFhtQlrHqUl5ZxRr1Yb5jsFgASmrFgfmBBAs fw3BCtkVop71egA2hOYaMDfVvf65MW2JipYMpg91QiLnYtT0JCjlrrHzJTxj kpXGMPQ.HTplorFWSYxpDtZqP210scL4LpCP8GSGntdxGqC.l+NvAtezvNPS KU5tC3N+cfCLb9Hc.pHYF0Vpz90ssi+fFYoxA1ZcRoM9ngT1mAT0UEX4nMPt tnNz8U2v57tl05AF05QUsdnnB6366yn0ug0YbMQmF3QkV5AUReTod.Pel2d3 GQXk+e0hpPRO9NsltIFcXiHgA7M0OJvtcmKTm0thh9CaoGC6iDKoBPUkP8xq O01qmotMoy1qXSpWzdob0eftdPhndaS.0pS1EWepsWiLI1JijTyL.hYEZYjl cVgTDiPPruEbhOvrPDIoEdxzhXLsPSlVXio0jY3IHioEXxzxnntXkXQCLJrK xxs8zoE1HuCZkX7T1hGj1QGpemtV4IJQ0EoSFyyaouqqqLMZKutsMOvvl+wU vpn4SbQKwb0RDR.ZngU0Mrs9gl19Al09gUs+55XUcCaqgngsej+fwewGpZ+c i+tSr8ah3EO5bXafYzFRliHrZFsObgHKk7pFRbTGVjcJFwAf1o8TWshdxTGD aj9ED6HCwH4vXWqPKyrl2J5dZVfG6Q1HBCTaQLrTFH7.YiUW29batVP1XYyW 4n3FhEQP6mlVl1pAl0pk+SSgghqY+0ScsfbvxVs5reogHPzLjfT.yBCsDqmr x5HyUzDzoyj.fxINXEJ4QbadIqzNAFMRNcLgY7LT6PKvz3P8.nJvFSE9HQ4Q u5rpU2XxMYixmni5AOc.tfDPsMiu5SVeM.yxJgChyu0lJaRP8K85fEnU+SkQ ViV.SkAaAZgLUfoEnE1TwbSjVnoG33hHESopp2P0mlbqyHsGg9VAILSLs2rX 4ggDeVRoV.xzn30siyAt0bzOnzQ+PWZKWepiTP+y3HkgDedFofLa3RkZ6sWH efqL1a0kTWciIOZgLFvlrhLP3zyqjhDIg1XuZPmdj6.vou8d706uGrTkkx82 CdNzYwnrSvnVqLlnj19DU1An.RwGr97UiBK3QYHpmLHLHWplWwY2JYIhvU4k 180wPE6CTVCJjeZpbQ.SxY+xYBSjVFogExJZyALJRTTarLgjT.SHEzJjBZBo l7Bs.iMwXxiUtPS1gJViR8pbhq0HUuYYsU7dn4pZ1o+NH3h3hHOAF3Jciw99 DP6WC4I0M2GQqh7lmETPebIihptvDra2cgoY5usjFK1D7qpRPC6hGqN+HUeT VVJWjFdWTw2WVSgWDjt58Q4gqx2mpJ4MexSU8Nk07lzs6izBB48tGqKMNGTK ZJq1npJu42EuO7I556KuGcSv937lnPvpUghBL4Pp4QpN+EZvMk+6OrJ8rXCm PQEB7pOXbPC7mCEk+xi2DqW3RE4X2RWBeX2qbjpktfGj.j8.lGvCHKDQDeRY 49wNcf8atNL86ShWOXDt2Z71vaf7ey+7w++fZ2cEg -----------end_max5_patcher----------- </code></pre>

One reply on “I’m a VJ now”

Leave a Reply

Your email address will not be published. Required fields are marked *