Week 35 – Particle Tests, PhD Interview, Rotoscoping

Particle Tests

This week began with more particle movie tests, focusing on a setting up a number of particle goals including: a poly based particle goal for the whole glyph emitted from a single central sphere; a nurbs based surface goal, using the tutorial from last week’s post; an outer sphere of particles emitted from a second central sphere; and finally, a rendered version of the nurbs shape with a transparent blinn material to accentuate the shape itself, with some reflection. Motion blur was also added to add a little dynamic visual effect, rendered using the maya hardware renderer.

The results were interesting , with the best result coming from the surface render, however, the poly goal rendered an unusual result more than likely due to the lack of caching the particles. The image below shows the composite without the poly goal as it added too much noise. I will recreate the render shortly but with cached particles to compare the render. The glyph itself was the note of A.

Another test involved a similar approach with the E glyph, but with alternative particle colours in the composite. The clarity of this one was less apparent.

The pipeline for creating the particle tests will need to be decided shortly, as time is swiftly running out, approximately 10 weeks to go before hand in. In order to create the surface flow for the central nurbs shape, I may have to repeat the process several times, as the attachment of some of the nurbs shapes result in the creation of unwanted geometry. This can be made more efficient through the creation of a custom shelf, adding the appropriate buttons to lessen the scrolling through the menus.

I re-rendered the A glyph with appropriate poly goal particles – it looks better with added detail with the goal weight set to the maximum of 1, but it also needs particles to emanate from the centre more obviously. So, the only thing now is to work on making the central particles more visible and add a set of collsion particles for the geometry of the internal spaces.

PhD Interview

I flew down to Bristol and subsequently travelled to Bath Uni for the interview with the head of computing, Phil Willis, and Peter Hall, one of the supervisors. I have to admit of my uncertainty of the success of the interview itself, due to their interest in the physics and mathematics of my proposed recreation of cymaglyphs in a 3D space. This has made me question a number of things regarding the fundamental nature of sound travelling through air – my work is based upon the findings of the cymascope team, and the interpretation of their images created in the cymascope. What I need to find out is whether the glyphs are created as a result of the circular nature of the vessel in which they are imaged. For example, if it were a shape of differing dimensions, would the reluctant glyphs (and subsequent sound in air) still be similar to what is currently produced? Ultimately, it is my lack of knowledge of the whole process of the creation of the glyphs which is the issue. I would be disappointed to find out that my final film was a naive interpretation of a project that I have spent a great deal of time contemplating. Of course, the film is an interpretation whatever way you look at it, but I would hope that it is leaning in the right direction. Time will tell if nothing else. Having said that, with regard to the success of the interview, the idea behind creating a usable application for the hearing impaired and the autistic, still holds water. No pun intended!

Rotoscoping Cont…

Yet more rotoscoping to be done, but I’m getting there eventually. There’s only a couple of shots left to do and hopefully that will be it. But never say never…

Week 34 – Advanced Production, Particles on Nurbs Surfaces

This week began with a tutorial on how to get particles to travel across the surface of Nurbs, as an alternative method to using surface flow techniques. This may well be more efficient for me once I have completed the whole glyph shape, where I can apply the technique to one shape rather than doing it multiple times using surface flow. So, the tutorial is as follows for my own reference:

Creating a Particle Flow Across Nurbs Surfaces

• First select your Nurbs surface and make sure its parameterization is 0 to 1 in the U and V direction. This will make controlling your particles easier later on. To check the parameterization on your surface open up the attribute editor and look at the Nurbs Surface History/ Min Max Range U and V it should state 0 for min and 1 for max. If it doesn’t, you need to rebuild your surface so it does have this parameterization.

• Next select your “NURBS” surface that you want the particles to move across, with the surface still selected go up to the particles tab in the Dynamics section. Open up the “Emit from object creation dialog box” and under the first section “Basic emitter attributes” choose surface as the “Emitter Type” and make sure the “Speed” attribute on the emitter is set to 0, this way the particles will stay attached to the surface when they get emitter.

Then, this is important, check the “Need Parent UV (NURBS)” check box underneath Rate (particles/sec). By checking this option you basically get two per particle (PP) attributes attached to your particles called parent U and parent V. These PP attributes basically give the particles coordinates for the object space that they are going to emit from, meaning they now have a reference to the dimensions of the surface they are emitting from, hope that makes sense.

• If you hit the play button you should see your particles getting emitting from all over the surface, if not then you did something wrong. To see the particles better open up the particles attributes window and under the “Render Attributes” section set the particle type to “Spheres”
• Also VERY IMPORTANT change the lifespan of your particles to anything besides LIVE FOREVER, random range is a good one to use.
• Next make your surface a goal for your particles. Select your particles and then your surface and then go to the Particles/Goal Dialog Box. Turn off “use transform as goal” and set the goal weight to 1.
• Now for controlling the particle movement across that surface.
• To get more of an understanding of object space and the U and V direction of your surface, turn on your surface origins. To do this first select your Nurbs surface that is emitting your particles and then go to the Display drop down menu/ NURBS Components/ Surface Origins.

• You should now see Red and Green lines; if you zoom in on where the two lines meet you will see a little U letter icon and V, easier to see if you turn on wireframe shaded mode. These icons designate the direction of your surface, think of it like x and y coordinates in world space. So basically again what the parent U and V attributes did is let the particle know the U and V direction of the surface which can be used to have control how they will move across that surface.
• Next we need to create per particle (PP) attributes on the particles called goal U and goal V.
• To do this select your particles, not the emitter, and open up the attributes editor. Under the “Per Particle (Array) Attributes” section you will see a bunch of PP attributes already available to the particle system, like position, velocity ex. You will also see the PP attributes parent U and parent V which would not have been available if you didn’t check Need Parent UV (NURBS)” when you created your emitter.
• Anyways these are the default PP attributes that Maya makes available to a particle system, however you may notice that there is no goal U or goal V attributes, well you have to make them available to the particles so they can use them. To do this hit the GENERAL button right below the “Per Particle (Array) Attributes” section. When the Add attribute window opens under the “Particle” tab you will see a whole bunch of PP attributes that you can make available to the particle system, choose goal U and goal V and then hit OK.
• The goal U and V attributes allow you to set the exact locations on your surface where the particle are attracted; or in other words will allow us to move the particles across a surface with great control.
• Next let’s try something. You should now see your goal U and V attributes under the “Per Particle (Array) Attributes” section. Right click on the goal U attribute and add a ramp to it, now hit play. You will see all your particles moving across the U direction of the surface in a single file line. The reason they move this way has to do with the ramp. The ramp should be going from white to black or vice versa.
• Now how goal U or V uses this ramp is like this:
• Goal U and V tell your particles what their goals are, namely your surface. By applying ramps to these attributes you are basically telling the particles to start from a certain location on the surface at their birth and then specifying an end position for the particles at their death.
• This idea goes back to the beginning of this tutorial when you set the parameterization of your surface to min max range of 0 – 1. The reason you do this is because the default range of a ramp when created is from 0 – 1 you can check this by right clicking the goal U PP attribute and selecting “Edit array mapper” you will see that the range is 0 – 1.
• You see your surface is constructed or parameterized in a 0 – 1 object space, so if you take the U direction of the surface for instance, the left side is at a value of 1 and the right side is at a value of 0.
• Anyways if you think of the white colour of the ramp signifying a 1 or the left side of the surface’s U direction; and the black colour of the ramp signifying a 0 or the right side of the surface’s U direction; then hopefully you can understand why these particles are moving.
• So at a particles birth it is assigned the white part of the ramp which again is the left side of the surface and mid-way through its life is at middle grey, halfway across the surface, and then at death is assigned the black part of the ramp which is the right side of the surface.
• OK so we have taken care of goal U or the direction the particles travel in the U direction of the surface, now we can take care of the V direction. Right Click on the goal V PP attribute and make a creation expression. When the expression editor pops up type this in and then hit create.
• particleShape1.goalV= rand (0,1);

• make sure the name of your particles matches what I have here
• So what this expression did was place the particles randomly across the surface in the V direction. And because in the rand function we are using the range from 0 – 1 the entire surface is used and we get particles across the entire surface in the V direction.

Some things to try to bring home the idea of how goal U and V move across your surface are:

• First edit your ramp on the goal U attribute. Change the white colour to a mid-grey and the black colour to a dark grey. If you rerun the simulation you will see that the particles don’t move across the entire surface in the U direction
• Then try changing the expression on the goal V attribute to limit the range in the V direction. If you change the expression to rand (.5, 1) and re-run the simulation you will notice that the particles are only distributed across half the surface in the V direction.

Also to make the particles take longer to travel along your surface make their lifespan a larger number.

Summary of this technique

The benefit her is being able, potentially, to control how the particles move across the surface. So the next thing will be to try having the particles emanate out from the central point of the glyph. The downside is that ‘streaks’ don’t work as well on these, so I will also include a set of particles sending outward towards a simple sphere which encapsulates all of the geometry of the glyphs but with streaked particles to imply a greater sense of movement. These decisions will no doubt change as I experiment more with compositing techniques.

11 Weeks To Go!

The plan at the moment is to get as many of the tracking and collision shots completed, hopefully finished within the week. As well as that, to take one of the shots through to the end using 2 or 3 cymaglyphs so that I have something to show the folks down in Bath when I go for my initerview on wednesday. It might be pushing it timewise, but I’ll give it a go.

Week 33 – Advanced Production, More Tracking and Rotoscoping, Cymaglyphs, IBL’s, Semester 2 Presentation and Report

This week is a continuation of last week’s efforts with the 2 and 3D tracking of the main shots, along with Rotoscoping of a number of the shots that need masks. In addition, there are still a few shots that require exporting into Maya for collision objects. the shot below is an example of 2D tracking on my head – to link up with the movement of the glyphs – 3D tracking on the microphone, as there will be some particle collisions on the wind shield via a cylinder shape in maya – and rotoscoping of my head so that the glyphs come out from my mouth and not the side of my face. That’s the theory anyway, we’ll see how it develops.

Aside from that, I have begun putting the slides for my semester 2 presentation together, which I have 15 minutes of talk time for, followed by 5 minutes of questions. I have also updated my personal reflection document for the Going Live submission, which is also due in next week.

IBL’s

Some of the originally planned shots didn’t turn out as I had hoped, for example, the ‘text book’ images of physics diagrams. What I need to get across in the film is some kind of narrative that educates, to a certain extent, what the nature of Cymatics is. As I was considering having some ‘floating’ diagrams in the film, some floating text might work as well – as in, a dictionary definition of what the word means could be a good starting point. So, creating text in Maya was easy enough (I used Garamond as it’s a relatively elegant solution), but extruding the letters turned out to be a bit more painful. The letters themselves are curves, and it required selecting each one individually before setting up a bevel plus operation. Some of the letters, like ‘e’ and ‘a’, have two separate parts so need to be selected together before bevelling. I found that some didn’t work for unknown reasons, so I had to reverse the curve direction etc.

Anyway, with that out of the way, I wanted to use IBL’s to light the text in order to give it a more real feel, as well as reduce the amount of time used to light the scene. I didn’t have an IBL of the actual scene so I downloaded several photos that I thought might be comparable for the text from the site ‘sibl archive’ within the hdrlabs.com website. Here is a test render of my first attempt:

I still need to decide on what kind of texture or colour I want on the text.

In terms of modelling the cymaglyphs, I have produced a few more in relation to the individual notes I identified last week. Here are G, E, C and A:

PhD Possibilities

In addition to my own work here, I have been invited to an interview at the University of Bath for an interview in relation to a possible PhD, concerning my studies with Cymatics. Fingers crossed…

 

Week 32 – Advanced Production, Nuke Rotoscoping, CymaGlyphs

One of the first things this week was to remind myself how to roto out shapes of people in my footage, specifically the ones where there are people in front of the camera with the sound system and subsequent cymatics behind them. So, back to digital tutors and a short course on rotoscoping and creating mattes:

Master Rotoscoping

Lesson 2 Drawing Basic Shapes

• Roto node – Bezier tool – draw shape
• You can have several Bezier shapes within the one roto node
• Will highlight the frames with blue keys

Lesson 3 – Editing basic shapes

CTRL key+ LMB over a control point will break the Bezier handles

• Or, add points
• If you have a point that needs smoothing, use smooth points tool to add Bezier handles (behind the add points tool)
• Cusp points takes it back to their original drawn point

Lesson 4 – Drawing a complex shape

Hand example

• First drop in the roto node – draw the shape of the palm of the hand
• Name the Bezier shapes as you go as there will be 6 all together
• Draw fingers, using the same roto node – make sure the shapes overlap – name them
• [Week 32 Image 1]

Lesson 5 – Animating masks for rotoscoping

move to the frame you want where the shape has moved (extreme frame – furthest away)

• select the line from the roto node (if you have a list of them) – it will highlight the line in dots
• drag mouse around it to move – you’ll get the 4 pointed arrow like you do in transform
• move and rotate to suit in the frame you’ve moved it to
• any points that are out, move individually to match
• it’s ok for the shapes to overlap

Lesson 6 – Controlling the shapes edge and opacity

if you have motion blur – to get feathered effect

• click on the point with the little perpendicular line coming out – click and drag on the line and you’ll see a dotted line coming out [image 2]

• this is creating a feathered line
• you might have to pull the line up before moving it out, depending on its orientation
• can always add a cc node, use its mask to connect to the roto so you can see the difference between the masked part and the background [image3]

• also have the option to use the feather and opacity settings on the roto node, but this is more heavy handed – depends if there is a lot of motion blur for example…

Lesson 7 – Merging shapes together to create a cutout

• 2 bottles example – both on the same roto, but they need to be on different rotos to isolate either one [image 4]

• so, create a second roto node, don’t connect it
• all you have to do is drag the selected bezier from the root of the selected roto and drag it onto the root of the new one [image 5]

• now use a merge node to join both – it depends on what opreation you choose whether which will be on top etc – the ‘from’ opreation will only affect the B pipe (in this case more colour correction) – [image 6]

• if you connect the viewer to the merge node here, using the ‘from’ operation and selecting the view style to alpha (A), you can see how the alpha channel of the cut out [image 7]

• From = difference

Lesson 8 – Reusing keyframe data with the viewer

• if you have looping data, you can reuse roto date to save time (bobble head example)
• frames 1 to 7 have been keyed, after that it loops
• first off, move to the frame where you want to start copying from – drag a box around the whole roto to select it
• RMB – copy – 3 curves (spline key values) [image 8]

• then, go to the frame where you want the data to be copied – RMB – paste – 3 curves (values) again
• adjust any points if necessary
• drawback of this method is that you have to do it frame by frame, but quicker than redrawing

Lesson 9 – Animating a complex rotoscope with ease

• create different pieces for different parts of the body (woman waving example)
• start with the parts that don’t move as much (larger parts)
• add in keys as necessary to cover movement
• add in keys and refine

Lesson 10 – Integrating rotoscopes with another piece of footage

• once you’ve done all your rotoscoping, bring in some footage
• might be a good idea to roto out some shapes close to camera and blur the background
• add a merge node
• B pipe for the background, A pipe plugged into the roto
• change the operation to matte

Lesson 11 – Outputting masks into an image sequence for later use

• roto nodes can slow down system
• pre rendering can speed it up – however, you can also lose data as a result – it depends – check format you prefer
• bring in a write node (still using woman waving example)
• plug in the write node to the roto node
• you can tell if an original piece of footage has an alpha channel because the coloured squares at the bottom of the footage would also have a white square as well as the rgb squares [image 9]

• in the write node, choose the alpha channel from the channels drop down list
• the alpha will appear as red because nuke just drops it into the first available channel, in this case red – if you changed the view style from the drop down list above the screen, you wouldn’t see anything in the green, blue or alpha channels (becasue they’re empty), you would see a white version of the mask in the red channel – it stores the info as black or white in rgb before the computer assigns a colour to it
• save the file as a png or exr, depending if you want totally lossless or not – add in hash signs for the file size numbers for the sequence [image 10]

• read in the saved sequence
• disconnect the A pipe from merge to the roto
• you don’t need the roto and write nodes now, but you could move them out of the way and leave them visible if you want [image 11]

• we want to copy the read node to become the alpha channel for the original footage
• bring in a copy node – hook up the viewer to the copy node (error!)
• we want the original images to be the background so drag the B pipe onto the footage
• A pipe onto the read node with the alpha
• in the copy node, select the drop down ‘copy channel’ and select the rgba.red – now you’ll see on the copy node a little white square indicating the alpha [image 12]

• now drop the A pipe from the merge node on to the copy node
• change the viewer to view from the merge node to see the results [image 13]

Applying the knowledge to my own footage

Following the lesson from digital tutors, I applied and modified slightly what I need to do for masking out people from the footage to allow the cymaglyphs to go behind them (who happen to be in the foreground) and in front of the boom box and other objects in the background etc.

So, I have used the roto techniques in simple form as follows:

• rotoscope the shapes as necessary and keyframe them – one node should suffice
• add in a colour correct node to see that difference if you want (cc goes directly into the footage, while its mask goes into the roto)
• one completed, write it out as exr or png file (in channels, select alpha)
• read back in
• add in a copy node (or shuffle) and copy either the red channel of the mask to the alpha of the footage, or alpha to alpha (depending if the original footage has an alpha channel already)
• B pipe of the copy node to the footage
• A pipe to a merge node using matte as the operation
• viewer connects to the merge node
• I used a constant as a background to check, then a colour correct in a second example to note that I can use the A and B pipes on the same footage
• [images 15 and 16]

• the roto and write nodes are not needed now
• Finally, as a possible set up for the remainder of my shots where I need to fit in the cymatic images between the mask and the background, here is a pipeline that works (although i’m sure there’s a better way!): [image 17]

Maya – Creating CymaGlyphs

Continuing with the creation of the glyphs, I seem to find a commonality between them, in terms of their ‘trumpet’ like or ‘flower’ like shapes that make up the interior. Clearly the gemoetry changes from frequency to frequency, but whether it’s five-fold, six-fold, seven, eight or ten-fold, there always seems to be connecting tube-like entities that form from the centre outward with tunnels or holes to the outer shell. I have been using circles of varying size to form a profile shape for one of these shapes, then lofting it and using Mash to quickly create copies and distribute them in a spherical arrangement before baking down the shapes to 1 frame and adding in any other features.

One important question, amongst many, is – “how long do I spend on making these as accurate as the photos are?”. The reason being that when I animate the glyphs together, they’ll be changing rapidly and therefore some of the detail may be unnecessary. It will probably come down to where I stand nearer the end of the project, and if I have time to tweak and add as much as I can.

I have identified which glyphs belong to which musical note, through the process of running through the footage frame by frame. In addition, I have also identified some chords – the idea now is to create all of the glyphs, if I have enough time, and sequence them in the appropriate order along with the music in the same manner that they are imaged in the cymascope. Here are a few examples of the identified glyphs (In order left to right – A, C, D, E, F, G):