INNOVATION- Final Leap Motion Tests

These are the final test that will be used as the main points of evidence in the research paper. These were developed iteratively from a number of previous experiments and were felt to be the most revealing tests.

First Person Scene Navigation

Manipulate Object

Pull Objects

 

Point

INNOVATION- Leap Motion UI Design Talk

Found a video from a Leap Motion Developer Meetup where they are presenting some findings from UI design research. While focusing exclusively on UI design they still raise some very interesting results that are highly applicable to my own project.

https://www.youtube.com/watch?v=XI3yvLOon08

Two of the most valuable points for the project I feel were that:

1- Users are not very aware of their hand position relative to the device when moving in Z space (forward and backward from the user). This must be taken into consideration when asking users to perform this gesture, particularly if accuracy on behalf of the user is required. Accurately pointing toward the screen (while physically moving the hand forward) also proves difficult, as our hands generally move in arcs and as such change their X/Y position as they move along Z.

2- Dynamic feedback in regard to on-screen position relative to user hand position is also absolutely crucial. Without this users can become very confused or disoriented, hampering the user experience. Any sort of small visual change to show awareness of user action is better and if it can reinforce action (or is some cases dissuade action) then this can only improve the experience.

INNOVATION- Hook/UDK system

This is the final Kismet node that handles the Leap Motion data. Using any number of these variables allows for a comprehensive system that can track the features of gesture that Hook identified in her research.

This is a simple example of how these variables can be used. This test is designed to register if a user is making slow or fast movements and respond in different ways. This example has the user spawning projectiles along the hands forward vector when a fast 'throw' gesture is made. Using scripts like this allows for great scope in how the Leap data can be used.

SPATIAL CONSTRUCTS- Final Screens

Final Shot

SPATIAL CONSTRUCTS- V4

SPATIAL CONSTRUCTS- V3

INNOVATION- Kristina Hook- Affective Interaction

After discovering Rosalind Picard and the field of Affective Computing I started to look for more research involved with this area and other academics writing about the potential for emotive communication with computers. I was also looking for more specific research around these principles that was more focused on how gesture can relate to this subject.

Through looking for other papers that reference Rosalind's work I discovered a researcher called Kristina Hook. Hook and her colleagues have focused on Affective Computing from an interaction design perspective: exploring how we can interface with computers through the communication of emotion. This is in order to add greater depth to the human-computer dialogue and create an environment that closer resembles human communication.

Hook's work was particularly relevant for the development of the project. Not only did it focus on affective computing through gesture interaction but it also included a series of design paradigms and an interaction model for affective gestures. These proved invaluable and formed the basis for the critical framework of the project.

My aim was now to see how well these interaction models held up in a digital entertainment context. Could affective interaction be used effectively in this environment? Can games utilize affective design principles when building interaction?

The discussed paper can be found here: http://soda.swedish-ict.se/145/1/PaperA.pdf

A breakdown of hand gestures through observation of numerous emotions portrayed by actors. (Hook et al 2003).

The Affective Gestural Plane model (Hook et al 2003).

Hook’s design principles are as follows:

Embodiment:

Embodiment is the notion that meaning can be created, manipulated and shared through engaged interaction with an artefact (Dourish 2001). This is to say that users can create and communicate meaning through interaction with a system and with each other through the system (Hook et al 2003). Embodied interactions must therefore physically capture an abstract emotional concept and allow for interpretation of meaning (Hook et al 2003).

Natural but Designed Expressions

When approaching gestural interaction there are often two opposing paradigms: designed gestures (Long et al 2000) and natural gestures (Cassell 1998). Designed gestures can be equated to sign language, having very specific semantics, while natural gestures are built around normal human expression (Hook et al 2003). Hook (2003) argues that affective interactions should strive to be both natural and designed: built from natural human action to facilitate affect but structured around meaning interpretable by a computer system. 

Affective Loop

The concept of the affective loop is to match the emotional communication channels of a computer system to those of a human (Hook et al 2003; Sundstrom 2005). By interacting with such a system, the users are engaging in an affective loop, where their emotional affect is created either by the interaction itself or is a result of the response to the interpretation of that interaction (Hook et al 2003; Sundstrom 2005). By interacting to that response the user is continuing the loop.

Ambiguity

Hook (2003) argues that ambiguity in an affective system is an integral component. Ambiguity allows for more personal expression and interpretation of emotion, giving users the opportunity to tailor the affective dialogue to their own life experiences, thus increasing affect (Hook et al 2003). Furthermore, an ambiguous system will also create a sense of mystery, keeping the user engaged (Hook et al 2003). Care must be taken however to ensure the system does not become too ambiguous as unclear communication of affect may cause frustration in the user (Hook et al 2003).

INNOVATION- Affective Computing

While looking for suitable research to develop the project I started to delve further into looking at human hand gestures. As I recovered papers and journals related to gesture it became apparent that many researchers were exploring the potential of gesture as a means of emotive communication.

The name 'Picard' was referenced regularly and following these references led me to one of the most important pieces of literature I had found thus far: Rosalind Picrard's 'Affective Computing'. This book, considered by many to be the magnum opus of the field, lays down a comprehensive foundation for the research into the field of emotive computer interaction.

Picard describes Affective Computing as the study of 'computing that relates to, arises from, or influences emotions'. Essentially, she explains that now that computers have the ability to register and interpret human emotion, we should be looking to how we can design software with emotion at the heart of the user experience.

I felt this to be a very interesting research field and that it properly captured what I wished to explore with the Leap Motion. Picard's work articulated the research field that I wished to explore and it became an invaluable resource for the duration of the project.

A shorter Affective Computing article by Picard can be found here: http://affect.media.mit.edu/pdfs/95.picard.pdf

INNOVATION- Cool Leap Motion Stuff

I came across a nice little resource for some really smart stuff being produced using the Leap Motion. Most of them involve interacting directly with a piece of software but it is great to see some using this device to manipulate real-world objects, such as a light show and a robot arm. Some of these apps are available on the Leap store, I have tried a few of them, and I will be observing any design choices that may be suitable for my own project. The list can be found here:

http://www.creativeapplications.net/processing/10-most-exciting-new-experiments-with-leap-motion/

SPATIAL CONSTRUCTS- 1984 Excerpt

 A good descriptive passage from the novel of the layout of Winston's room:

For some reason the telescreen in the living-room was in an unusual
position. Instead of being placed, as was normal, in the end wall, where
it could command the whole room, it was in the longer wall, opposite the
window. To one side of it there was a shallow alcove in which Winston
was now sitting, and which, when the flats were built, had probably been
intended to hold bookshelves. By sitting in the alcove, and keeping well
back, Winston was able to remain outside the range of the telescreen, so
far as sight went. He could be heard, of course, but so long as he stayed
in his present position he could not be seen. It was partly the unusual
geography of the room that had suggested to him the thing that he was now
about to do.

It was after twenty-two hours when he got back to the flat. The lights
would be switched off at the main at twenty-three thirty. He went into the
kitchen and swallowed nearly a teacupful of Victory Gin. Then he went to
the table in the alcove, sat down, and took the diary out of the drawer.
But he did not open it at once. From the telescreen a brassy female voice
was squalling a patriotic song. He sat staring at the marbled cover of the
book, trying without success to shut the voice out of his consciousness.

SPATIAL CONSTRUCTS- 1984- Experimenting with nCloth

I needed to develop the bed in my scene a little and felt that it needed a duvet. I started by creating a plane with plenty of sub-divisions and used soft select to push and pull the polys around in an attempt to create life-like folds. This technique however did not produce the desired result and I started to look into other options.

I had seen tools in Maya regarding 'nCloth' but had never looked into exactly what it was or how it could be applied. After a little research and experimentation I discovered that nCloth can provide exceptionally powerful cloth simulation that would be perfect for creating a natural looking folded duvet.

 

I started by creating a high poly plane that I converted into an nCloth object. This was then positioned on the bed as a duvet would normally be found and primitives were placed around it. These primitives were converted to 'Passive Colliders' so that the cloth would collide against them and I could therefore use these primitives to deform the plane how I wished.

 

I hid these planes so that I could see the scene a little better and set up a simple animation. The primitive on the right was rotated in such a way as to mimic someone using their arm to move the duvet aside as they get out of bed. This allowed the plane to deform and bend in a very natural way.

 

Further deformation can be seen. Settings such as friction and gravity had to be carefully tweaked in order to achieve the desired result.

 

An example of the finished animation. When the cloth is in a desired position I simply went into the outliner and deleted all references to nCloth settings and exported the mesh as the final deformed polygons. This could then be dropped into the final scene and is ready for a games engine.

SPATIAL CONSTRUCTS- 1984- Updated Build

The scene has come leaps and bounds recently. I have modeled a lot of the interior as well as a significant amount of the props I would like to include. The lighting is also working really well and adds a lot of atmosphere to the scene. I particularly like how the light from the windows is working, as the criss-cross pattern really adds to the 'busy' nature of the kitchen worktops, and the clear window pane lands perfectly on the desk where Winston's diary can be found. This acts as a great visual cue that will hopefully draw the players attention to this important narrative point.

SPATIAL CONSTRUCTS- 1984- Initial Build

This initial build was developed primarily to determine the size and geometry of the room. Although I used the correct UDK scale in my Maya scene, and included a cuboid matching the size of the player and some basic props, when the scene was imported into UDK it could often feel bigger or smaller than it looked in Maya. This required a 'back and forth' approach of importing into UDK and tweaking in Maya in order to determine a comfortable scale for the scene.

Winston's flat in the book is quite complex: containing a living room, bathroom and kitchen. In the film his room is comparatively simple: a single box room with few distinctive features besides his bed, sink and telescreen. I decided to design a combination of the two, containing things to one single room but introducing a kitchen and other distinctive features to help populate the room.

This build also helped in determining a rough lighting setup for the scene. I wanted the window light to be the most prominent light source in the scene and for it to 'point' to an important narrative prop in the scene: the diary. This blue light is to be contrasted by warmer electrical lights in the room that will subtly draw attention to other interesting elements in the scene.

I am happy with this layout and will get to work on modelling the various props in the scene.

SPATIAL CONSTRUCTS- 1984 Film Screen-caps

These are a series of screen-caps from Michael Rudford's film version of 1984. The film remains quite faithful to the book and provides some great visual reference. Some elements differ from the book and I will be using a combination of both to develop my interpretation of the scene. 

Big Brother

The door to Winston's room. I particularly like the old worn light switch.

Rudford's interpretation of the telescreen. I'm not sure how much I like this design, as it does not look 'high-tech' enough and will try to develop my own version of it that more closely matches the book.

Winston's 'Victory Cigarettes'.

A shot of the Telescreen watching Winston.

This scene is particularly interesting. This is Winston pulling out a loose brick to reveal the hiding place of his secret diary. This is very different to the book, where Winston keeps his diary in the drawer of a desk, stating that hiding it is useless: if they are in his flat looking for the diary they will find it and he is doomed anyway. This concept is described through Winston's inner monologue and as such would be difficult to convey visually. Therefore the director obviously wanted a more visual representation of the secrecy of the diary and as such chose to conceal it behind this loose brick. I am not sure which version I would like to use for my own scene and will be considering the visual and narrative impact of both. This scene also lacks an important feature: in the book, Winston writes his diary in a small alcove next to the telescreen where he cannot be seen. I do not understand why this detail was omitted from the film but I will be including this element in my design.

A good shot of the bed, chair and other props.

A clear shot of the windows.

A shot of the surrounding area around Winston's flat. 

Some interesting props to populate the scene.

A good shot of the 'Ingsoc' logo as well as the spy helicopters that watch citizens through their windows.

I particularly like the light next to Winston's head.

This scene is actually in Winston's neighbor's house, however is filled with great props that I will bring into Winston's room for the purposes of the scene.

A shot of Winston's view from his window at night. I particularly like the lighting and have decided to set my scene at night as well. This will allow for dramatic lighting as well as reducing the work required to render the outside, as most details will be obscured due to the dark.

The building opposite Winston's room. This differs from the book where most of Winston's view is obscured by the building opposite, most of which is covered by a huge poster of Big Brother that stares right into Winston's room. I think that this is a really strong visual detail the was omitted from the film and will be including this in my scene if time allows. Another detail that was not included in the film was Winston's view of his place of work: the towering Ministry building, The Ministry of Truth . This gleaming pyramid towers above the rest of London and is strong focal point that I am keen to include. 

The building opposite Winston's room. I particularly like the style of windows and will incorporate them into my design.

The clock in Winston's room. All clocks in this world are 24 hour, the book opens with the clocks 'striking 13', and is a small detail that I will be sure to include.

A great shot of Winston's favorite drink: Victory gin. 

SPATIAL CONSTRUCTS- 1984- Winston's Room

I have decided that I would like to build Winston's flat for my Spatial Constructs submission.

I chose this area for a number of reasons:

Often I feel that in most stories, the protagonist's home is one of the most interesting areas in the story. It often reveals much about this character and gives the viewer an insight into their lives and habits. In the book for example, Winston's room is mainly filled with gin bottles, as he spends most of his free time drowning out his sorrows, as well as the pain of the varicose ulcer on his ankle. I felt that by focusing on this area, I could include lots of little details that paint a larger picture about the character that lives here.

Thematically, Winston's room is also very important, as it is here where he writes his diary. This diary is Winston's first fearful steps down the road of 'thoughtcrime' and the simple act of owning this object puts him in grave danger. He uses this diary to articulate and better understand his own thoughts about the state of his life and the world around him. I therefore felt that this area had great significance within the context of the story and this element would help add a sense of narrative to the scene.

Finally, I felt that from a production standpoint, Winston's room looked like an interesting area to build that would require a number of techniques and close observation to create effectively. Being a room with three solid walls and a fourth with a long window, it significantly lowered the amount of required assets. The majority of the character of the room would come from the various scattered props and this allowed for the scale of the project to be changed as required. An outdoor environment viewed through the window is currently planned but could be cut if necessary. The texturing process will also be very enjoyable as all the surfaces are very old and worn and I thoroughly enjoy texturing surfaces like this.

I plan to take a series of screen-caps of the film and note down descriptive passages in the novel in order to get a better understanding of the layout and content of this room.

SPATIAL CONSTRUCTS- Brief

For this particular module we were given the task to pick a piece of writing from a list of authors and develop an environment that reflected a particular scene or event in this writing.

I picked George Orwell's 1984, as it is a story that I know very well, and has a particularly interesting context and setting. I felt that sticking with an author I was familiar with was an important decision as it saved me a lot of time, as I was not trawling through various author's works to find an interesting environment.

I will be using both the book and Michael Radford's 1984 film release of the same name as reference points for development while I consider what scene I would like to build.