Magic Windows Final Project

For my final for Magic Windows I want to keep developing a project I have been working since the beginning of the semester as I’m a fellow at NYC Media Lab and A+E Networks to create an AR experience.

Our project focuses in 1968, as it  was an unique moment in the life of downtown Manhattan, and a year that is said to have changed the world.

By creating a site-specific cinematic AR time-travel experience, we are able to transport users back to 68′ Washington Square Park as they encounter those who waved protest signs at the same place, and often on behalf of the same values.


The idea is to tell a story of 2 characters, recreating their steps from 50 years ago, and reaching a climax once they meet in the center of the park. Each character represents a different side of the park and shows its own perspective. When the users meet, around the arch and the fountain, there is a new scene created upon this encounter.

The assets are being collected from archival footage and will be 2d still photos or animated gifs.

Main interactions consist on working with:

  1. Geolocation: the park is split in two different sides. As the user is closer to point A/B the respective experience is triggered.
  2. Proximity to specific elements: triggers secondary audio that is related to the image.
  3. Image anchor: as geolocation is not as reliable the meeting of the two characters will be triggered upon scanning an image – Washington Sq Arch.

This Project is being developed by Hadar Ben-Tzur with partnership with NYU Library Archives. The storytelling and research is a collaboration with Joana Evans.

AR Tarot Reader

In this post I will go through the research/ideation/first prototype of an AR Tarot Reader app to augment and give life to these ancient cards in order to  help users connect with its symbolism, stories and, ultimately, themselves.

Tarot and Emotional Intelligence

Tarot is a divination ritual originated in the mid-15th century to help people connect through symbols to understand the present and forecast the future. Those symbols are pulled from a history of human myths and archetypes and are part of what Carl Jung would call the collective unconscious. It is an interesting tool where, through tangible objects such as cards, participants can project their feelings and thoughts. By allowing them to see the parallels between the stories represented in the cards and their own experiences,  people can take a moment to meditate on their own environment and behavior – action that can influence future outcomes. Therefore, Tarot is a powerful tool for self knowledge.

With the rise of religious bricolage, people are increasingly partaking in the Tarot ritual both by visiting formal card readers and by purchasing their own cards, becoming readers themselves. Accordingly, several workshops, events, books, blogs, YouTube channels and Mobile Apps on the subject have been developed in the past years.


Tarot Apps

As a digital creator, I have been taking a close look at the current Tarot mobile apps. Most Tarot apps today either function with two main approaches: as an online “card taker”(as they are randomly/digitally picked for you), or as a guide to learn the meaning of the cards according to  the outcome of a physically played one – sometimes they can also combine both of these functions. Still, I believe that neither of those actually works effectively for the purpose of helping the user identify with the card’s stories and meditate.

I think that Tarot is a tactile experience – holding the deck in your hands, shuffling the cards, turning them over one by one…if you make it 100% digital you take great part of that Tarot magical moment out. Still, by playing it digitally, it is really easy to access informations and symbolic meanings. Unfortunately, when you make an app that is only a guide, you have to manually look for that specific card name and its meaning and, when you finally find it, you will discover that the information provided feels incomplete as it is usually displayed as shots (a paragraph or two) of content.

Below follows some examples of the Top 2 apps that I selected. I really LOVE the design of the first on and the UX/UI is great.

Golden Thread Tarot


Tarot and Augmented Reality

With those pros/cons in mind, I started to think about creating an App that would use Augmented Reality to bring the Tarot cards to life, as a tool to promote emotional intelligence. And I was not the only one, there is already a developed version as you can check:

Again, even though I see a lot of potential in the approach and the AR integration works great, it simply lacks content. For me it works more as an Augmented Reality Tech prove of concept than an actual tool that people would use to play Tarot. Also, it is very game-icky. Accordingly I see that as an opportunity to create a more interactive and content-based experience.


My idea

As a way to explore how to use UX in AR and to tell the stories of the cards, my idea is to  and personify each Tarot card. Based on the idea of the article/prototype experience AniThings: Animism and Heterogeneous Multiplicity  as the user places the desired Tarot under their phone’s camera, the card would show some information and serve as a virtual button, that when pressed would tell its symbology, story, and meaning.

By coming to life I mean start talking, as I would work mainly with sound. Visually I would display text and maybe some sparkles or some kind of smooth visual feedback to let the user know that by pressing the card he/she is activating the sound. I believe that working with too much visual animations or actual visual characters would take some of the meditative characteristics that I aim to incorporate in the experience.

Press play to check the prototype:


Further on, maybe even with AI, people would be able to have an actual conversation with each card, like a “Tarot Alexa”.

Touch the Mona Lisa: virtual buttons & object augmentation ideas

This week with Vuforia I explored creating an AR button.

By touching the Mona Lisa you are able to rotate the cube! Ho awesome haha – but it does give me some ideas and helps me practice my C# scripting.


The readings for class this next week comprehended two articles: Anithings, Animism and Heterogeneous Multiplicity and Developing Augmented Objects, A process perspective.

Both of them explore ideas to add digital qualities to physical objects, but while the first article create the things and gives them human characteristics,  the second one incorporate electronic functionalities to everyday existing objects.

As a homework for class we were supposed to think of an object we would augment. There are plenty of objects that I can imagine augmented in my house and in my everyday life. Maybe my purse or my pockets, reminding me of the checklist of things I can’t forget to put inside it; maybe my closet, to help me choose the outfit for tomorrow; maybe my notebook, to show me personalized, easy to read news; or my neckless, to help me send a “poke” to my loved ones; my fridge, mirror…. and so on. But one article had a framework on ideating an prototyping augmented objects.

So let’s try using it:


So let’s define my augmented object:

  • Problem Definition: when I play the guitar is an awful experience to have to check for the lyrics and chords through apps such as Ultimate Guitar in the computer or in the cellphone. The phone keeps falling from my lap and it would be nice to have an playlist to play to, plus the size of the letters are usually small.
  • Definition of the AO usage context: when I will play the guitar. Usually in the sofa, in the living room. Hands are not available.
  • Requirements definition: show lyrics and chords in a visible,  interactive way along the music. Command by voice.
  • Selection of the object to be augmented: not sure. I don’t think it should be the guitar… maybe an object to be placed in a coffee table in the living room that would project the lyrics somewhere?

I’m still not sure what would be the right object to augment to solve this problem. I’ll keep ideating and will probably discuss in class about it. Also, is Augmented Reality the way to go to augment this object? Don’t think so, right. I mean, in the AO Usage context my hands are not available…so at least not in the phone.

Let’s see what I will come up with!


Dance & Code & Pathways

In Choreographic Interventions class, where we work along with dancers to combine dance and coded visuals to create interactive performances, we created our first one minute duet using a live drawer and a dancer. Press play to see the result!

Our group created a solo full of spirals and that played with repetition. The dancer breaks out of the spiral and seamlessly transitions to other  movements. The code that created the visuals was based on the concept of circular and smooth movements that could increase or decrease in intensity and power.

The use of transparency and the ellipses that are commanded in size by the arrows on the keyboard and in move to the mouseX and mouseY directions were meant to create that idea. It was built based on this p5.js sketch created by Purin Phanichpant.

You can play with the code below!
Don’t forget to change the size of the circles using the up/down/left/right arrow keyboard keys.

Check the code here.

To see how we merged the code and the dance movements, check the video below.

This project was created by Chaery Moon, Chian Huang, Yasmin Schoenmann and I.

Starts with coffee: an AR experience

In our second assignment for Magic Windows we were asked to play around and create a storytelling experience using Vuforia.

That was not the first time I used Unity+Vuforia. In a previous post made for my Animations class last semester, I followed a basic tutorial, and added a dancing monster to a Starbucks logo. Thus, this time I went all the way focusing on concept and in the storytelling aspect of the experience.

This is my morning coffee mug:

I really like it and it has a lot of sentimental value attached to it. My mother collects mugs from the places where she travels to, and even though i do not have the same habit, it seems I do have an attachment for mugs from places where I did live in.

I lived in Tel Aviv from Feb 2015 to Sep 2016 and I have great memories from it. So because of that, every time I have coffee in the morning, which means every day, every morning, I kind of stare at it and play in my head with the tiny Bauhaus city illustrated in the mug.

Accordingly, I decided to keep playing and use it to tell a story in AR:


Starts with Coffee

Because in the cycle of our daily routines, everything starts with coffee.


In this experience, the user is invited to interact with the mug, unveiling videos and gifs that tell the story of a common week day in New Yorkers lives. The story is mainly a collage of resources I found on the web. Together and in this composition, they tell a story of an everyday week day. Starting with the alarm, brushing our teeth, breakfast. Than commute > work >commute. And all the possibilities held for the evening such as party, dinner, sex, movies and of course going back to sleep.

As the mug is  circular, its an ongoing experience, like our lives, right!? I thought it was interesting to recreate the “scenes” meaning morning/day/evening based in the three images displayed in the mug that I used as targets when creating the Vuforia trackers.

I look forward to exploring more ways of storytelling and augmenting objects when using AR. Having started to master the set ups and feeling more comfortable with Unity I’m also looking forward to start scripting more in C#. With that, besides the user input of triggering the augmented environments when locating targets, I will be able to establish more complex relations and therefore add way more user interactivity.


Birkat Habayit: My Electronic Blessing for the Home

In Eletronic Rituals class we have been discussing the definition of rituals and trying to understand what this concept means in the post modern *digital* era.

For me, ritual is interpreted as an active process of engagement and meaning making to enact a sense of belonging that makes the collective real (Burroughs, 2014). In this affirmation, I recognize that ritualization involves “a particular way of naming the social world” (Couldry, 2002), still, I would like to focus my approach on the subject as “a way of acting that is designed and orchestrated to distinguish and privilege what is being done to other, usually more quotidian, activites.”.

Therefore, for our first assignment, where we should imagine an “electronic” ritual and prototype the necessary systems to perform it, I decided to revisit and create a digital version of one of my family’s traditional jewish home blessing.

Since my boyfriend and I moved to our new place in New York  my family have been asking me to do the blessing of the home. According to both of my grandmothers, The “evil/greedy eye” isn’t something we should play with:  “It’s for protection Ilana!!! We have to send those evil/greedy eyes away!”. Still, even though I moved around 4 months ago, I haven’t done the blessing yet.

The blessing of the home or ברכת הבית (Birkat HaBaiyt) is a Jewish prayer often inscribed on wall plaques or hamsas and featured at the entrance of some Jewish homes. In the home, the blessing is traditionally hung on the wall next to the front door or next to a window. It is meant to drive any evil spirits out of the house and protect the occupants within. There are various versions of the prayer.


Hebrew Transliteration English Translation
ברכת הבית:
בזה השער לא יבוא צער.
בזאת הדירה לא תבוא צרה.
בזאת הדלת לא תבוא בהלה.
בזאת המחלקה לא תבוא מחלוקת.
בזה המקום תהי ברכה ושלום.
Birkat habayit:
Bezeh haššshaˁar lo yavo tzaˁar.
Bezot haddirah lo tavo tzarah.
Bezot haddelet lo tavo bahalah.
Bezot hammaḥlaqah lo tavo maḥloqet.
Bezeh hammaqom tehi b’rakhah v’shšalom.
Blessing for the home:
Let no sadness come through this gate.
Let no trouble come to this dwelling.
Let no fear come through this door.
Let no conflict be in this place.
Let this home be filled with the blessing of joy and peace.

(source: Wikipedia)

So I decided to — finally — bless my home AND do it through AR.

The blessing is beautiful! I mean, is a good mantra to have. I don’t think I really believe in “evil/greedy eye”, but I do believe it is an interesting passage, and a good meditation to say to yourself and to the people who live with you. Besides, is a tradition, and repeating a ritual from my family, rooted to my religion, makes me feel like I belong somewhere and am part of something bigger.

“For attitudes to become definitive, they must be cultivated by practice, and the name for sustained, value-laden attitude practice is ritual. […] In ritualizing, human beings discover, then embody and cultivate their world views, attitudes and ethics.” Grimes (2006, p. 135)

I believe that the ritual I recreated can remind people to take care of their home, to nurture a safe space and feel grateful about it. I think that by doing it in a more interactive you can make it more playful and engaging for people to bless their own spaces. It did indeed help me root and feel connected to my family.