The ITP Assistant

During my Hello Computer class, I developed a bunch of experiments such as the emoji story creator, invisible map and the dada poem reciter. Those were more fun and simple weekly assignments projects, where I learned a lot and helped me explore the use of voice interfaces.

Therefore, for my final for this class I decided to create a bot using dialogflow in order to create a functional voice interface tool to help people complete a specific task. My idea was to user test and also think about the UX behind voice interfaces as well as develop a platform that could eventually be used in parallel (or even more used, who knows!?) than the webpage designed to fulfill the same functionality.

So what do I – really – need help with?

My main inspiration for this project was the current Google Assistant and that amazing – but also scary – video released in this years Google I/O where the digital assistant calls a hairdresser to book a hair saloon appointment for you.

As most impressive as that is, still,  for my daily tasks the best use that i make of my Google Home/Assistant is mostly to ask for the temperature, to play a song, and eventually to help decide what should I wear according to the weather. Which is nice, but I would imagine that this tool should have way more potential to be functional in my life. So what is something that I make often and would like to get done by voice command in at least a weekly basis?

Differently from the Persona in the Google’s I/O example, that wants have her haircut done and need to book hours for it, as a student something that I schedule often is Office Hours with Residents and Professors. And ITP has a website for that.



The website collects all links from the residents/professors calendars and display their Bookable Events, which you can click and book as Office Hours. Despite some professors that do a double check with an internal NYU account, as long as you have any google @gmail account you are enabled to do that.

The website does its job, but  it can be annoying as you have to go back to check if the person with the skill you want is available for the day you are also available. If the person is not, you have to go to the home and to the steps again to find another resident or teacher that could help you… and so on.

So, what if I could create a voice assistant that could book ITP office hours for you?

Would I succeed on developing it? If I did, would it indeed be useful? Would people be willing to switch from using only the website and find a voice interface actually more helpful?


Setting up DialogFlow and Firebase for a first prototype

I followed Nicole’s instructions in class and set up a simple working prototype from the DialogFlow/Firebase side for creating the ITP Assistant.

My idea for now was to make the simplest UX possible that could fulfill the task of booking office hours, even if that woudn’t be good enough for the final interface, but could work as a proof of concept and then grow from there. This is the interface I aimed to create:

So that part was working,  as you can check in the video below.

My code at this point was very simple.

I added a JSON file with the names of the residents (office hours with teachers too would have to wait a bit!), with their skills, and, once i had received the string inside my skills Intent, I would parse it and try to match it with the existing skills on the JSON. If it found a matching skill I would push the Resident’s name into an array, than then would be randomized, and one of the matching resident’s selected. Once selected, while the DialogFlow would be saying to ‘hold for a sec while I book office hours”, the code would run a function to  get that Resident’s URL and actually book the office hours. – that then would be said back by the Dialogflow with the date and time scheduled. You can check the code here.

So that’s when my problems begun…

Finding out that Google’s Calendsr API couldn’t work for me

(I’ll finish writing by tomorrow Oct 26th)

Working with puppeteer (fun!)  and Booking office hours through my server

(I’ll finish writing by tomorrow Oct 26th)

Trying to make firebase and pupeteer work together (not successful)

(I’ll finish writing by tomorrow Oct 26th)

Next Steps

(I’ll finish writing by tomorrow Oct 26th)

Talking Robots and DialogFlow

The idea of having an Artificial Intelligence capable of resembling perfectly, both speech-like and physically like, a human being is a very romanticized idea throughout our culture. Ironically, as we move more and more towards the ability of creating machines that – even if not yet perfect ones –  and somewhat succeed in this matter, User Experience shows us that maybe it was not the way to go after all.

Both episode 1 and episode 2 of the Podcast Sandra and the reading on Upending the Uncanny Valley  explore this irony and question the validity of investing in such resemblance. The Uncanny Valley theory, that describes the common unsettling feeling people experience when androids (humanoid robots) and audio/visual simulations closely resemble humans in many respects but are not quite convincingly realistic, was an interesting term to discover and that I could relate a lot to.

I think this is an interesting paradox and ironic in a way. I would be interested in exploring this discomfort further and playing with it as well as exploring and experimenting with its boundaries as an UX designer.

Making a poem & Potential automated poetry slams

At one point Elementary School, in one of my Art classes,  I remembered we played with the ideas of the Dada movement by following the instructions on how to make a Dadaist poem made by Tristan Tzara:


For the purpose of working with my assignments for Coding from A to Z and Hello computer, where we were asked to edit a text input and make  your code “speak it to you”using the Speech Synthesis API, and taking this exercise as a reference, I created a webpage where you can create your own digital Dadaist poem according to Tzara’s method and make your machine recite it to you.


You can make your own poem here and check the code.