Project Skyward

Improved Ornithology Learning Experience

Skyward is my M.A. capstone project at NYU Steinhardt, it is a mobile app that allows the user to identify the birds they see, collect them virtually and play as them in a 3D role-playing game.

This project was a winner of the 2017 May & Samuel Rudin Award winner at NYU Steinhardt.

Software Used: Sketch, InVision, Adobe Illustrator, Adobe Photoshop, Adobe Audition, Adobe Premier Pro

Final Deliverable: Interactive Prototype & Design Document

Where Did This Idea Come From?

Over the past year in New York City, I’ve visited the American Museum of Natural History(AMNH) several times. The museum has many exciting exhibitions, from giant dinosaur fossils to tiny underground ecosystems, but my favorite exhibitions are always about birds. However, as most exhibitions about birds are only still specimens and videos, I find them not very appealing to most of the visitors in the museum. The Hall of North American Birds, for example, is just a large room with specimens hiding behind glass and nothing else. There are introductions under the specimens, but not many people seem to be interested enough to read them – besides, the lighting condition is not very ideal, which makes it difficult to read. On the other hand, the specimens don’t show enough of the birds’ life in the wild. There is some background setting behind the specimens, of course, but they are only captured moments - a scene of a golden eagle spreading its wings standing on its nest on some rocks which supposed to be a part of a mountain sure looks cool, but it doesn’t provide enough information on the bird itself, and also the role human may or may not play in their lives.

What I saw in the museum and my own love for those flying creatures led me to this question: what is a better way to show people the life of birds and our relationship with them?

The Plan

This design model diagram I made is a combination of the "Double Diamond" and the model provided by professor Bouwmeester. Since we're not actually implementing and this design and developing the final product, the third diamond was hidden in this process. The entire design process followed this order.

design model.jpg

Research and Ideation

I took a variety of approach during the entire design process to collect information about the subject matter, gather feedback and test my idea along the way. This includes interviewing users, sending out questionnaires, brainstorming with fellow students, talking with ornithology specialists, and also of course, looking for literature online.

In the final research paper(M3 in the above graph), I talked about the background of the idea, the problem I’m trying to solve, the audience I was trying to reach, the landscape of existing solutions, the context and content of my possible solutions and the goal of the product.

"How might we design a new way for people who are interested in birds to learn more about ornithology?"

By the end of the ideation phase, three ideas emerged from what I’d learned.

1. Collect & Play

The user virtually collects the bird they spot in a museum or in the wild, the app identifies the bird and offers simulation games for the user to play as the bird they collected.

idea 1 collect and play.PNG

2. Virtual Bird Tracking

Using AR technology, the app turns the museum into various bird habitats. The user then uses the app to track the virtual birds in the museum and learn about them.

idea 2 virtual tracking.PNG

3. Build Your Own Bird

The user will be given different types of physical features of bird to choose from, and they have to build their own bird to adapt to certain environments. This customized bird will then be put into a simulation to test its ability to survive.

idea 3 build your own bird.PNG

The Solution

Eventually, idea 1 was selected in the end of the research and ideation phase, with a few modifications.

The solution is an app that allows you to virtually collect birds you see in the wild, museum or even on the internet and learn about them using mini games and simulations provided as rewards for collecting, which will create a link between museum exhibits and real life bird watching, provide knowledge of ornithology immersively as well as raise awareness of bird conservation.

The core functions of the app will be:

1. Bird Identification for museum and field trip using pictures taken by the user.

The user goes to a bird watching trips and/or a museum and takes pictures of real life birds they observed in the wild or bird specimens in the museum. The user can use a build-in camera of the app or allow the app to access their system album to retrieve pictures. These pictures will be imported into the app and be analyzed using a recognition system. The app will then identify the bird automatically. If the user doesn’t have a picture that has good quality, the app will take a different approach by ask a few questions about the color, body type, location and behavior about the bird to identify it. Also, if the user wants to take pictures using the built-in camera, they can choose to take several pictures in a row and identify them later, without having to run the identification each time a picture is taken.


2. Keeping a virtual collection of the birds the user identified and providing detailed information on them.

The birds the user identified will be added to the user’s own collection as different entries, the pictures user has taken will also be saved into each entry of bird in the collection. Information about the bird will be provided by the app, such as formal name, habitat, sound and a small chunk of descriptive text. By reading this the user will learn more about the birds they collected.

The user will be able to link their social media to the app and share their entire collection or a specific bird they collected with their friends.


3. A progression system that rewards the user according to the number of birds they collected.

As the user adds entries to their collection, they gradually unlock contents as rewards and make progress. There are two types of reward: badges and interactive contents. For instance, when a user collects a flightless bird for the first time they will get a “Flightless” badge, and when they collects 100 birds they will get a “100 Birds” badge. All of the badges will have unique visual design. The interactive contents is the other part of the reward system, which will be introduced below.


4. A simulation game where the user plays as the bird and complete certain objectives.

When the user reaches certain milestones in their collection, they will unlock levels in the simulation game. In each level of the simulation game, the user controls as a bird in third person view, roam around a game world and complete different objectives. Those objectives will be designed to teach the user about the bird’s habitat, diet, behavior, life cycle, etc. Once unlocked, the levels can be replayed repeatedly. By completing the game the user also progress forward and have the chance to unlock more contents.

The logic model of the app is presented below:

logic model v1 (without assumptions).jpg

Concept Pitch Video

This is a presentation of my work in the first semester. It features some new wireframes that was not included in the M5, as well as new visuals to demonstrate the user experience. For detailed description of the design, please check the final paper.

Test and Iterations

This concept pitch video was used for my concept testing. I sent it to people from all kind of background – museum workers, designers, software developers, ornithology experts, family and friends.


Main Improvements

Based on the feedback from concept testing, I started to revise the original prototype while adding new functions and improving user interface. Here are some major adjustments I made to the original prototype in following iterations. These improvements can be seen in the prototype section above.


1. Bird Radar. The radar was added as a better way to draw data from the eBirds, and it’ll make bird watching more fun. It helps the birdwatchers on their field trips to learn what’s near them and what to expect. Since it doesn’t have the checklist function as the eBird mobile app does, it’ll be more friendly to amateur birdwatchers and will not overwhelm them.


2. Achievements, missions. To motivate the birdwatchers even further, I added the mission system and the achievement system.


3. Diversifying sources of experience gain. In the new design, user can now gain experience in more ways: by successfully identifying birds, completing missions, earn achievements and completing game levels.


4. Redundant design. Now the user can access picture ID and sound ID functions in different places, making it easier for the user to ID a bird the second they see it.


Usability Test

The usability test was conducted on an earlier version of the prototype. Although the latest version hasn’t been through a wide range testing process yet and the prototype of the game part is not functional, I’d still like to point out a couple of things that came up during an earlier test and how they were improved.


1. On an earlier version of the prototype, the amount of information on the list view and grid view of the collection page was almost identical, which eliminated the meaning of two different view modes. Now the list view mode of the collection page was made more compact to show more specimens so the user can make quick selections.


2. Photo cropping. After doing research on existing picture recognition platforms, I realized that it’ll improve the performance of the app if the user helps the software find the bird they want to identify in a picture. So I added a new step in the picture recognition process where the user needs to crop the picture and put the bird in a blue frame.


3. Getting rid of the “multiple shot” mode. Originally I thought taking multiple shots of the same bird will be beneficial to the recognition system. However, then I found that this is not a good solution because 1) the recognition system doesn’t necessarily need multiple pictures to identify a bird, 2) in real life bird watching activities, it’ll be hard to take pictures of the same bird from different angle, 3) talking multiple pictures will slow down the user flow and lower the efficiency of bird ID. So in the final prototype I deleted that option.

Final Prototype


Skyward is an interactive application which accompanies the user while they explore the world of ornithology. It helps the user while they observe birds, no matter in the wild or at home, and learn about the life of the birds they see in an engaging and interesting way.Initially, this idea, along with several others, all originated from my own personal interests. With the help of professor Hoadley, I decided to choose the topic of ornithology above all else because I believe it can make the most of my design thinking and technical skills.


In the research phase I received generous help from people at the American Museum of Natural History, where I was an intern at the Science Visualization Group. Mr. Barry Joseph, my supervisor, introduced me to professor Peter Capainolo, a seasoned ornithology expert, who later showed me the backstage of the department of ornithology in the museum, and answered many questions of mine. Mr. Joseph and Mr. Capainolo, along with my colleagues in the museum also gave me invaluable feedback on my concept pitch video at the end of the first semester. The birding events held at Bryant Park in NYC led by Mr. Gabriel Willow was also a great source of information, I’d really like to thank him for his patience and kindness.


The design process of this project took two semesters – this is the first time I work on something like this, a full scale, individual design project, that includes multiple phases and different research/design methods. Throughout the entire process, professor Hoadley and the DMDL/G4L community have been giving me feedback on every step. Without them, I don’t think I would be able to finish this project alone.


I hope Skyward will appeal to the general public and motivate people who are interested in birds to learn and have fun. Although the design process is coming to an end, I know there’s still much to do for the project, and there are still many things I need to learn – my career as a designer has just begun.


Future Work


As said before, the usability test has yet to be done on the latest version of the prototype. The future work related to the prototype will be:


1. More in-depth analysis with specialists from different fields such as ornithology and user interface/user experience design.


2. More engagement with casual users in bird watching events and see if they are interested in the app.


For now, the game part of the app is still under development. Although according to current evaluation, the users like the idea of becoming a bird and experience the bird’s life, but without a playable game prototype, it’s hard to tell if there’s any flaw in the game design. Regarding the game, there are some questions still need to be answered. For example, how well will the users learn about the behavior of the bird after playing a certain level? Does the unlocking of game levels feel rewarding enough for them? How do they interact with the game’s UI? These questions will be answered in future in-depth analysis and large-scale playtest with the game.