Tuesday, March 3, 2020

Trash Panda


Note: Trash Panda is part of Lambda School Labs, in which I played a role in. Since I'm not in charge of Trash Panda, I cannot directly provide support regarding bug fixes/improvements.

What is Trash Panda?

Recycling isn't always straightforward. What do those numbers on plastics mean? What should I do with that broken refrigerator? How can I dispose this abstract object I can't tell what it is?

On top of that, recycling regulations differ across cities. Did you know some cities allow you to recycle items that other cities prohibit? The same goes for compost. This leads to a lot of incorrectly recycled items, placing unnecessary impact on the environment.

That's where Trash Panda comes in. Using your location and camera, the web app can identify the item you have and specifically point to you if, how, and where to recycle it. Prefer not to use the camera? You can search in Trash Panda's extensive database, which also offers interesting tidbits about your items to recycle. Trash Panda helps you master the art of recycling so you can rest assured you're doing the environment justice.

Compatibility notes

  • Trash Panda recommends location and camera permissions so it can properly identify how to recycle items in your area and use the image recognition model.
  • Trash Panda is meant for mobile devices due to the nature of how the app is used, particularly the camera. Desktop users will see a splash screen prompting users to load the page on their phone.
  • On iOS, Trash Panda only works on Safari since only Safari can use camera permissions.
  • On Android, you can install Trash Panda on the Google Play Store.

My role in Trash Panda

The Trash Panda team consists of four web developers, four data scientists, and two UX designers. I work as one of the four data scientists.

(That's right; despite this site's portfolio of web-based works, I wanted to try something different. There's a lot of different things to learn in the data science field!)

To build Trash Panda's image recognition model, us data scientists first needed to download multiple thousand images of recyclable products. Then, for each and every image, we had to draw a bounding box that labels the exact item we're trying to identify.


(Example of a bounding box. We want to identify the Nintendo 64, not the soda can.)

Sounds tedious? That's because it is. Thankfully we're computer scientists and know how to automate some things.

  • For downloading, we wrote python scripts that can automatically scrape images from Bing and Google Images. It works well, but we had to be careful about search queries (cork is a wine bottle cap, as well as a name of a city).
  • For drawing bounding boxes, we wrote a large python script that programmatically labels images. Some images, like those with just the item and a transparent background, easily label well with the script. Other images, like a white piece of paper on a white table, not so much. Although our script helped us save lots of time, we still had to manually label thousands of images. 

Once we got all our images downloaded and labeled, we used YOLO, an object detection neural network, to train our model. In a nutshell, the computer gathers information about what a specific item looks like from each labeled image and tries to generalize what it learned as a math algorithm. The training process requires a large amount of GPU power, and as such, we used a cloud computer, AWS SageMaker, to train the model.

Once the training finished, we got a single weights file that contains all the math algorithms. We deployed this file to a server, which end users utilize to determine the recyclable items in their own images.

If you want to learn more about image recognition, this site explains more about the YOLO neural network.

1 comment:

  1. Nice write up! It was a pleasure working with you. Thank for all the work especially on implementing the downloader, writing the script to append backgrounds to PNGs, getting the manual label checker going, beginning the exploration into Sagemaker and making the code more OS neutral (i.e. Windows friendly).

    Since you clearly have some domain knowledge that extends beyond data science, I'm going to have to ask for reccomendations, particularly on how to allow commenting like this on blog posts.

    ReplyDelete

If your comment won't send, check if third-party cookies are enabled.