How to quickly pick movies at a film festival using jQuery, Google Spreadsheets, and Python

My Dad and I are going to TIFF’12, which is one of the biggest film festivals in the world. Unlike many of the other top-tier film festivals, TIFF goes to incredible lengths to make the festival accessible to everyone, not just people in the film industry. There’s roughly 400 films this year, and each are shown at least two or three times over the week and a half that TIFF takes over virtually every movie screen in Toronto.

Picking movies for TIFF is always a complicated process, and one that changes every year. This is the first time you can sign up online – you used to fill out a crazy sheet of paper listing your first and second choices and hope your sheet got picked first. This year, you get a randomly assigned 1 hour window, but you can now see which films are already sold out and pick replacements.

I’m currently finishing up my internship at Dropbox this week, and I’m also on the opposite coast as my dad, so picking movies was going to be especially tough this time around, but I think I have a good system down.

  1. I scraped the TIFF webpage to build a spreadsheet that had the name of each film and a link to its description, stills, and trailer.
  2. I put shared the spreadsheet with my dad using Google Docs
  3. We both rated each film
  4. I wrote a script that organized the films based on our ratings
  5. I’ll use this sorted list to pick films

In case anyone else wants to do this, I’m making my scripts available:

Scraping the TIFF movie list:

Since the TIFF site uses jQuery, and has a nice DOM hierarchy for each film, this was pretty easy to do in my browser’s Javascript console: parse-film-page.js

The result is a CSV formatted list of each movie. Just put that in a Google Spreadsheet (or some other spreadsheet program) and you can quickly click each link, decide how interesting the movie looks, and add your rating in the spreadsheet.

In case you just want the CSV, you can download it here: tiff-12-films.csv

Sorting film picks

My dad and I added our scores to each movie, 1 (avoid) to 5 (must see). Google Spreadsheets made this really easy for us to both do at the same time. Since we are both fully employed and the rating process takes hours, this was very helpful.

I exported the spreadsheet with scores back out to a CSV and implemented a Python script that put each movie into one of six buckets:

  1. Both of us rated ’5′
  2. One of us rated ’5′
  3. Both of us rated ’4′
  4. One rated ’4′, one rated ’3′
  5. One rated ’4′, one rated ’2′
  6. Both rated ’3′

The script spit out another CSV that I’ve imported back into the Google Spreadsheet, which I’ll use when I’m picking movies in my one hour window. I’ll try to fit in as many of the first bucket before going down the next, until I’ve picked all twenty movies we’ll see over the five days we’ll be in Toronto.

You can find the sorting script here:

Good luck to everyone doing their TIFF’12 picks this weekend!

Helicopter camera live test

Figured out how to record video from the helicopter’s optical flow camera. Here’s me waving my hand over it quickly:

Video [mp4]

Remotely displaying the Helicopter Brain Prototype’s camera.

Now that vSPI lets me quickly send data between the HBP FPGA board and my computer, I’ve been able to see what the helicopter sees in real time (~100fps).

It only took a day (well, a grad student day) to set up. Most of the time was debugging the proprietary camera interface and figuring out how to render video using OpenCV.

The Python script running on my desktop computer, which grabs frames from the FPGA over vSPI and renders the video is only 44 lines. The code running on the FPGA itself to send the image data to the computer is less than 10 lines.

Announcing vSPI: An Open Source SPI Slave

I’d like to announce the v1.0 release of vSPI!

It’s a fast serial port (up to 27.9 mbps on my Atlys Spartan-6 Devkit). It allows anyone to transfer data between their FPGA or ASIC project and their computer. I’m planning on using it to monitor the RoboBee brain in our Helicopter Brain Prototype (HBP). Pretty soon you’ll be able to see what the helicopter sees on my computer screen!

It’s written in Verilog, has an optional Xilinx EDK peripheral interface, and a easy to use python library for your PC. Go try it!

You can access the project here:

The license is pretty flexible. The only thing you have to do in order to use it is let me know that you’re using it. I’ll keep a public list on the website of who is using vSPI. If you need to secretly use it, let me know and we’ll figure out some alternative licensing scheme. My only condition is that you don’t use it for things that hurt or kill people (such as missile guidance systems).

HBPv2 boards are back and working

Great news!

The HBPv2 boards came back last Friday, and I’ve been testing them since.

The boards, fully assembled, weigh 4.16g. We calculated that the boards needed to be under 6.5g to take off, but our goal was 4.5g to achieve a longer battery life, easier flight control, and room for additional sensors. Doing better than our goal lets us add even more sensors and get longer flight times since the rotors don’t have to work as hard.

Our two big concerns with the v2 boards was the regulator circuit and the interface with our optical flow camera since both had problems in the v1 design. It was pretty easy to test the regulator circuit: just plug it in and see if it regulates. I’m happy to say that it did. Yessssssssss!

After a little debugging, we got the camera up and running too:

I drew a test pattern on a Post-It note (left), then photographed it using an OF camera attached to the HBP board. The result is the image in the middle. If you stand back, you can faintly see the same test pattern in the lower right corner of the image. I took the same image, and turned up the contrast and smoothed it a bit so that it’s easier to see. Software running on the HBP does a similar operation to compensate for the OF camera’s low range.

Now we know the HBP hardware is working. We’ll refine the RoboBee brain RTL and software running on top so it can fly on its own.

Baking components to a PCB

The PCB passed all tests, so I sent it out for assembly. We’re paying another company to have a “pick-and-place” robot solder all the tiny components to our PCB. Some of the parts are just too small to do by human hands. Our FPGA’s pins are 0.8mm apart, and there’s 484 pins. Best to let a robot handle it.

While I’m waiting for the assembly to come back, I thought I would document how I tested one of the regulator circuits. Since I’m just testing the regulator circuit, we only have to solder the components in that circuit, not all of them (and certainly not the FPGA, phew).

Still, the components are very small (several 0402s which are 0.04in x 0.02in) and too small for me to use a soldering iron. There are some soldering iron ninjas that can do it, but sadly I’m not one of them. 

Instead, I “baked” the components onto the board. This involves spreading tiny amounts of solder paste onto each of the PCB pads, carefully dropping the components on top, and putting the whole thing in the oven to reflow the solder. It’s like baking cookies, except much more precise and not tasty.

The first thing you need when baking PCBs is a good microscope. Here’s the one I used:


The second thing you need is a way of spreading solder paste in very small quantities. I use a sewing pin attached to a xacto knife base. It’s very precise and weighted well: 


You can’t see it on the left, but there is a little drop of solder paste on the head of the pin. Looking in the microscope, you can see that the solder paste is just tiny spheres of solder suspended in some liquid.

The trick is to use the pin to spread the solder paste across each of the pads on the PCB. It took a few hours to get it just right for the regulator circuit I was testing. You need to make sure each pad is well covered, but not shorting out (connecting) to neighboring pads. The slightest movement becomes large under the microscope, so you have to calibrate your coffee intake just right. I’ve heard some labs keep a few beers around to keep a steady hand. Or so they say.

Once you’re done spreading solder paste over each pad, you VERY CAREFULLY drop the components on top. You have to be very precise, since a component that’s dropped incorrectly can short out pads and may require a lot of repainting. It sounds easy, but when you’re dropping 20-pin components with a pair of tweezers under a microscope, it’s a different beast.


The photo above shows the result of the painting after dropping the components on top. Click the photo to see hi-res version. 


Here’s a zoomed in version above. You can see the spheres in the solder paste, and notice that there’s plenty on each pad. The trick is to get enough so that you’ll get good connections, but not so much that the pads short out. And don’t worry about the pads that look shorted on the right, they’re the same pad.

Once all of the pads are painted and the components are dropped on top, it’s time to bake the PCB. We have a special oven for this purpose, though it’s in Chinese so we just memorize which buttons to press in which order. You can specify the temperature profile, which specifies what temperature of the oven at various times. Some parts are sensitive to the profile, but we’re not using anything delicate and stuck with the default profile others have used.


After eight minutes, the PCB was all baked:


You can see the spheres in the solder paste have all melted together and now bond the component pins to their PCB pads. Click the photo for the hi-res version.

Here’s a zoomed in version:


And lastly, here’s what the board looks like to the naked eye. Looks pretty good I think:


Once the board was fully baked, we then powered the regulation circuit and make sure it was generating the correct output voltage. Once it did, we knew the regulation circuit was working properly. Woohoo!

PCB weight reduction

Just weighed the new PCB. It’s lighter!

The v2 PCB weighs 1.62g, which is 21% less than the v1′s 2.05g. This should make flight more stable and use less power, or at least give us an extra 0.4g to for more sensors.

HBPv2 PCB weigh-in

HBPv2 PCB weigh-in

We went from a 31mil (0.031 inch) board thickness to a 20mil board thickness, which is why the board got lighter. Although the board thickness shrunk by about 35%, he board didn’t get 35% lighter because the non-filler material didn’t shrink (traces, etc).


Get every new post delivered to your Inbox.