The script, shown at the bottom, currently requires a svn version of boto with revision of at least 1595. Assuming you have python and subversion installed, the following will get you up and running with the script, which can be downloaded here. Running the script concurrent times will take significantly much less time, therefore stopping the script midway through is of minor consequence.

svn checkout http://boto.googlecode.com/svn/trunk/ boto-read-only
cd boto-read-only
(as root) python setup.py install
cd ..
rm -rf boto-read-only
./convert_to_rss.py your_bucket_name [aws_access_key_id aws_secret_access_key]


Note that you may alternatively put your aws_access_key_id and aws_secret_access_key into the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY respectively.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69

#!/usr/bin/env python
import os, sys
 
try:
    import boto.s3
    boto.s3.key.Key.change_storage_class
except ImportError, e:
    sys.stderr.write('Package boto (svn rev. >= 1595) must be installed.\n')
    sys.exit(1)
except AttributeError, e:
    sys.stderr.write('Invalid version of boto. Required svn rev. >= 1595.\n')
    sys.exit(1)
 
def convert(bucket_name, aws_id, aws_key):
    s3 = boto.connect_s3(aws_id, aws_key)
    bucket = s3.lookup(bucket_name)
    if not bucket:
        sys.stderr.write('Invalid authentication, or bucketname. Try again.\n')
        sys.exit(1)
    print 'Found bucket: %s' % bucket_name
    sys.stdout.write('Converting: ')
    sys.stdout.flush()
    found = converted = 0
    try:
        for key in bucket.list():
            found += 1
            if key.storage_class != 'REDUCED_REDUNDANCY':
                key.change_storage_class('REDUCED_REDUNDANCY')
                converted += 1
            if found % 100 == 0:
                sys.stdout.write('.')
                sys.stdout.flush()
    except KeyboardInterrupt: pass
 
    print '\nConverted %d items out of %d to reduced redundancy storage.' % \
        (converted, found)
 
def main():
    def usage(msg=None):
        if msg:
            sys.stderr.write('<Error>\n%s\n</Error>\n' % msg)
        sys.stderr.write(''.join(['Usage: %s bucket [aws_access_key_id ',
                                  'aws_secret_access_key]\n']) % sys.argv[0])
        sys.exit(1)
 
    if len(sys.argv) == 2:
        bucket = sys.argv[1]
        msg = ''
        if 'AWS_ACCESS_KEY_ID' in os.environ:
            aws_id = os.environ['AWS_ACCESS_KEY_ID']
        else:
            msg += 'Environment does not contain AWS_ACCESS_KEY_ID.\n'
        if 'AWS_SECRET_ACCESS_KEY' in os.environ:
            aws_key = os.environ['AWS_SECRET_ACCESS_KEY']
        else:
            msg += 'Environment does not contain AWS_SECRET_ACCESS_KEY.\n'
        if msg:
            usage(msg + 'Please set values in environment or pass them in.')
    elif len(sys.argv) == 4:
        bucket = sys.argv[1]
        aws_id = sys.argv[2]
        aws_key = sys.argv[3]
    else:
        usage()
 
    convert(bucket, aws_id, aws_key)
 
if __name__ == '__main__':
    main()

Yesterday, in order to extract this information from Facebook, I spent some time and made some significant modifications to photograbber. You can download the patch here. The changes are as follows:

• GUI checkbox option to download entire album if tagged
• Saves photo caption and comments in {pid}_comments.txt
• Saves photo tags in {pid}_tags.txt
• Photos modify time is set to the upload time of the photo
• Comment file modify time is set to the time of the last comment
• Tag file modify time is set to the time of the photo upload
• Improved error and thread exit handling (not perfect though :-/)

Features specific to the complete album download feature

• Stores pictures and metadata for an album within a folder named {username}-{album_name}
• Replaces invalid characters and spaces with underscore
• Saves album comments in ALBUM_COMMENTS.txt
• Album folder modify time is set to the modify time of the the album

Photo captions and comments are stored in the following form:

Photo Caption
The only men running in bikini bottoms.
 
Sun Sep  6 18:20:52 2009 John Smith
high five.  a million high hives.
 
Sun Sep  6 20:05:40 2009 Sally Thomson
OH my gosh ... I love it!

Album descriptions, locations, and comments are stored in the following form:

Album Description
The best vacation ever!
 
Album Location
Aruba
 
Tue Jul 21 15:33:46 2009 Sally Thomson
whatever u do don't get abducted! jk ;)
 
Tue Jul 21 15:35:44 2009 Bryce Boe
I don't even know how that's possible there as everyone was so nice.

I have yet to make a Windows and OS X build, but if you’re not afraid of a terminal and have svn, wget, patch, and python you can get and run my modified version of photograbber via:

svn checkout http://photograbber.googlecode.com/svn/trunk/ photograbber-read-only -r38
cd photograbber-read-only/
wget -qO - http://cs.ucsb.edu/~bboe/public/patches/photograbber-r38+bboe.patch | patch -p0
echo "Run via: python pg.py"
python pg.py

This weekend I competed in the Defcon 18 Qualifiers with team Shellphish. We unfortunately only placed 15th, nonetheless, it was an exciting and challenging weekend. Below is my write up for the Forensics 200 challenge.

I don’t recall the phrasing they gave (if you remember it please let me know), however this was the file they provided: The caption read, “find the key” and linked to this file: f200_02b7b50f575759cff7.tar.lzma

Running file on f200_02b7b50f575759cff7.tar.lzma simply returned data however fortunately the lzma extension was useful to identify that this possibly be uncompressed with 7zip (lmgtfy lzma file). On ubuntu there is a package called p7zip thus:

bryce@sarek:f200$ mv f200_02b7b50f575759cff7.tar.lzma f200_02b7b50f575759cff7.tar.7z
bryce@sarek:f200$ p7zip -d f200_02b7b50f575759cff7.tar.7z
 
7-Zip (A) 9.04 beta  Copyright (c) 1999-2009 Igor Pavlov  2009-05-30
p7zip Version 9.04 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,8 CPUs)
 
Processing archive: f200_02b7b50f575759cff7.tar.7z
 
Extracting  f200_02b7b50f575759cff7.tar
 
Everything is Ok
 
Size:       1730560
Compressed: 487746
bryce@sarek:f200$ tar -xf f200_02b7b50f575759cff7.tar

This extracts 1121 png image files.

bryce@sarek:f200$ ls | head
f200_02b7b50f575759cff7.tar
IMG_0001.png
IMG_0002.png
IMG_0003.png
IMG_0004.png
IMG_0005.png
IMG_0006.png
IMG_0007.png
IMG_0008.png
IMG_0009.png

Using pnginfo one can verify that each image is similar with respect to its attributes (width, height, bitdepth, channels, etc.). Using manual image inspection we see a mostly transparent image with some white and black pixels. Thus using python’s imaging library (PIL) we simply write a quick program that will combine all the images into one:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

#!/usr/bin/env python
import Image
 
def main():
    new = Image.open('IMG_0001.png')
    w, h = new.size
    for i in range(2, 1122):
        im = Image.open('IMG_%04d.png' % i)
        data = im.split()
        for pixel, value in enumerate(im.getdata()):
            if value[3] != 0: # not transparent
                x, y = (pixel % w, pixel * 1. / w)
                new.putpixel((x, y), value)
    new.save('f200_result.png')
    new.show()
 
if __name__ == '__main__':
    main()

Running this eventually produces the following image, from which the is.gd url http://is.gd/ced7F is the key. Yeah Sexy CPR.

I started using Facebook in October of 2004, during my freshmen year of college, just a few months after it became available at UCSB (June 24). To sign up I was required to have an email ending in @ucsb.edu and once signed up I could easily communicate with people at UCSB, and more specifically people in my courses; a useful feature they later removed. Eventually I had the ability to post and tag photos and well as create events to which I could easily invite all my friends. Life was simple and so was Facebook.

Over time, Facebook expanded to allow anyone with an email address to sign up. While I initially didn’t like that just anyone could join our exclusive college website, I gained the ability to easily keep in contact with a few more people thus I was happy with that feature. However, when Facebook released their developer API, I was initially happy, though soon later I was sorely disappointed with the way Facebook provided third-parties with my information. Primarily I had always wanted a way to make myself appear non-existent to third-party applications, except for those I explicitly allowed. This is a feature that now appears utterly unobtainable.

These last few weeks Facebook has been hot news with respect to their dishing out of users’ information to Microsoft, Pandora, and yelp. Their information whoring has prompted Minnesota’s Senator, Al Franken, to include instructions on his website detailing how to restrict the information flow. Wired’s article, “Facebook’s Gone Rogue; It’s Time for an Open Alternative” pretty much covers many of the recent issues.

Thus, like many other Facebook users, the recent changes included the straw that broke the camel’s back and therefore I am prepared to leave Facebook behind for good. However, before doing so there are a few tasks that I wanted to accomplish prior to leaving. These tasks are as follows:

1. Obtain a copy of all the albums I am tagged in
2. Import my Facebook contacts’ emails into my Google contacts
3. Import my Facebook contacts’ profile pictures into my Google contacts
4. Import my Facebook contacts’ birthdays into my Google contacts

At this point I have accomplished all these tasks, thus while I now can leave Facebook, there are a few precautionary things I’d like to do before finally clicking that delete button such as removing image tags and posting a link to this blog posting. Below I will detail the steps to accomplish the tasks I have outlined.

Obtain a copy of all the albums I am tagged in

Quite plainly there are two primary routes, which I think are good. The first involves a Firefox extension, facePAD, and the second involves a desktop application, photograbber.

FacePAD simply will allow you to right click on an album link and download all of its images to your download directory. The two drawbacks to this approach are that it’s not automated, and if you want to group pictures into folders by albums, you’ll have to first manually create the folders, and second move all the downloaded pictures into that folder. On the upside, this is a great way to quickly download a few albums, and it’ll download any picture that is viewable by you.

The other approach is using the opensource desktop application Photograbber. Photograbber is written in python thus allowing it to run on Windows, OS X, and Linux. As of today, photograbber is at revision 38, which only allows you to download the pictures that you are tagged in, or the pictures that a particular friend is in. These pictures are all downloaded into the same directory and thus to me is somewhat worthless.

Fortunately, because photograbber is open source, and it was written in python I fairly quickly was able to hack together modifications to photograbber that allow it to accomplish precisely what I want. Which is, downloading albums that I am tagged in where photos in each album are grouped into their own folder. I have made available my patch to photograbber which you can use to patch the source and run yourself. Alternatively, I have also repackaged photograbber for Mac so you don’t have to mess with patching the application (download). A window’s repackage will follow shortly. Running the application is pretty simple, but if you need any assistance please post a comment. I have now made a zip file available which should work on windows. Unzip, and run pg.exe.

While photograbber is fully automated once started, it does have a few drawbacks. The first drawback is that it has a more restricted view of pictures than what you see when you browse Facebook manually. This is because your both you and your friends can restrict applications from having access to your photo albums, thus to get 100% coverage of all the albums you are tagged in, you’ll unfortunately have to manually check to see what albums were downloaded and compare that to all the albums you appear in. Additionally pictures cannot be organized by date (I haven’t checked the EXIF data), which I think is an important feature. I’ll probably add in this feature sometime before I actually delete my account, so check back for updates.

Import my Facebook contacts’ emails into my Google contacts

I didn’t find a direct way to solve this problem, though I’ll admit I didn’t look very hard as I stopped with the first result I found for “export facebook emails to gmail”. The method suggested involves using a Yahoo email account. From Yahoo’s contact page you can select “Tools” followed by “Import…” and finally you have the option to import from Facebook. This uses Facebook Connect, so you will have to authorize Yahoo to access your information. Following the import you can select “Export…” from the “Tools” menu and export to a “Yahoo! CSV” file, which you download to your computer. Finally, on the Gmail contact page you can select “Import” from the upper right and provide it with the downloaded CSV file.

Import my Facebook contacts’ profile pictures into my Google contacts

This may not be essential for many people. I wanted to accomplish this so that all of my contacts on my Android Device have pictures next to them. For this step I used phaceboogle. At first I thought this web service was a little sketchy, however it uses both Facebook Connect, and Google’s OAuth thus I was cool with using it. Phaceboggle did almost exactly what I wanted, though rather than copying the thumbnail picture, phaceboogle copied the un-cropped profile picture, which in some cases appears with the wrong aspect ratio. Nevertheless, I was satisfied and thus completing another step.

Import my Facebook contacts’ birthdays into my Google contacts

In order to accomplish this task as I describe, you will need an Android Device as it utilizes the Android application, ebobirthday. This app connects to Facebook via the Facebook API and copies over all the birthdates for your contacts. The next time your phone syncs to your Google account, many of your contacts will have their birthdate field completed. Now you can add a birthday calendar to your Google Calendar (See section ‘More’). Task completed!

Well that’s all. Feel free to post alternative methods to accomplish these tasks, or other tasks, in the comments. Bye bye Facebook!

Connor, you are welcome and happy birthday buddy!

I was just texting with a friend who happens to be learning C++ in which I wanted to text her, “the STL is your friend when you figure it out”, however at that point in the message I was over the dreaded 160 character limit thus I shortened that to “the STL is your friend WYFIO”. I immediately sent a second message defining WYFIO and laying claim to defining the acronym. You’ll note I’m not laying claim to the first use of the acronym; that’d just be ridiculous.

My next thought process was that someone else had to have already defined this particular acronym. However a Google, bing, and YAHOO! search turned up no results save for this one (post #2) which uses the acronym in the same way I am, but fails to define it.

Let it be known that here on this 9th day of February, 2010 I, Bryce Boe, first defined the acronym WYFIO as being short for “when you figure it out”.

I even previously tweeted this announcement just to lay my claim a tad quicker.

If you asked me just six months ago if I liked to run, I would have said “Hell no!” I have hated running for as long as I can remember, aside from the bit of running required in the sports I played. Running in PE up until tenth grade was a requirement to receive an ‘A’ in the class, and running was occasionally required training for volleyball. Running was never something I would have chose to do, thus as I drank more and exercised less throughout college I became terrifically out of shape.

Now if you ask me if I like to run, I would say, “Why yes I do”. Furthermore, if you ask if I liked to run for beer, I would respond, “Hell yes! I love running for beer!” Not only has my opinion of running changed, but also I am in the best shape of my life; well at least since starting College. So what changed?

Well, on June 27, 2009 I was introduced to a running club by the name of the Hash House Harriers, or more specifically the Santa Barbara kennel, H3SoB, which is short for the Hash House Harriers of Sant’o Barbara. A quick read through the Wikipedia entry on the Hash House Harriers will inform you the club started in 1938, Kuala Lumpur by a guy named ‘G’ in which the runners would celebrate completing the run with beer. Thankfully that tradition continued on, and I am now part of it.

Let me tell you, I was initially skeptical about the idea of running and drinking. After all, it sounds kind of counterintuitive. However, after just one run, I knew this was something exactly for me. Before I started running with the Hash House Harriers I drink quite a bit on a weekly basis and I could hardly run. After just a few weeks running with the Hash House Harriers I drank about the same however could noticeably run much further distances without tiring. Now, after nearly 6 months of running I not only can run even further distances, but believe it or not, I actually drink less on a weekly basis. I’ve become more health conscious, and further more I gained the desire to wake up on a Saturday to simply go for a run, sans beer. Running with the Hash House Harriers has changed my life.

I want to give you a little more insight on this club I am now part of as aside from the beer aspect, we are not a run-of-the-mill running club. Those that have been to what we refer to as a hash run, or simply a hash, know we don’t usually stay on well-defined trails. Rather, we often run off trail through storm drains, or through what we like to call the “shiggy stuff” which usually means low brush, weeds, or for those who are not careful, poison oak. The three things nearly every run calls for are flashlights, vessels (cups), and virgins (first time hashers).

Additionally, despite having never been part of a normal running group I would wager we are much more vulgar and thus much more fun. For instance, when the virgin’s are initiated in the club, they are informed, “After consuming your drink, you must place your vessel over your head“, at which point everyone joins in for our “Head! Who said head? I’ll take some of that, and I did” chant which goes on to talk about uprooting trees and shrubs and flowers with viking horns on our head. Wait, who said head?

Furthermore, after a runner has been with us for a decent period of time they are awarded with a sufficiently vulgar and maybe not so relevant name. Those that brought me to the hash were named Pre Med Head, who is now in medical school, and Try This Tip who cuts meat. Some other names in our group are PokeHer, Spread Eagle, Maxlode, and Picassol. After my 8th, or so, run I was given the name Cunt Cockulust because I was slightly mistaken for liking calculus. Many people in the hash just call me The Count so I go around doing my impressions of The Count from Sesame Street which usually is, “I love to count, things. Ah ah ah ah.”

Finally, we fairly often run in themes such as the bikini run, the toga run and the miniskirt run. Of those, I’ve only taken place in the bikini run as pictured below. Additionally everyone dressed up for the Halloween run, which happened to be my Virgin Lay also known as the first time I hared, or led the run; I dressed up as Super Hare which is also pictured below. The hare(s) are those that leave the pack about fifteen minutes early and using chalk and flour mark the trail everyone else runs. Hares often leave what are called checks to deceive the runners into following a false trail if they don’t pick up the real trail.

I encourage you all to seek out your local Hash House Harrier kennel and get a first hand experience. If you need more convincing check out the Washington Post article titled, “Making a Hash of Exercise“. On-On!

In order to deliver drive-by downloads, the teams had to bring users to their webpage by boosting their search result on the search server for desired search terms. The users’ browsing pattern was they would search for a random word from a news site and then visit one of the resulting pages chosen according a Pareto distribution, meaning the top search result would have the highest probability of being picked with the last result the lowest.

The challenge there was to figure out how to boost the search results for particular keywords, unfortunately most of the teams simply duplicated the news site; this definitely was the safest strategy though slightly disappointing to me as I wrote the search engine code. I can claim that neither the search engine, nor the index server broke during the competition. The crawler had a few issues with some team’s pages, which I quickly fixed. I have made the search engine code available that was used in iCTF09.

In addition to writing the search engine, I wrote three challenges for the competition. The first challenge I wrote was called “The Difference”, or Forensics 4. The only information people were given was the title and the following image. The first step to solve this challenge is recognizing the image is the xkcd comic “The Difference“.

After downloading the original image one should notice the two files are different sizes with the original being 32KB and my image 46K. At this point a pixel-by-pixel comparison is required which reveals that 60 of the pixels differ and the values of the differing pixels are always larger in my image. This is where “The Difference” first comes into play by subtracting the value in my image from that of the original.

Doing so produces the results shown in the_difference_values.txt. Each line contains the difference value and the index of the pixel. The values in this list range from 33 to 41 with exceptions 10, 12 and 15. Additionally the numbers appear in groups of three where last number in the group is sometimes larger. After a bit of pondering the realization sets in that ASCII lowercase values range from 97 to 122 with space being 32 and hyphen 45. It should hit you like a ton of bricks that you need to sum the groups of three together and then convert the number to its ASCII value.

The string “sixty-four seventeen” now appears and you think you’ve solved the challenge, but no, it’s not quite done. This is where “The Difference” comes back into play in that you have to subtract these two numbers to get forty-seven. There were 596 submissions for this challenge of which 2 were correct. I’ve made the source to both generate and verify this challenge available.

The second challenge I wrote had 75 submissions and 0 correct solutions. This challenge was one of the Trivia 3 problems that had a text file and the following blurb associated with it:

Rummaging through the attic one afternoon Sally found a notebook which on the cover read ‘What happened in 95?’. On one of the pages she found the following two paragraphs of text which appeared as gibberish to her. She copied the two paragraphs down as best she could, however there were forty-five characters which she couldn’t make out from the first paragraph, which she simply neglected. What are those 45 characters?

The concept behind this problem is a one-time pad, where a completely random sequence of characters the same length as the original text is generated and then XORed with the original text to create the cipher text. At this point the original text can be discarded and recovered by XORing the cipher text with the one-time pad.

The challenge to this problem is finding where the 45 missing characters go, and what they are. One additional difficulty is figuring out that the ASCII values need to be subtracted by 95 prior to XORing so the numbers being XORed fit within 32 bits. This can be discovered by taking the min value that appears in the paragraphs which is 95 corresponding to ‘_’ and additionally the max value that appears which is 126 corresponding to ‘~’. The ’95′ that appears in the blurb was to hint at this.

Finding the places where the characters should go requires XORing the current one-time pad with the cipher text. Plain text English with no punctuation or spaces will result at the beginning of the output until nonsense text is output. This is the point in which a character needs to be added to the one-time pad. This process needs to be repeated for all 45 characters.

At this point, XORing the one-time pad character with the desired character will result in the missing character. The solution is the concatenation of all the missing characters. This is “sdnalsilennahcehtdnuoradnuofebnacsehcnarbidun” which is “nudibranchescanbefoundaroundthechannelislands” reversed.

I’ve made the code to generate this challenge available. You’ll notice it loops until the one-time pad contains the characters of the solution. The original text is a passage from Joyce’s Dubliners that contains the oldest reference I know of to the phrase, “how goes it“. I did not write a solution for this problem.

The third and final problem I wrote was called 0xDEAFBABE or Trivia 2. A binary file (renamed to include .mid to resolve content type issues) was provided which the program file correctly informs that it is a midi audio file. By playing the file one hears about one note a second. Many teams tried to figure out what the notes were as they would be on a musical scale, such as c sharp or b flat, however that was a bit over thinking it. The solution was simply to print the ASCII character that corresponds to each midi note number. These numbers range from 0 to 127, thus they’re perfect for text. The solution to this was “does this not sound super cool?” There were 207 submissions of which 21 were correct. I have made my source to generate and verify this challenge available.

If you competed in the iCTF and attempted or completed any of these challenges let me know what you thought and how long you spent on them. I recall someone in the chat yesterday saying they were losing their sanity trying to solve “The Difference”; that gave me a good laugh

Both orientations occur the week before school starts, and cover topics such as what UCSB students are like, sexual harassment, time management, and whom to contact if […]. Our department’s required TA training seminar takes place in fall quarter, thus not prior to when TAs might have to teach, and was strictly presentation based. In none of these three requirements was there any discussion of pedagogy, the study of being a teacher.

I went into my first discussion section with no idea on how to lead an effective discussion section, other than what I personally saw in my undergraduate Computer Science discussion sections, most of which I did not attend. Those that I had attended were not discussion sections, but rather mini-lectures with a question and answer session, thus this was the model I followed. Rather than trying to help students make real world connections with what they were learning, I simply presented material and answered questions they had about assigned projects.

Despite how I now believe I could had a much bigger impact, I was given Outstanding Teaching Assistant awards both quarters I TAed based on my end of quarter reviews by the students, and professor recommendations. I am not attempting to brag here, but rather show that my instruction as a TA is representative of our department’s best TA instruction and that’s sad. It can be so much better.

Undergraduates coming to UCSB for Computer Science, benefit from being taught by some of the brightest minds in the country. However, if you think these professors are much different when it comes to pedagogy than our TAs, you are sadly mistaken. Our professors are not hired based on their teaching ability, but rather their ability to bring in research money through grants.

I’ve confirmed this through discussion with faculty involved in the hiring of new faculty. I asked one of our faculty members, “Suppose two candidates were identical in research experience, but one had more teaching experience. Would that give the candidate an edge?” The response I received was that no two candidates are ever identical in research experience, and that any teaching experience is completely neglected.

In my opinion this is unacceptable. At UCSB undergraduates are paying more than they should to receive a college education. At the very least they should be taught by professors who not only know what they’re teaching, but know how to teach it as effectively as possible. Teaching shouldn’t be a requirement, as many faculty members see it, but rather an opportunity to share knowledge.

Before I conclude, I want to mention that we have some incredible faculty in UCSB’s Computer Science department, many of who are not only excellent researchers, but also excellent instructors. Of those I’ve discussed this with, none of them had any formal teacher training, therefore are solely relying on past experience and observer feedback. Those that are excellent instructors have done incredibly well, but they could do so much better. As can I.