Replicating A Shared Photo Stream to Dropbox
This is an adventure I went on to share the photos in my Shared Photo Stream synced via iCloud to Dropbox so some of my family members who aren’t fortunate enough to bask in the glory of Jobsian utopia (read Android users) can still bask in the glow of my adorable daughter.
Part 1: The Obvious
Anyone starting down this journey will no doubt find themselves googling for and finding some great articles like
- Sync your iOS Photo Stream with Flickr, Dropbox or anything else or
- Automatically Sync Photo Stream to Dropbox
which will inevitably point you towards useful tools like
Those are all well and good and will have you either indirectly or directly poking at the
~/Library/Application Support/iLifeAssetManagement/assets/sub folder. If your goal is purely to rip out all of the images from your personal Photo Stream (the thing that backs up your Cameral Roll to that cloud thing everyone keeps talking about) and shove them into another folder/disk/cloud storage location, then read no further. If you’re looking to replicate specific and Shared Photo Streams (the iDevice based way to share a photo album with other cool kids with iThings), then read on intrepid Interneter.
Part 2: Two steps forward, one step back..
So back to that weird folder. If you go take a look at it, you’ll see that there are a bunch of long, randomly generated folder names that look a lot like UUIDs. Case in point:
$ ls ~/Library/Application\ Support/iLifeAssetManagement/assets/sub/ | head -5 0101da2e4b5c0cbb04cb629f450a74e92da655b1ef 010330a1dc5cb15f4ae350fbcbba4c9e9c64fcd754 010350cb5ff5423c25ac772e3dad355a1a46a1a259 0103e9abfafdc7cde42a11080c6474b70184017ae3 01113fc2c3b9a1685bd799d522c315fc9b26fae7b0
In checking out how many of these folders there were:
$ ls ~/Library/Application\ Support/iLifeAssetManagement/assets/sub/ | wc -l 155
I noticed something interesting. The number of folders is the same as the number of images that iPhoto told me I had:
It matches! We might be on to something… Still nothing about where the Shared Photo Stream pictures live, but definitely closer.
Part 3: There’s data in them thar hills!
Now if I’m a developer and I’ve got a bunch of randomly generated unique IDs flying around, I’m going to store them somewhere so that my application can reference them sanely. Most cases that’s going to be in a SQLite database because they’re a fast and light (no SQL server required) way to store structured data to disk. Let’s see what we can find:
$ cd ~/Library/Application\ Support/iLifeAssetManagement/ $ find ./ -name \*.db .//ILifeAssetManagement.db
Let’s crack that open and see what’s in there
$ sqlite3 ILifeAssetManagement.db .tables AMAsset AMAssetsToImport AMUploadTask AdminDataEntity
nothing too obvious, let’s grap some random data from an interesting looking table (pre-filtered to protect random data of mine that may be sensitive):
$ sqlite3 -header -csv ILifeAssetManagement.db "select uuid, filename from AMAsset limit 5;" uuid,filename 012cb5246303ae259eb324f66de94d33606cfd3ab6,IMG_1963.JPG 01fe04a0f4f0f75d46afccf5a00518707a439e024a,IMG_1957.JPG 01f0be3aa5a8b90b9fd582a3e3e07747f865cb1037,IMG_1956.JPG 019e24d6a91dd83e3cff5e959fee200efbfe23640b,IMG_1955.JPG 015a0fde068d3062572fb9391d6fa32cae585e0744,IMG_1926.JPG
Definitely some interesting things there. If you check on your own system, you can verify that there are indeed files like
~/Library/Application\ Support/iLifeAssetManagement/assets/sub/012cb5246303ae259eb324f66de94d33606cfd3ab6/IMG_1963.JPG in that sub folder. Not necessarily helpful to our end goal, but it’s interesting data and it will provide clues for how we can get the pictures we do care about.
Part 4: A chance discovery
While tab-completing to hop in and out of the
sub folder, I couldn’t help but notice tripping over the
sub-shared folder. Given that shared streams are the name of the game, this looks very promising.
$ cd ~/Library/Application\ Support/iLifeAssetManagement/assets/sub-shared/ $ ls | head -5 016CF733-783E-4E77-868E-7781899E320D 01B037B0-2895-4F73-B228-B3E8AB4AEB10 01EB9EA8-AA28-49C6-A24C-A0CC73EEC917 01FC4414-BC97-4EE9-A49F-A00C292E0E0D 021A0E74-1CB3-48FE-A24F-CF094C9313E4
Hooray, more UUIDs…
Let’s try that directory count trick again. I know that I happen to have two shared Photo Streams on iCloud account, and, via iPhoto, I know that one has 233 pictures and the other has 169.
$ ls ~/Library/Application\ Support/iLifeAssetManagement/assets/sub-shared/ | wc -l 402
The good news is that this is the right place to be poking around (233 + 169 = 402), the bad news is that Apple has decided to dump all of the pictures from all of the shared Photo Streams into a single folder, identified only by UUID. To get to the end of this, we’re going to have to find some datasource that helps us separate the two.
Since I know from my previous digging that there’s a SQLite database tracking the UUIDs and image names, there’s probably a similar source for the shared pictures. At this point, I got a bit lazy and just decided to grep the whole App Support folder for a couple of the UUIDs that I knew about and see what turns up.
$ cd ~/Library/Application\ Support/iLifeAssetManagement/ $ fgrep -R '016CF733-783E-4E77-868E-7781899E320D' ./ Binary file ./state/albumshare/1431844669+11256150207431692312/Model.sqlite matches Binary file ./state/albumshare/1431844669+11256150207431692312/PersonID.sqlite matches $ fgrep -R '01B037B0-2895-4F73-B228-B3E8AB4AEB10' ./ Binary file ./state/albumshare/1431844669+11256150207431692312/Model.sqlite matches Binary file ./state/albumshare/1431844669+11256150207431692312/PersonID.sqlite matches $ fgrep -R '01EB9EA8-AA28-49C6-A24C-A0CC73EEC917' ./ Binary file ./state/albumshare/1431844669+11256150207431692312/Model.sqlite matches Binary file ./state/albumshare/1431844669+11256150207431692312/PersonID.sqlite matches
PersonID.sqlite keep coming up, very interesting…
It turns out that
Model.sqlite is where the party is, there’s a table in it called Albums which correlates the human friendly name to a UUID used elsewhere in the database. The particular table of interest is AssetCollections which maps the Album’s UUID to the indiviual asset (image) locations. A simple join will tell me which UUIDs are associated with which album so I can extract the ones I care about out of
Part 5: Tying it all together
Now we know where the images are stored, how to differentiate the ones we want, and where they’re headed (Dropbox). Let’s
write a quick bash script to put this all together spend 4 hours writing a Ruby class and script to automate this.
There’s a few minor twists and turns, but the overall process isn’t different than what I described. The script I wrote does all of the above and saves the images into a folder named after the Share Photo Stream in the destination folder specified.
The code is free and up for grabs on my Github page at https://github.com/braxtone/shared-photo-stream-backupper. Please take a look and fork it, I’d love to see what other people build on top of it.
As a quick reference, here’s the help text:
$ ./photo_stream_backup.rb -h Usage: photo_stream_backup.rb [options] -s, --streams X,Y,Z The name of one or more streams that will be backed up, use "all" to back all of them up -d, --destination DEST The destination folder for the images found, ie ~/Dropbox, etc -v, --[no-]verbose Run verbosely -h, --help Display this screen
and here’s what an example run looks like:
$ ./photo_stream_backup.rb -s all -d '~/Dropbox/' Backing up stream 'Wife and Life' Backing up 169 images... Backing up stream 'Family pics' Backing up 239 images...
I’ll leave automating this as an exercise for the reader, save to say that things like automated folder watching is probably the easiest way forward and will be more deterministic than a cron job.