My pretty face [ László Monda's Blog ]
Exploring the cyberspace, one quadrant at a time!

Archive for the 'twitter-backup.sh' Category

Let's backup our tweets using twitter-backup.sh

Sunday, September 5th, 2010

Update (2010-10-23): I've just uploaded twitter-backup.sh to Google Code.

I've written a very simple BASH script to backup my tweets.  It's very easy to use:

0
1
2
3
4
5
6
7
8
9
$ ./twitter-backup.sh
Usage ./twitter-backup.sh TWITTER-USERNAME
$ ./twitter-backup.sh mondalaci
2010-09-05 17:46:04 URL:http://twitter.com/statuses/user_timeline/mondalaci.xml?page=1 [40899/40899] -> "twitter-backup-mondalaci-2010-09-05_17-46-03/1.xml" [1]
2010-09-05 17:46:06 URL:http://twitter.com/statuses/user_timeline/mondalaci.xml?page=2 [42928/42928] -> "twitter-backup-mondalaci-2010-09-05_17-46-03/2.xml" [1]
2010-09-05 17:46:07 URL:http://twitter.com/statuses/user_timeline/mondalaci.xml?page=3 [42753/42753] -> "twitter-backup-mondalaci-2010-09-05_17-46-03/3.xml" [1]
2010-09-05 17:46:09 URL:http://twitter.com/statuses/user_timeline/mondalaci.xml?page=4 [42784/42784] -> "twitter-backup-mondalaci-2010-09-05_17-46-03/4.xml" [1]
2010-09-05 17:46:11 URL:http://twitter.com/statuses/user_timeline/mondalaci.xml?page=5 [42872/42872] -> "twitter-backup-mondalaci-2010-09-05_17-46-03/5.xml" [1]
2010-09-05 17:46:11 URL:http://twitter.com/statuses/user_timeline/mondalaci.xml?page=6 [6465/6465] -> "twitter-backup-mondalaci-2010-09-05_17-46-03/6.xml" [1]
2010-09-05 17:46:12 URL:http://twitter.com/statuses/user_timeline/mondalaci.xml?page=7 [75/75] -> "twitter-backup-mondalaci-2010-09-05_17-46-03/7.xml" [1]

I've followed the holy way of Unix, the KISS principle when developing this little script:

0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
#!/bin/bash
 
if [ $# -ne 1 ]; then
    echo "Usage $0 TWITTER-USERNAME"
    exit 1
fi
 
username=$1
 
backup_dir=twitter-backup-$username-`date +%Y-%m-%d_%H-%M-%S`
mkdir $backup_dir
 
page=1
while true; do
    dest_file=$backup_dir/$page.xml
    wget -nv -O $dest_file http://twitter.com/statuses/user_timeline/$username.xml?page=$page
    page_size=`stat -c%s $dest_file`
 
    if [ $page_size -lt 1000 ]; then
        break  # We've reached a final, empty page so let's exit from the loop.
    fi
 
    page=$(($page+1))
done
 
rm $dest_file  # Delete the last, empty page.