My pretty face [ László Monda's Blog ]
Exploring the cyberspace, one quadrant at a time!

Archive for the 'Miscellaneous Projects' Category

Stick your file to a specific path with stickfile

Thursday, January 20th, 2011

Update (2011-04-22): Zach let me know in the meantime that there's a much easier way to implement stickfile in BASH:

0
1
2
3
while true; do
      inotifywait -q -q -e modify -e delete $2
      cp $1 $2
done

Moral of the story: I should have searched for inotify command line which would lead me to inotify-tools which contains inotifywait.

And now to the original post:

My employer uses SonicWALL NetExtender for his VPN needs. Saying that I'm not a fan of IPsec would be definitely an understatement, but my major problem is that NetExtender overwrites my resolv.conf upon every connection which screws the hostname resolution on my LAN from my laptop. chmoding or chowning resolv.conf doesn't help because it gets re-chowned and re-chmoded by NetExtender.

I was thinking about overwriting resolv.conf on a regular basis from a script but it seemed rather inelegant. But how should I do it otherwise? With inotify, of course.

Here's the script I've written which you should save as "stickfile" to a directory that is featured in your $PATH.

0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
#!/usr/bin/env python
 
import os
from sys import argv, exit
from shutil import copyfile
from pyinotify import *
 
def add_watch():
    global watch_manager, watch_descriptor, file_to_stick
    watches = watch_manager.add_watch(file_to_stick, IN_MODIFY | IN_DELETE_SELF)
    watch_descriptor = watches[file_to_stick]
 
def remove_watch():
    global watch_manager, watch_descriptor
    watch_manager.rm_watch(watch_descriptor)
 
class MyProcessEvent(ProcessEvent):
    def process_IN_DELETE(self, event):
        self.stick_file()
 
    def process_IN_MODIFY(self, event):
        self.stick_file()
 
    def stick_file(self):
        global file_to_stick, file_to_clone
        print 'Sticking %s as %s' % (file_to_clone, file_to_stick)
        remove_watch()
        copyfile(file_to_clone, file_to_stick)
        add_watch()
 
if __name__ == '__main__':
    if len(argv) != 1+2:
        program_name = argv[0]
        print 'Usage %s: FILE-TO-CLONE FILE-TO-STICK' % (program_name)
        exit(1)
 
file_to_clone = argv[1]
file_to_stick = argv[2]
 
watch_manager = WatchManager()
notifier = Notifier(watch_manager, MyProcessEvent())
add_watch()
 
while True:
    try:
        notifier.process_events()
        if notifier.check_events():
            notifier.read_events()
    except KeyboardInterrupt:
        notifier.stop()
        break

After I created a valid resolv.conf and saved it as /etc/resolv.conf.orig I only had to execute the following as root before starting up NetExtender:

0
stickfile /etc/resolv.conf.orig /etc/resolv.conf

Overclock.net Mechanical Keyboard Guide Atom Feed

Sunday, August 29th, 2010

I use RSS / atom feeds pretty much all the time to minimize information overload but the Mechanical Keyboard Guide of Overclock.net doesn't make my job any easier because they don't provide any feeds and the thread moves very fast.

I couldn't tolerate this anymore so I've created a webscraper that provides atom feeds for this thread. Parsing HTML into a DOM and executing XPath queries on the DOM is something that I have a vast amount experience with and this project didn't take a long time either. I've been testing it for more than a month and it's rock solid. The only glitch is that sometimes posts are randomized between very short time intervals which is a minor inconvenience.

The script below is executed on a hourly basis by cron and its content is saved to http://monda.hu/overclock-net-mech-keyboard.xml

0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
< ?php
 
include 'config.php';  // include $database_{servername, username, password, dbname}
 
$start_url = 'http://www.overclock.net/computer-peripherals/491752-mechanical-keyboard-guide-10000.html';
 
function DOMinnerHTML($element)
{
    // Borrowed from php.net
    $innerHTML = "";
    $children = $element->childNodes;
    foreach ($children as $child) {
        $tmp_dom = new DOMDocument();
        $tmp_dom->appendChild($tmp_dom->importNode($child, true));
        $innerHTML .= trim($tmp_dom->saveHTML()) . ' ';
    }
    return $innerHTML;
}
 
function page_to_entries($html)
{
    $domdocument = new DOMDocument();
    @$domdocument->loadHTML($html);
    $domxpath = new DomXPath($domdocument);
    $xpath = '/html/body/div[2]/div/div/div/div/div/table';
    $nodelist = $domxpath->query($xpath);
    $entries = array();
    foreach ($nodelist as $node) {
        $link_node = $node->firstChild->childNodes->Item(2)->childNodes->Item(1);
        $id = $link_node->textContent;
        $url = $link_node->getAttribute('href');
        $username = $node->childNodes->Item(1)->firstChild->childNodes->Item(1)->textContent;
        $comment = DOMInnerHTML($node->childNodes->Item(1)->childNodes->Item(2)->childNodes->Item(8));
        $entry = array('id'=>$id, 'url'=>$url, 'username'=>$username, 'comment'=>$comment);
        $entries[] = $entry;
    }
    return $entries;
}
 
 
function query($sql)
{
    if (($result=mysql_query($sql)) === false) {
        die(mysql_error());
    }
    return $result;
}
 
function register_entry_and_get_timestamp($id)
{
    if (!is_numeric($id)) {
        die("Entry ID is not numeric!");
    }
    $result = query("INSERT IGNORE INTO mechanical_keyboard_guide SET id=$id");
    $result = query("SELECT timestamp FROM mechanical_keyboard_guide WHERE id=$id");
    $row = mysql_fetch_assoc($result);
    $timestamp = strtr($row['timestamp'], ' ', 'T') . 'Z';
    return $timestamp;
}
 
function add_updated_timestamp(&$entries)
{
    for ($i=0; $i<count ($entries); $i++) {
        $entries[$i]['updated'] = register_entry_and_get_timestamp($entries[$i]['id']);
    }
}
 
function print_entry($entry)
{
    $url = $entry['url'];
    $username = htmlspecialchars($entry['username']);
    $comment = $entry['comment'];
    $updated = $entry['updated'];
 
    print "<entry>\n";
    print "     <title>$username</title>\n";
    print "     <link href=\"$url\"/>\n";
    print "     <id>$url</id>\n";
    print "     <updated>$updated</updated>\n";
    print "     <summary type=\"html\">< ![CDATA[$comment]]></summary>\n";
    print "\n";
}
 
// Set up MySQL connection.
if (mysql_connect($database_servername, $database_username, $database_password) === false) {
    die('Failed to connect to the MySQL server.  Please check the $database_servername, $database_username, $database_password variables in config.php');
}
if (mysql_select_db($database_dbname) === false) {
    die('Failed to select the MySQL database.  Please check the $database_dbname variable in config.php');
}
 
// Set up cURL.
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
 
// Fetch last page.
curl_setopt($ch, CURLOPT_URL, $start_url);
$last_page_html = curl_exec($ch);
 
// Get the page ID of the page before the last page.
$last_page_url = curl_getinfo($ch, CURLINFO_EFFECTIVE_URL);
preg_match('/-([0-9]+)\.html$/', $last_page_url, $matches);
$last_page_id = $matches[1];
$almost_last_page_id = $last_page_id - 1;
 
// Fetch the page before the last page.
$almost_last_page_url = "http://www.overclock.net/computer-peripherals/491752-mechanical-keyboard-guide-$almost_last_page_id.html";
curl_setopt($ch, CURLOPT_URL, $almost_last_page_url);
$almost_last_page_html = curl_exec($ch);
 
$almost_last_page_entries = page_to_entries($almost_last_page_html);
$last_page_entries = page_to_entries($last_page_html);
$entries = array_merge($almost_last_page_entries, $last_page_entries);
add_updated_timestamp($entries);
$last_updated_timestamp = $entries[count($entries)-1]['updated'];
 
 
print '< ?xml version="1.0" encoding="utf-8" ?>';
?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>Overclock.net's Mechanical Keyboards Guide</title>
<subtitle>Overclock.net's Mechanical Keyboards Guide</subtitle>
<link href="http://www.overclock.net/computer-peripherals/491752-mechanical-keyboard-guide.html"/>
<updated>< ?php print $last_updated_timestamp ?></updated>
<author>
<name>Overclock.net's Mechanical Keyboards Guide</name>
<email>laci_nospam@monda.hu</email>
</author>
<id>urn:uuid:60a76c80-d399-11d9-b93C-0003939e0af6</id>
< ?php foreach ($entries as $entry) print_entry($entry) ?>
</feed>
</count>

As for the SQL table structure, it's not particularly complex.

0
1
2
3
4
CREATE TABLE IF NOT EXISTS `mechanical_keyboard_guide` (
  `id` INT(11) NOT NULL,
  `timestamp` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY  (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;

How to make Konsole tab names always reflect the hostname of the current host

Saturday, August 23rd, 2008

Put this script to some directory that precedes /usr/bin in $PATH, like /usr/local/bin to wrap ssh:

0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
#!/bin/bash
if [ "x$KONSOLE_DCOP_SESSION" = "x" ]; then
    exec ssh $*
fi
 
function restore_old_session_name {
    dcop $KONSOLE_DCOP_SESSION renameSession "$old_session_name"
}
 
if [ "x$KONSOLE_DCOP_SESSION" != "x" ]; then
    old_session_name=`dcop $KONSOLE_DCOP_SESSION sessionName`
    hostname="${!#}"  # we expect the last arg to ssh to be the hostname
fi
 
trap "restore_old_session_name" SIGINT SIGTERM
 
if [ "x$KONSOLE_DCOP_SESSION" != "x" ]; then
    if [ "x$hostname" != "xecho FISH:; /bin/sh" ]; then
        dcop $KONSOLE_DCOP_SESSION renameSession "$hostname"
    fi
fi
 
/usr/bin/ssh $*
 
if [ "x$KONSOLE_DCOP_SESSION" != "x" ]; then
    restore_old_session_name
fi

Make Apple Trailers work with MPlayer plugin on Linux

Saturday, August 23rd, 2008

I've just written a Greasemonkey script that does the above job.

Kill Right Click Blocking Copyright Monkeys!

Tuesday, August 28th, 2007

Has it ever occured to you that you visited a site, wanted to do some mouse gestures and a nasty pop-up window said to you: "This page is copyrighted. You are not allowed to copy from my page."?

If your answer is yes, it's time to install my Greasemonkey extension. Works well against most of such pages.

Goodbye Feedy, Greetings Google Reader!

Thursday, July 26th, 2007

I’ve recently trashed my feed reader, Feedy. I started it about a half year ago, but it took too much time to maintain it. I knew about Google Reader for a long time, but I didn’t like it’s interface. Now that I use it for two weeks I have no doubts about its superior usability. I’m very satisfied with it.

Although I doubt anyone would use Feedy, I put it out, because I think it has a nice UI and a good database layout that prevents feed duplication pretty effectively so it may be a good base for developing a more robust feed reader.

feedy.tar.bz2 is not a release. A release is a carefully put together end product that is easy to install. Feedy is not easy to install, but check out the README file for some information.

Lastly but not leastly here are some screenshots:

The news page of my custom developed feed reader, Feedy
The config page of my custom developed feed reader, Feedy

How to Automatize HTTP Requests Using Greasemonkey

Thursday, May 31st, 2007

There are some sites that I use daily. The sad thing is, many of them don't provide me any mechanisms to log in automatically, so I have to always do it manually. I also had to fill the same form in on these sites and carry on the same operation over and over again. This was very frustrating for me so I decided to come up with a solution. Greasemonkey seemed to have the potential to solve my burning need.

I want to share the core function I've written that can help you in such scenarios. It's a simple, but very useful one.

0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
function submit_form(method, action, elements, target)
{
    var form = document.createElement('form');
    document.body.appendChild(form);
    form.method = method;
    form.action = action;
    for (element_i in elements) {
        var element = document.createElement('input');
        element.setAttribute('type', 'hidden');
        element.setAttribute('name', element_i);
        element.setAttribute('value', elements[element_i]);
        form.appendChild(element);
    }
    if (target) {
        form.target = target;
    }
    form.submit();
}

Requests are typically triggered based on the content of window.location.href or document.referrer.

Greasemonkey has some ugly annoyances, but fortunately most of them can be worked around. The LiveHTTPHeaders add-on can also help you tremendously constructing the fields of the forms.

Planet Archiver 0.2 Released

Sunday, August 6th, 2006

Planet Archiver was getting naughty so I made a small bugfix release, version 0.2.

Planet Gnome Archiver screenshot

Changes:

  • searchbar.php: Change the signature of monthdays() to be able to correctly compute the previous day.
  • searchbar.php: Force decimal conversion in leadingzero(number) to properly convert "08" and "09" to their decimal equevivalent instead of applying octal conversion.

Planet GNOME Archives Is Down, Planet Archiver Is Available

Wednesday, June 7th, 2006

Four days after I set up Planet GNOME Archives, Jeff Waugh, editor of Planet GNOME mailed me:

Hi László,

I found your blog entry about Planet GNOME Archives as a referer to the
Planet website.

Planet, as it was designed for Planet GNOME, specifically did not include an
archival feature, because we felt it was inappropriate for a third-party
site to archive blog entries. The original author may edit the entry, remove
it from his/her blog, or change the readership policy in the future and this
really is up to them, *not* a third-party archival site that they are unable
to control.

At some stage, Planet itself may include an archival feature, for aggregator
admins that wish to do this. But many admins will approach it the same way I
have for Planet GNOME: no archival whatsover, leave that in the hands of the
authors themselves.

So, this is a long-winded, history-filled way of suggesting that you should
not make your archive of Planet GNOME publically accessible.

Thanks,

- Jeff

For these very reasons, you should be careful about putting your archive out to the public. You should negotiate this with the admins of the Planet you wish to archive. I’ve made Planet GNOME Archives private and published its source as the project Planet Archiver 0.1. Jeff found it a cool hack that useful for many on-line resources and so do I.

Planet GNOME Archives Is Alive

Saturday, June 3rd, 2006

Update (2006-06-07): Planet GNOME Archives is no longer available publicly. See my post: "Planet GNOME Archives Is Down, Planet Archiver is Available" for the explanation.

I'm addicted to Planet GNOME. It's the richest resource I read on a regular basis. Lots of talented people writing here about mostly GNOME related and generally Free Software related stuff and one can learn a lot by reading their posts.

I had only one problem with it, though: PlanetPlanet is not able to generate feed archives, so it is not possible to go back in history and read the earlier posts that the GNOME folks made. If I forget to visit or can't visit Planet GNOME for 3 days or more, I drop behind and there are phases in my life when I don't have so much time and it's a pain in my ass. I could use a web based feed reader with cron, but unfortunately none of the readers are good enough for my choosey taste, so I had to choose a different approach to solve this issue.

Ladies and Gentlemen, I'd like to introduce Planet GNOME Archives [link removed]! (I like the name, came out with it myself.) It must be pretty damn intuitive to use, because of the extra JavaScript love I gave to this babe. Motto: "No readers get left behind!"

I haven't used JavaScript for a long time, but despite of this fact, it was really a pleasure. Enjoy!