Ian Lewis' Blog

I just set up Plagger to send me e-mails to my cellphone and gmail when there are updates to my replies rss feed on twitter. This took way longer than anticipated because of how long it took to install and configure Plagger properly.

First installing it took a long time. I installed it via cpan and Plagger has a huge number of dependencies. Many of the dependencies are not set properly so cpan fails to install it more often then not. I had to force install the IO::Socket::SSL package because the tests failed. I didn't need TLS for mail and it's an optional package so I'm not sure why I needed SSL in the first place.

Once plagger was installed there isn't a terribly large amount of info about how to set it up. You basically have to create a config.yaml file in the same directory as the plagger executable or create a yaml file with any name and specify the location with the -c flag.

The yaml files configuration are easy to read but what options are available for each plugin is a bit obtuse without any examples to look at. There are some but it took me a while to find them since they aren't listed on the plagger cookbook which is linked to from the homepage. To find out what options are available to each plugin you'll either need to read the examples or the bottom of the plugin's source code files to get a description of what it does and it's options. I'm also not sure what all of the types of plugins do but I gather than Subsription plugins are plugins for reading RSS feeds and incoming data, Publish plugins are for publishing RSS feeds, email, html etc., and Filters are for parsing and/or modifying the data.

I used three Plugins to achieve what I wanted to do which was to get e-mail sent to my gmail account and my mobile. I used the Subscription::Config, Filter::HTMLScrubber, and Publish::Gmail plugins. The config plugin pulls from an rss feed and caches the results. It only passes on changes to the feed to other plugins. The Gmail plugin is essentially a SMTP/SMTP TLS plugin which sends e-mails when it gets updates from the subscription plugins. Unfortunately the version of the plugin that installed with cpan had a bug as it called the MIME::Lite::extract_addrs function which is not present on the machine where I was running plagger. I needed to update the Gmail.pm plugin file to the newest version in plagger's svn. That version includes a fix around like 214 to call the extract_full_addrs fuction if the extract_addrs fuction doesn't exist.

Next, my mobile doesn't support html mail very well so I had to scrub the HTML from the RSS feeds with the HTMLScrubber plugin. Unfortunately though the Gmail plugin sends e-mails as text/html encoded so I needed to further modify the Gmail.pm file to mail as text/plain around line 90.

After that the template file that is used to generate the e-mail, gmail_notify.tt, was missing so I needed to get that from svn as well and I put it in my .cpan/assets/common directory and added that directory to the templates search path in my config.yaml. The contents of the template are also html so I needed to edit the template to remove all the html tags.

After that I just put config.yaml in the same directory as my plagger executable and added a line to execute plagger every 10 minutes to my crontab. This also had it's glitches as plagger prints debug and info messages to stderr so you can't just do "plagger > /dev/null" you have to have something like "plagger > /dev/null 2>&1" so that cron doesn't email you with plagger info messages every 10 minutes.

Anyway, after a few hours of messing with it, it finally worked. Here is the config.yaml I used.

Just played a bit with graphviz and made some simple graphs. Graphviz is an open source suite of programs for generating graph diagrams from a number of text formats, the simplest of which is the dot format.

Dot format is a simple language that is used to describe the graph that will be generated. Here I created a very simple directed graph (that's a graph with arrows) using login ids for some friends on Twitter:

digraph G {
   a2c -> IanMLewis -> Voluntas;
   a2c -> jbking -> IanMLewis;
}

Essentially the parser will look over all of the strings of text and create a node for each unique string. It then links the nodes that were linked together in the text. Here IanMLewis shows up twice but only one node is created and it's linked to twice, by jbking and a2c.

So you can generate the graph image for this by running one of a few command line tools. Each one generates graphs in different styles. The dot tool, generates a graph in a linear fashion based on what unique node names it sees first. I generated the following diagram with this command:

dot -Tgif test.dot -o out.gif

neato is another tool which will generate a diagram from the same file but it generates it a bit differently. Instead of a directional diagram it tries to generate a graph with the least energy configuration. That is to say, it chooses a relatively pleasing arrangement for the graph.

neato -Tgif test.dot -o out2.gif

There are many, many options for changing the colors, layout, shapes and text of nodes etc. that I won't go into but check out the graphviz documentation.

It essentially looks like the area where graphviz would be most useful is for mashups with other applications where you would generate graphviz dot formatted files based on some data from the application. One example application listed on the website is generating diagrams from sourcecode automatically using doxygen. Others might include dynamically generating network diagrams.

Just watched this video of Bob giving a presentation at C4[1] about erlang. It's about a year old and Bob had been using erlang for almost a year at that point. Bob always seems to have his hands in things just before they become super popular. Erlang seems to be one of those things. JSON was another. He's even the guy that coined the term jsonp before that was really popular. Anyway, highly informative. I might give erlang a try a little later on.

I have wanted a good dictionary application that supports a number of dictionaries/dictionary formats for studying Japanese but I've been sort of frustrated that there is no single application that does what I would like. I have looked mostly at stardict and epwing because they are closest to the type of dictionary I want. Specifically I have the following requirements:

1. It must work in Linux (epwing and stardict)
   
2. It must work in Windows (epwing and stardict)
   
3. It must support epwing format (epwing)
   
4. It should ideally support an open dictionary format like stardict. (stardict)
   
5. It must support lookup via a popup (epwing and stardict)
6. It must support installing dictionaries as a non-root user (epwing)
   
7. It should support minimization to the system tray (stardict)
   

1 and 2 and 5 are supported by both stardict and epwing but epwing's popup support is less than ideal and epwing supports nothing but epwing and doesn't minimize to the system tray. The way epwing looks up words from multiple dictionaries is also less than ideal,

Stardict on the other hand supports most of the things I want but doesn't support epwing or any way of converting the dictionaries from epwing to stardict format. There are a number of tools to read epwing dictionaries so I think this could be done, but I don't think stardict dictionaries currently support images or media like epwing dictionaries support so converting epwing dictionaries would lose that data. Stardict also has a  number of strange plugins and a net dictionary protocol which seems strange. I'm not sure if the server side code is open either.

I've been looking ot the code for each of the dictionaries and it seems like while epwing's code is simpler and easier to read, stardict supports most of the features I want already so it would be better to explore how to add the features I want to add. Development on both projects is pretty much halted so I should be able to add features at my own pace without stepping on current development.

I'm not looking forward to spending a lot of time on it but my frustration with dictionaries has reached a bit of a boiling point.

Kris Zyp (the author of the JSONSchema proposal) mentioned jsonschema on his blog at json.com. Thanks Kris!!

I just released the first version for a project that I've been working on since the Python Onsen. It's a validator for JSON Schema written in Python. It's based on the JSON Schema Proposal Second Draft.

The source can be downloaded here: jsonschema-0.1a.tar.gz
The documentation can be found here: jsonschema (version 0.1a) documentation

JSON Schema's purpose is to allow validation of JSON documents much like XML Schema, DTD. You can use it to define what kind of data should be present in the document as well as the structure of the data. You might have some JSON for a contact like so:

And you could describe this in JSON Schema with the following:

This can be validated with something like the following Python code:

It can be easily extended to include support for new properties or to override the default validation for standard properties so I think it could be used for a wide range of applications. I plan to use it for a Form Maker application (code) on GAE. Let me know what you think!