How The Timeline Was Integrated

So… yesterday, I posted the news item about launching the new timeline feature on the Unified Republic of Stars. Today, I wanted to discuss a little bit about how I did it, since extending MediaWiki in this way was… well, sometimes it was frustrating and sometimes it was just plain maddening.

Data Conversion

Because my timeline, previous to this project, was essentially just a list of events separated by headers for each year, I needed to convert this data into an actual database format that could be manipulated and read by my extension.

The first step was designing a database that would hold the settings I thought I might actually use. The result was a table structure like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
CREATE TABLE urswiki_timeline_events (
        id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
        start_year INT NOT NULL,
        end_year INT NULL,
        age INT NOT NULL,
        order_index INT NOT NULL,
        title VARCHAR(255) NULL,
        caption VARCHAR(255) NULL,
        notes TEXT NOT NULL,
        icon_url VARCHAR(255) NULL,
        image_url VARCHAR(255) NULL,
        text_color VARCHAR(10) NULL
);
CREATE TABLE urswiki_timeline_events (
        id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
        start_year INT NOT NULL,
        end_year INT NULL,
        age INT NOT NULL,
        order_index INT NOT NULL,
        title VARCHAR(255) NULL,
        caption VARCHAR(255) NULL,
        notes TEXT NOT NULL,
        icon_url VARCHAR(255) NULL,
        image_url VARCHAR(255) NULL,
        text_color VARCHAR(10) NULL
);

(There is more. For instance, I have a category table and a mapping table between events and categories. Additionally, because this is a sci-fi wiki, I also have an “age” table that defines the three ages of the URS. But this gives you an idea.)

This allows me to define a range of years (because I don’t need to go as granular as days and months) the way to order it, a title, caption, notes field, icon, image, and a different text color for a single item. Relatively flexible.

Now, I needed to get the data into the database. For this, I wrote a 40 line script to turn this:


=Age of Colonization=

===2035===
* [[Space Vehicles of The Republic|Sanctuary Space Vehicles]] founded on Earth in Las Vegas, Nevada.

===2051===
* The book Testimonies is written by Joel Goldman, a physics graduate student, and Peter James, a philosophy doctoral candidate. Initially regarded as a joke, it would later become the foundational document of the [[Church of the Cosmic Angel]] religion.

Into this:

1
2
INSERT INTO urswiki_timeline_events VALUES (0, '2035', '2035', '1', 1, NULL, NULL, 'Sanctuary Space Vehicles founded on Earth in Las Vegas, Nevada.', NULL, NULL, NULL);
INSERT INTO urswiki_timeline_events VALUES (0, '2051', '2051', '1', 1, NULL, NULL, 'The book <u>Testimonies</u> is written by Joel Goldman, a physics graduate student, and Peter James, a philosophy doctoral candidate. Initially regarded as a joke, it would later become the foundational document of the Church of the Cosmic Angel religion.', NULL, NULL, NULL);
INSERT INTO urswiki_timeline_events VALUES (0, '2035', '2035', '1', 1, NULL, NULL, 'Sanctuary Space Vehicles founded on Earth in Las Vegas, Nevada.', NULL, NULL, NULL);
INSERT INTO urswiki_timeline_events VALUES (0, '2051', '2051', '1', 1, NULL, NULL, 'The book <u>Testimonies</u> is written by Joel Goldman, a physics graduate student, and Peter James, a philosophy doctoral candidate. Initially regarded as a joke, it would later become the foundational document of the Church of the Cosmic Angel religion.', NULL, NULL, NULL);

That was actually pretty easy. The code is here:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
$input = file('./data.txt');
$output = fopen('./data.sql', 'w+');
 
$age = 1;
$year = 0;
$order = 1;
$count = 0;
echo 'Size of file: '.sizeof($input)."\n";
foreach ($input as $lineNumber => $line) {
        if (strpos($line, "===") !== false && strpos($line, "===") == 0) {
                $year = trim(substr($line, 3, 4));
                $order = 1;
                //echo 'Found year: '.$year."\n";
 
        } else if (strpos($line, "=") !== false && strpos($line, "=") == 0) {
                if (strpos($line, 'Colonization') !== false) {
                        $age = 1;
                        echo "Found: Age of Colonization\n";
                } else if (strpos($line, 'War') !== false) {
                        $age = 2;
                        echo "Found: Age of War\n";
                } else if (strpos($line, 'Betrayal') !== false) {
                        $age = 3;
                        echo "Found: Age of Betrayal\n";
                }
 
        } else if (strpos($line, "*") !== false && strpos($line, "*") == 0) {
                $notes = trim(addslashes(substr($line, 1)));
                $insertStatement = "INSERT INTO urswiki_timeline_events VALUES (0, '$year', '$year', '$age', $order, NULL, NULL, '$notes', NULL, NULL, NULL);\n";
                fwrite($output, $insertStatement);
                $order++;
                $count++;
        }
}
//echo 'Last Note: '.$notes."\n";
echo 'Counted '.$count.' number of timeline items.'."\n";
 
fclose($output);
$input = file('./data.txt');
$output = fopen('./data.sql', 'w+');

$age = 1;
$year = 0;
$order = 1;
$count = 0;
echo 'Size of file: '.sizeof($input)."\n";
foreach ($input as $lineNumber => $line) {
        if (strpos($line, "===") !== false && strpos($line, "===") == 0) {
                $year = trim(substr($line, 3, 4));
                $order = 1;
                //echo 'Found year: '.$year."\n";

        } else if (strpos($line, "=") !== false && strpos($line, "=") == 0) {
                if (strpos($line, 'Colonization') !== false) {
                        $age = 1;
                        echo "Found: Age of Colonization\n";
                } else if (strpos($line, 'War') !== false) {
                        $age = 2;
                        echo "Found: Age of War\n";
                } else if (strpos($line, 'Betrayal') !== false) {
                        $age = 3;
                        echo "Found: Age of Betrayal\n";
                }

        } else if (strpos($line, "*") !== false && strpos($line, "*") == 0) {
                $notes = trim(addslashes(substr($line, 1)));
                $insertStatement = "INSERT INTO urswiki_timeline_events VALUES (0, '$year', '$year', '$age', $order, NULL, NULL, '$notes', NULL, NULL, NULL);\n";
                fwrite($output, $insertStatement);
                $order++;
                $count++;
        }
}
//echo 'Last Note: '.$notes."\n";
echo 'Counted '.$count.' number of timeline items.'."\n";

fclose($output);

Maybe it’s not the cleanest written script, but you get the idea.

Needless to say, that was the easy part. After that, the MediaWiki hacking begins.

The way I saw this working was three-fold:

  • A special page for creating/updating/deleting timeline events.
  • An API extension for returning events in a JSON feed.
  • A parser hook for inserting the timeline on any given page with a few parameters.

I’ll go over each of these in turn, since each was actually built one at a time.

MediaWiki Special Page

Special Pages are pages that show up in the administration section of the wiki. Literally Special:SpecialPages. These pages are intended to give an administrator some degree of control over the site’s system messages, see what the most linked to and wanted pages are, and so on. The Timeline special page, in this case, was going to be an administration panel where an administrator could see existing timeline events, edit or delete them, and create new ones.

The coding style guide for MediaWiki says that extensions with new special pages should call their file “SpecialEXTENSION.php”, so this one became SpecialTimeline.php.

Timeline Administration

Fig. 1

All in all, coding this page was not that difficult. Showing the list of events for a given year was just a database query as was building the “year cloud”. Where the problems began was when I decided to make the actual editing and deleting of events AJAX driven rather than the standard POST. The problems were caused because MediaWiki’s AJAX handling is… unique.

The docs are pretty clear about how to register for AJAX events. It requires something like this:

1
2
3
$wgUseAjax = true;
$wgAjaxExportList[] = 'ursUpdateTimelineEvent';
$wgAjaxExportList[] = 'ursDeleteTimelineEvent';
$wgUseAjax = true;
$wgAjaxExportList[] = 'ursUpdateTimelineEvent';
$wgAjaxExportList[] = 'ursDeleteTimelineEvent';

The problem is in trying to discover the actual method signature for what those callbacks should be. If you read the MediaWiki documentation, you’ll see that it’s actually pretty… skimpy. So this required diving into the source itself. The first was looking at the ajax.js file that MediaWiki uses for client side requests then adding logging into AjaxDispatcher.php to figure out how requests are then handled.

Essentially, the method signature is defined to be one argument for each element of a parameter array. Ugh…

To get around this, I defined my request as so:

$.ajax({
	url: wgScriptPath + "/index.php?action=ajax",
	type: 'POST',
	data: {
		rs: 'ursUpdateTimelineEvent',
		rsrnd: new Date().getTime(),
		rsargs: [
			eventObj
		]
	},
	success: function(data) { }
});

The rs parameter is the name of one of the registered AJAX callback functions. I was unclear whether rsrnd was required but… what the hell.

Normally, rsargs would take an array of unnamed parameters but I’ve gotten around that by defining a JavaScript object and passing that as the single element of the array. This allows me to define the function like so:

1
2
3
4
5
6
function ursUpdateTimelineEvent($args) {
        $id = $args['id'];
        $startYear = $args['start_year'];
        $endYear = $args['end_year'];
        $title = $args['title'];
}
function ursUpdateTimelineEvent($args) {
        $id = $args['id'];
        $startYear = $args['start_year'];
        $endYear = $args['end_year'];
        $title = $args['title'];
}
Timeline Administration

Fig. 2

With this, I was able to take all the values, process them, insert, update, or delete from the database and return a JSON value to be used by the frontend.

I know this sounds like it wasn’t a big deal but it took a good couple of hours to dig through the source just to get a sandbox experiment working. Like I said, none of this is documented.

Anyway, with all of this, I was able to get basic CRUD functionality working and the Special page was essentially complete.

One final note… to lockdown the page from non-administrators, I had to add a couple of function calls to the object definition. These aren’t well documented so I’ve included the code here.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
class SpecialTimeline extends SpecialPage {
        function __construct() {
                parent::__construct( 'Timeline', 'editinterface' );
        }
 
        public function SpecialTimeline() {
                global $wgLanguageCode;
                SpecialPage::SpecialPage('Timeline');
                wfLoadExtensionMessages('Timeline');
        }
 
        function execute($query) {
                global $wgUser;
                if (  !$this->userCanExecute( $wgUser )  ) {
                        $this->displayRestrictionError('Timeline', 'editinterface', true, 'doTimeline');
                        return;
                }
 
                $this->setHeaders();
                doTimeline();
        }
}
class SpecialTimeline extends SpecialPage {
        function __construct() {
                parent::__construct( 'Timeline', 'editinterface' );
        }

        public function SpecialTimeline() {
                global $wgLanguageCode;
                SpecialPage::SpecialPage('Timeline');
                wfLoadExtensionMessages('Timeline');
        }

        function execute($query) {
                global $wgUser;
                if (  !$this->userCanExecute( $wgUser )  ) {
                        $this->displayRestrictionError('Timeline', 'editinterface', true, 'doTimeline');
                        return;
                }

                $this->setHeaders();
                doTimeline();
        }
}

Note the __construct() method and the displayRestrictionError method call. I had to look up the latter in the source to find out exactly what to call in there. What the former does is keep the link to the page from appearing on the Special Pages list while the latter blocks users from landing on the page if they know the URL.

API Extension

The next part involved extending the standard MediaWiki API with a new module to export all the timeline events. I’ve had some experience extending the API to add additional fields to modules that already existed but I’ve never done one from scratch.

The documentation, as far as it goes, isn’t so bad when it comes to extending APIs. The main thing that needs to be explained is how the data actually gets sent to the feed. See the code below.

1
2
3
4
$result = $this->getResult();
$tags = array('events');
$result->setIndexedTagName_internal( $tags, 'event' );
$result->addValue(null, 'events', $events);
$result = $this->getResult();
$tags = array('events');
$result->setIndexedTagName_internal( $tags, 'event' );
$result->addValue(null, 'events', $events);

Line 1 fetches the ApiResult object that is basically what gets sent in whatever format is requested from the API. Line 2 defines what the parent element will be named. Line 3 (and this isn’t really well documented at all) tells the API that each event is a child of events. Then, finally, line 4 adds the data to the result.

The $events variable contains an indexed array of either PHP objects or associative arrays holding your result. Remember that the member names will be used to define sub-fields of each event. You can see an example of what gets output here.

So… that wasn’t that big of a deal. Next up…

Parser Hook

A parser hook is what allows a site editor to use a code like <timeline/> and have a timeline suddenly rendered when the page is saved. You can also pass parameters through the hook using basic HTML parameter tags that can then affect how things get parsed. I wanted to use this because I thought it would be interesting to tag timeline events by planet and wiki category so that planet pages could list timelines of events specific to that planet and things like, say, baseball leagues could list events such as Republican Series winners and other league specific events related to that topic.

Essentially, what I wanted was a master timeline, then a whole bunch of sub-timelines that parsed and displayed data in a bunch of different ways. For a storyworld like URS, I thought that might be infinitely interesting. We shall see if it actually is. To make this happen, I first made MediaWiki aware that a new parser hook existed.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
$wgHooks['ParserFirstCallInit'][] = 'timelineInt';
function timelineInt(&$parser) {
        $parser->setHook( 'timeline', 'timelineRender' );
        return true;
 
}
 
function timelineRender( $input, $args, $parser, $frame ) {
        $planet = $args['planets'];
        $category = $args['categories'];
 
        $planets = "";
        if (strlen(trim($planet))) {
                $planets = implode('|',split(',', trim($planet)));
        }
        $categories = "";
        if (strlen(trim($category))) {
                $categories = implode('|',split(',', trim($category)));
        }
 
        $output = '<div id="dynamic_timeline" class="timeline-default" style="height: 350px; margin-top: 20px; margin-bottom: 50px;" data-planets="'.$planets.'" data-categories="'.$categories.'"></div>';
 
        return $output;
}
$wgHooks['ParserFirstCallInit'][] = 'timelineInt';
function timelineInt(&$parser) {
        $parser->setHook( 'timeline', 'timelineRender' );
        return true;

}

function timelineRender( $input, $args, $parser, $frame ) {
        $planet = $args['planets'];
        $category = $args['categories'];

        $planets = "";
        if (strlen(trim($planet))) {
                $planets = implode('|',split(',', trim($planet)));
        }
        $categories = "";
        if (strlen(trim($category))) {
                $categories = implode('|',split(',', trim($category)));
        }

        $output = '<div id="dynamic_timeline" class="timeline-default" style="height: 350px; margin-top: 20px; margin-bottom: 50px;" data-planets="'.$planets.'" data-categories="'.$categories.'"></div>';

        return $output;
}

The timelineRender method is then where the hard work of actually generating the output happens. In this case, since the Timeline is Javascript based, all I needed to do was ensure that the right div gets generated and passes the properties on to be read by the script.

Because this is generated at render time and, thus, output with the rest of the HTML document, the JavaScript could then use a $(document).load() handler to execute the timeline loading script. This script would then check to see if any parameters are set, pass those to the API to restrict the data set, and render the appropriate timeline.

This actually proved to be the easiest part, since I’d actually done more complicated parser hooks in the past and all I needed to do was output a simple HTML element with a little data.

The End… For Now

This post has gotten really long, so I’m going to leave out the actual JavaScript code I wrote to initialize the SMILE Timeline library and populate the date ranges. Suffice it to say, that it took a little experimentation to get it working but the documentation is fairly good despite being relatively minimal.

Needless to say, I’m pretty happy with how this whole thing has turned out. MediaWiki, though a pain to extend because of crap documentation, turned out to be flexible enough to incorporate a feature like this while a couple of libraries like the XOXCO Tags library made the admin nice and pretty.

If anyone’s interested in the source code, leave a comment and I’ll package up the whole thing and post it so others can use it. The only proviso is that it’s been written for MediaWiki 1.16.5 since that’s what my server is running and some things may have changed (specifically resource loading).

Questions? Comments?

Comments