cablegatesearch.net

cablegatesearch.net copied to clipboard

cablegatesearch.net Copyright (C) 2011 Raymond Hill

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/gpl.html.

rhill 2012-02-27 Important note: Files here no longer reflect cablegatesearch.net in its present state, much work as been performed since the last commit to enhance significantly performance, add features, and restructure the site. This has been true for a while, I just thought I should leave a note here for your information.

rhill 2012-02-28: Forgot the most important information: cablegatesearch-create-and-populate-all.sql.7z still contains the redacted versions of cables, while the web site contains all the cables, non redacted.

WHAT IS THIS?

Here are the files required to set-up a Web site which allows full-text search of the Cablegate's cables released by Wikileaks.

These files have been used by http://www.cablegatesearch.net/ since Dec. 9th as a free service to allow people to easily browse the released cables.

I consider the code here to be raw and still quite under development, and rather untidy and often messy, however, there has been interest to maybe reuse the code here in order to create an offline, portable full-text search of the cables, hence this early release.

Obviously I will keep working on this code, to make it more tidy, better documented, and to introduce optimizations as the number of released cables goes up, and performance issues are identified. Etc.

This website's code has been running on:

• Apache/2.0.52 (CentOS)
• PHP 5.1.6
• MySQL 4.1.20

This website's code has been developed on:

• Apache/2.2.16 (Ubuntu)
• PHP 5.3.3-1ubuntu9.3
• MySQL 5.1.49

The footprint is rather smallish, but you might want to adjust the cacher's settings in cacher.php: $CACHER_MAXITEMS_LO = 29000; $CACHER_MAXITEMS_HI = 30000; These indicate the max number of files in the cache directory. When the number of files is higher than $CACHER_MAXITEMS_HI, oldest files will be deleted so that no more than $CACHER_MAXITEMS_LO are left in the cache.

FILES

• browse.php: Experimental, not completed, to browse the tags used in the cables

• cable.php: Web page to display the content of a single cable

• cacher.php: Cache management, to save CPU cycles. Can be turned off. There need to be a directory named "cache" with read/write permission for all for the cacher to work properly. This was ported from a previous project, naming is inconsistent.

• cablegate-do.php: XMLHttpRequests' entry point

• cablegate-functions.php: Functions reused across multiple PHP files. It needs cleanup.

• cablegate-cable-functions.php: Functions related to cable-specific operations.

• cablegate-media-functions.php: Functions related to media-specific operations.

• cablegate-search-functions.php: Functions related to search-specific operations.

• cart.php: A tool to collate and persist a specific set of cables

• crowdsource-central.php: Live feed of #wlfind Twitter hashtag, latest comments, etc.

• dbconnect.php: To connect to the MySQL server, this is where you fill in your database name, user id, password.

• extras.php: Unsorted features are put in there

• finediff.php: FineDiff class, used to by cable.php to render cable content

• footer.php: Contact, credits, etc. This file is included as footer in the main PHP files.

• header.php: The HTML code shared by all web pages, which appears just below the title bar

• globals.php: This file is included by a few PHP files. It's likely going to be replaced soon by a serialized PHP variable useful across many pages on the site.

• history.php: Documents the addition, modification, deletion of cables by Wikileaks

• index.php: Redirect to search.php

• manning-logs-diff.php: The alleged logs of alleged Cablegate leaker Bradley Manning

• media.php: A tree for all sources/articles for the published cables

• overview.php: An overview of the published/unpublished cables by country & embassy

• release-notes.php: Release notes, I'm not very good at keeping this up to date.

• search.php: The main web page to launch a full-text search

• uorigin.php: Unclassified cables by origin

• mootools-core-1.3-loader.inc: HTML snippet to load Mootools Core 1.3. I use a proxy loader which requires a key, which is generated according to the domain name cablegatesearch.net. You will have to change this to your own key, or simply load directly mootool-core.js (see http://mootools.net)

• CSS files: cablegate.css cablegate-list.css = to be used for list of cables cablegate-cart.css = to be used for cart operations

• Javascript files: cablegate.js: more stuff (need to restructure these files) cablegate-cart.js: cart management cablegate-search.js: list of suggestions for search keyword cablegate-uorigin.js: allow to filter published/unpublished cables for a specific embassy mootools-more.js: Mootools More, for visual effects, date handling, etc.

• Art files (credits to http://openiconlibrary.sourceforge.net/): background1.png bookmark-add.png bookmark-del.png bookmark-ghost.png bookmark-info.png bookmark.png close.png collapsed.png edit-clear-2.png edit-delete-5.png expanded.png external-link.png go-next.png go-previous.png knewsletter.png list-add-4.png list-remove-4.png noforn-mini.png noforn.png open.png view-pim-news.png /images/magnifier.png

• cablegatesearch-create.sql: MySQL statements to create the structure of the tables required to run the site

• cablegatesearch-create-and-populate-all.sql.7z: MySQL statements to create the structures of the tables required to run this site, and to fill them with the data of the latest snapshot of released cables. Since this file is way too large to host here (currently ~1GB), download it using this torrent magnet: magnet:?xt=urn:btih:753c465276ea16465581df723abae965357fe2f0&dn=cablegatesearch-create-and-populate-all.sql.7z&tr=http%3A%2F%2Ftracker.openbittorrent.com%3A80