Testing a Titan

We recently acquired a Nvidia GTX Titan graphical processing unit (GPU) for statistical computing at work, specifically double-precision floating point operations on the CUDA API. Before I lock it away in the server room I would like to see how it compares to my primary GPU at home – a Nvidia GTX 680, and my older GPUs – a pair of AMD Radeon HD 5770. This act of comparison is called benchmarking – running a number of standard tests and trials in order to assess the relative performance of a piece of hardware or software.

Continue reading Testing a Titan

OCO-2 v7 L2 Lite: Warn levels visualised

NASA Jet Propulsion Laboratory has this week released the OCO-2 L2 Lite files. These files contain a subset of the information in the standard OCO-2 L2 product. They are meant to be significantly smaller but still contain all necessary information for typical science analyses.

The lite files also contain warn levels which indicate the quality of the data points. At warning level 0 we can be very certain about the quality. Above warning level 12 we can expect significant error, and data points above warning level 15 should not be used at all. To see how much data is excluded at different warning levels I have mapped the data points for the month of May 2015.

Continue reading OCO-2 v7 L2 Lite: Warn levels visualised

The Australian Panopticon

The Panopticon is a theoretical prison designed by English philosopher and social theorist Jeremy Bentham in the late 18th century. The shape of the prison places every inmate within view of a watchtower. Although it is physically impossible for the watchman to observe all cells at the same time, the fact that the inmates cannot know when they are being watched means that all inmates must act as though they are watched, effectively changing their behaviour.

Continue reading The Australian Panopticon

This conversation is (not) private

Is it possible for you to have a private conversation with another person over the internet right now? You might be surprised by the number of ways ways your digital communications can be intercepted.

“The question is are we going to allow a means of communications which it simply isn’t possible to read. My answer to that question is: no, we must not” – British PM David Cameron

Continue reading This conversation is (not) private

Operating System support lifecycle

Windows 10 and CentOS 7 are good for about 10 years of free updates.

Operating System vendors provide free upgrades and security patches to keep our computers safe and functional during the support lifecycle. Some vendors announce the lifecycle ahead of time (see chart above), while others simply end support for old versions soon after each new release.

Without support your device gradually becomes less secure and less useful unless you can upgrade to a new OS version. Sometimes upgrading is not possible and you’re forced to replace a computer that isn’t broken.

To avoid troublesome upgrades or early obsolescence, always choose an operating system with a long planned support lifecycle. Windows 10 and CentOS Linux 7 will receive free security updates for the next 10 years.

TorGuard VPN on Android

If you connect to the internet through a public WiFi hotspot, or at school or at work, the metadata and contents of your online communications can be intercepted and viewed (or altered) by the network operator or another malicious party on the network. If you connect from an Australian residential address or personal mobile device, the Australian Government will be storing your metadata from 13 Oct 2015. One of the ways to protect your privacy is to send all of your internet traffic through a Virtual Private Network.

Continue reading TorGuard VPN on Android

Parallel Computing in Tessera RHipe

This tutorial will show you how to perform parallel computation on a Hadoop cluster in R using Rhipe, with CSV files as input and output.

The EVE Online computer game universe consists of 5201 solar systems for players to explore and conquer in virtual spaceships. Each solar system is connected to an average 2.6 other systems by jump gates which allow instantaneous travel between systems.

Continue reading Parallel Computing in Tessera RHipe

Converting HDF5 to CSV

Hierarchical Data Format 5 is a popular file format for storing and managing large amounts of data. It is the format used by NASA for their ACOS and OCO-2 data products, which both contain (among other things) column-averaged CO2 in units of dry-air mole fraction (Xco2). This tutorial demonstrates how to extract the average daily Xco2 value – and total reading count per day – from the HDF5 files into a CSV file for analysis in Excel or Gnumeric.

Continue reading Converting HDF5 to CSV