Tag Archives: Open Source Software

Shading between curves in R

As a R learner programmer, it took me unconscionably long to work out how to use polygon to shade under and between curves, despite searches of the R manual and R-help – they just didn’t start far enough back. So, for anyone else scratching his or her head over polygon (and so I can find it again when I forget how it’s done), here are the series of steps I went through to figure it out.

The function takes in an x vector and a y vector, defining a set of coordinates that, in order, taken in order trace around the area to be shaded. Thus for a set of points 1-10, defined individually as x.1, y.1 to x.10, y.10,

x <- c(x.1,x.2,x.3,x.4,x.5,x.6,x.7,x.8,x.9,x.10)
y <- c(y.1,y.2,y.3,y.4,y.5,y.6,y.7,y.8,x.9,y.10)

The area inside these points is shaded by

polygon(x,y,col=gray(0.8))

To apply this to two curves, both normal distributions between -3 and 3, one half the height of the other

x <- seq(-3,3,0.01)
y1 <- dnorm(x,0,1)
y2 <- 0.5*dnorm(x,0,1)
plot(x,y1,type="l",bty="L",xlab="X",ylab="dnorm(X)")
points(x,y2,type="l",col="red")
polygon(c(x,rev(x)),c(y2,rev(y1)),col="skyblue")

The first half of the x-vector in the polygon is just the values of x itself, corresponding to the part of the polygon that is tracing out the upper curve along increasing values of x. The second part for of the x-vector in the polygon is the reverse of x, corresponding to the part of the polygon that is tracing out the lower curve along decreasing values of x. The first part of the y-vector is the y values of the upper curve, and the second part of the y-vector is the y values of the lower part of the curve.

That shaded the area between the curves along the full plotted range.

To shade only a defined portion, say the area from x=-2 to x=1.

x <- seq(-3,3,0.01)
y1 <- dnorm(x,0,1)
y2 <- 0.5*dnorm(x,0,1)
x.shade <- seq(-2,1,0.01)
polygon(c(x.shade,rev(x.shade)),c(dnorm(x.shade,0,1),0.5*dnorm(rev(x.shade),0,1)),col="yellow")

To shade the same defined portion as a gradient, from red to yellow (from the built-in heat.colors palette):

x <- seq(-3,3,0.01)
y1 <- dnorm(x,0,1)
y2 <- 0.5*dnorm(x,0,1)
x.shade <- seq(-2,1,0.01)
par(oma=c(1,1,1,1),cex=0.7)
plot(x,y1,type="l",bty="L",xlab="X",ylab="dnorm(X)")
points(x,y2,type="l",col="gray")
l <- length(x.shade)
color <- heat.colors(l)
for (i in 1:l)
{
polygon(c(x.shade[i],rev(x.shade[i])),c(dnorm(x.shade[i],0,1),
0.5*dnorm(rev(x.shade[i]),0,1)),border=color[i],col=NA)
}

This draws a succession of individual polygons between the curves, adjusting the color along the gradient as it goes. (Note that the loop above is for i in one to letter-L).

TiddlyWiki: a demonstration

I expressed my enthusiasm for TiddlyWikis in general terms in [cref favorites-tiddlywiki a previous post], since I find them mindbogglingly useful (to borrow a phrase) for keeping running notes on just about any topic. As I set up a TiddlyWiki to capture notes for a course project, it occurred to me that I might contribute a step by step description of how I go about customizing a new Tiddlywiki.

First step is to download an empty TiddlyWiki from http://www.tiddlywiki.com, courtesy of a large, friendly, blue button. That places a file called empty.html on my Desktop, so the next step is to rename it to something appropriate.

Then I open the page in Firefox, remembering to permit Javascript. The default view shows a GettingStarted tiddler front and centre, which invites the user to set the SiteTitle and SiteSubtitle (which should be self-explanatory), and stipulate the tiddlers to appear on the MainMenu and DefaultTiddlers (which are opened when the site is launched). Click on the images to show at full size.

Before I modify any content, I select options in the sidebar, opening up the options submenu. I enter the name I wish to use to sign my edits, uncheck SaveBackups and check AutoSave. The browser will issue a warning if one attempts to navigate away from unsaved changes, but I prefer to have my changes saved automatically rather than risk a distracted click on the wrong button. SaveBackups tends to generate a confusing bunch of files, although there is the option to put these in a separate subdirectory from the main file, which might be useful, and is accessed through AdvancedOptions. Changes to options are saved in the form of a cookie, so will reset to default when cookies are deleted.

Back under GettingStarted, I click SiteTitle to open that tiddler, click edit (beside the title) and enter “Demo Tiddlywiki”. A similar edit to SiteSubtitle changes that to “making tiddlywiki show its stuff”. The other two I leave, since I have some more customization to do.

Modifying the appearance: adding a template

The style is a default, and I have a couple of styles I prefer, so in a separate tab in Firefox I browse over to to TiddlyThemes and call up the Haemoglobin template. Built in to default TiddlyWiki is the ImportTiddlers plugin, accessed either through the list of Shadowed Tiddlers under the More tab on the right sidebar, or under the Import menu in the backstage toolbar. The backstage toolbar is accessed by clicking on backstage, which either appears in the extreme upper right hand corner of the TiddlyWiki page, or which can be made to appear by moving the mouse over that spot. Enter the URL for the desired template in the box provided, select the desired tiddlers – in this case HaemoSideBarOptions, PageLayout, contentFooter, and StyleSheet.

If ImportTiddlers does not work, I cut and past the contents of these Tiddlers from source into my new TiddlyWiki. It may be important to do so in a certain order: for the Haemoglobin style, the side bar should be added before the page layout, or the link to create new tiddlers disappears. The new page layout places the contents of the MainMenu in the coloured strip across the top.

Adding functionality through plugins: addition of the tagCloud plugin

I find a tag cloud a compact and effective index, and use the tagCloud plugin, available from TiddlyTools, to generate a tag cloud. Using ImportTiddlers, or manually, I load or copy the plugin over to my new tiddlywiki. Manual loading involves copying and pasting the code from TiddlyTools into a new tiddler in my tiddlywiki named (exactly) tagCloud and tagged as (exactly) systemConfig. The tag systemConfig is required for Tiddlywiki to recognize the plugin.

I then create an tiddler for the tag cloud itself called Topics (Tag cloud) (or some variant of) and containing the call to the tagCloud plugin, <<tagCloud systemConfig>>. The error that appears disappears on page reload.

Then I add the tag cloud tiddler to the DefaultTiddlers and MainMenu, using [[Topics (Tag cloud)]], the double square bracket being needed to that the Tiddlywiki code will recognize the two words as a cross reference. Now the tag cloud appears front and centre on loading, and a link to it appears in the main menu, which in the restyled Tiddlywiki runs right to left.

Creating new content

To illustrate the creation of a tiddler, here is the open edit box for one called What’s this?, showing the markup required to create an external link, in this case to Wikipedia. Making a new entry involves clicking new tiddler, on the upper right sidebar and, presented with the editing panel, entering a unique entry title, the text and the tags to go with the text. In the body of the entry, anything that in the form of WikiWords or enclosed in [[double square brackets]] is treated as a link to a current or potential entry (current entries are bold, nonexistant entries are italic). The escape character, to avoid unwanted link creation, is ~, eg, ~TiddlyWiki will not create a link. In the tag box, any single word or phrase in double square brackets, is used as a tag. More information about Tiddlywiki markup, including links and images, is found at TiddlyWiki for the rest of us.

The styled Tiddlywiki, with default tiddlers What’s this? and Topics (Tag cloud), is here:

In recent versions of TiddlyWiki, the styling is controlled entirely from the StyleSheet, which is a great improvement on having it scattered over several tiddlers. The stylesheet itself is CSS, and can be edited for further customization.

Curating

Keeping a tidy TiddlyWiki requires some decisions about preferred terminology, best made prospectively or at least early, and attention to detail. It is case sensitive, so if the title of a tiddler starts with a capital, then all references to that tiddler need to start with a capital. Should nouns in tags be singular, plural, or mixed, eg, article(s)? Tag-variants and non-existant tiddlers tend to proliferate. A list of all tiddlers appears by date of last edit under Timeline on the lower right hand menu and alphabetically under All. Nonexistent tiddlers, ie, those referenced but not defined, appear under More > Missing; this list can be used to hunt down tiddlers spawned by a mis-spelling. Previously used tags appear under the Tags tab, or in the tag cloud. The tag cloud updates when the wiki is reloaded or the tiddler is closed and reopened. Size of the font indicates the frequency of use. Click on a tag, and a drop-down list shows which tiddlers it applies to; this, too, can be used to hunt down tag variants and bring them in line.

I keep a collection of definitions, reviews and useful sites on my Tiddlywiki Backpack page. For some elegant implementations, see: see the Tiddlyspot gallery, and Tiddlywiki in Action.

A few of my favourite tools: Wine

Wine (named in recursive fashion “Wine is not an emulator”) is used to run Windows applications on Linux; essentially, it interprets Windows commands for Linux and vice versa. I first met Wine in the form of Darwine (the “Dar” being for the underlying Darwin windowing technology in OSX), ported and maintained by Mike Kroneberg, and described at Low End Mac in an article by Alan Zisman. An alternative approach to installing wine, which I have not explored, is to install Wine via MacPorts.

The utility of Wine is that it does not require one to partition the hard drive, reboot, or to own and run any version of Windows. I have a full installation of Windows Vista Pro running on Parallels, but for certain lightweight, self-contained programs, I can launch Darwine and be running in a quarter of the time it takes for Parallels and then Vista to lumber into motion. WinBUGS, which I’ve been learning for Bayesian analysis, runs quite happily on Darwine, although I had to install by downloading and unpacking a zip file (per instructions for Vista or 64 bit systems), as the installer itself gave an error. After that, about the only tricky part was figuring out the file structure of the pseudo-C drive and how to access the files on the Mac, since they’re in an invisible folder. FirstBayes (Bayesian learning software) did not run, even after I figured out how to do the required registry hack, but then FirstBayes didn’t care much for Vista, either. Notepad++ runs without any trouble, as does Plant Studio (an old Windows program for building 3D plant models). Here is the official application database, rather games-heavy.

Darwine is no longer being developed, but its successor, now named Wine, is still in development. As my copy of Darwine survived beyond the Snow Leopard Event Horizon and still does what I need it to do, I will upgrade later.

A few of my favourite tools: Tiddlywiki

This started as part of my earlier post on DokuWiki and MAMP, and then grew to demand a separate entry. Tiddlywiki, a demonstration of über-Javascripting, goes places other wikis cannot. It is a completely self-contained, platform- and server-independent wiki, capable of being run from any rewritable media and on any system that supports a modern browser. Everything, script, entries and plug-ins, is included within a single .html file; installation consists of downloading the empty file, setting a couple of options (name and saving), and starting to edit.

Why? Because I as I said before, I like wikis, I work cross-platform, and I find I work best by separating out my projects. So, one Tiddlywiki per project. To date I have run Tiddlywikis on all three major operating systems, though using Firefox by preference as a browser.

A series of articles by Mark Gibbs at Network World (Jan – Feb 2009) covers the basics. Some of his readers express reservations about the quality of the documentation, but it occurs to me that that in part may be because much of the documentation appears in Tiddlywiki form (see, for example, tiddlywiki.com). A potential user coming to it without previous experience may need a few moments to decipher the conventions of the interface (in a traditional tiddlywiki, look to the right hand menu for actions and lists of previous entries), as well as come to terms with the jargon (although, in 2009, “tiddler” cannot seem any more whimsical than “tweet”).  Tiddlywiki for the rest of us provides a more conventional presentation of the basics, and tiddlywiki.org‘s documentation is built on a MediaWiki engine. Admittedly there are aspects, like the styling, that are intricate beyond simple documentation. I have reached the point that I can tinker with other peoples’ styles, but have not progressed beyond that; fortunately, there are a couple of styles I quite like.

My uses are admittedly rather bland: keeping ongoing research notes on emerging treatments in a disease area, keeping bibliographies, keeping a off-line, but still portable list of resources for my medical/regulatory writing activity, keeping a concordance for a loose series of short stories. Other people are much more adventurous, using tiddlywikis for home, education and commercial websites, presentations, weblogs, course notes, course materials, bibliographies, and developing and presenting scholarly papers – see the Tiddlyspot gallery, and Tiddlywiki in Action. As tiddlywiki is open source, developers have produced a number of variants, including a popular one one that incorporates the tasks-projects-place structure of Dave Allen’s Getting Things Done (GTD) strategy, complete with checkable to-do lists; I’ve flirted with this, but have always returned to Backpack.

My own collection of Tiddlywiki Resources page is here, on Backpack. I started it after I spent far too long hunting around for a site that contained a plug-in I knew I’d seen in passing and wanted to include. My page on Tiddlyspot (a service for hosting Tiddlywikis online, worth an entry in itself), on the German Imperial Navy, shows a number of my favourite modifications, including restyling to the Blueberry template (from Tiddlythemes) and the addition of a tag cloud.

A few of my favourite tools: DokuWiki and MAMP

From the first time I met one, I took to wikis for information gathering like a bluebottle fly to an old fish-head, partly because of their non-linear nature, and partly, since they were web-based, I could style and lay them out to suit myself. With a wiki, I can start collection information at any point and with the most rudimentary framework, grow and restructure at will, design my own menus, and create multiple cross-links between entries. My present wiki engines of choice are DokuWiki and Tiddlywiki. DokuWiki was originally designed for documentation in small companies, which makes it ideal for my present challenge of keeping up with my learning on four different stats packages (R, SAS, STATA and WinBUGS). I keep programming notes, reusable code snippets, a personal Rosetta stone of command-equivalents, and lists of links and references in the form of a DokuWiki run on a MAMP server (see below) on my MacBook.

Familiarity breeds comfort, but DokuWiki’s advantages are that I have found it straightforward to install and configure (particularly now that the user access controls are provided through a web interface), it is FOSS and platform independent, the developer is actively maintaining it, there is an active user community, the mark-up syntax is easy to work with and readable in the raw, and it does not require a database but writes everything to flat-files and folders. The latter, for me, is a key point: at my scale of operations, flat-files are just fine, and they have the major advantage that they are program- and platform-independent. There’s no database to get corrupted (been there, done that). There’s no vendor lock-in. If I ever want to migrate, I probably won’t be alone and someone will have written a tool; even if they haven’t, flat-files, meet Perl; I know you know each other .

For comparisons of wiki software, see the relevant page on wikipedia and WikiMatrix. A sampling of reviews: From Open Source Magazine (Badger, 2007), and Linux.com (Popov, 2008). Implementations I’ve noticed around the web are DokuWiki’s own wiki and the Zotero documentation.

As for the platform, I’ve used the built-in OSX server in the past, but turning on the OSX server involved opening ports that I didn’t necessarily want open, even if I had been confident in my editing of .htaccess. So when I found out about WAMP, I went looking for a Mac version. MAMP, which is intended as a development server with Apache, PHP, MySQL, and Perl. Download and installation of MAMP were straightforward with the help of an on-line tutorial and instructions, and from there it was a matter of setting up and configuring DokuWiki according to the instructions, with a little help from the equivalent instructions for WordPress – blogging software is a possible alternative to a wiki for keeping notes, if they are more linear.