Mike Meredith's home page

Not an R tutorial

I've just been using Google to find basic information about key concepts in R, and found it difficult to find good sources. Many of the pages go into unnecessary detail, assume a background in programming in other languages, or use abstruse examples.

So some recommendations seem to be in order.

The closure assumption with SCR

In this post I want to explore what is "closure" in SCR, why it is important, and what happens if it is violated. I'll also explore some ideas for open population models which can be used to mitigate closure issues.

New version of overlap package

Animal activity patterns are in fact aligned to daylight intensity - in particular, sunrise, sunset and zenith - not to the time on your clock or GPS unit or camera ( Nouvellet et al, 2012 ). Since the times of these events vary with the time of year, using clock time instead of sun time can lead to wrong inferences. Nouvellet et al (2012) give the example of hunting behaviour of African wild dogs: impala are almost always taken before sunset and kudu after sunset, but this effect disappears if time is measured relative to 6pm instead of sunset.

Introducing the saveJAGS package

• you see progress as the files accumulate on the disk,
• you can load the output saved so far, run diagnostic checks and get preliminary results,
• if there's a problem, or if you already have enough iterations, you can terminate the run early without waiting for it to finish,
• you can save long chains and all the parameters you might need in files, then load into your R session only selected parameters and thinned chains,
• you no longer need to worry about "Cannot allocate..." memory errors when the JAGS run finishes,
• and of course you still have your samples in case of power outages or other disasters - that was the original purpose of the project,
• maybe Windows 10 is "another disaster": it will restart your computer to install updates, terminating anything you, the mere owner of the thing, wanted to run.

The Malaysian CRS monster

Does Akaike play dice?

Categorical covariates in JAGS

In R, the class factor is used for categorical variables. A factor has a levels attribute with the names of the categories, in alphabetical order by default, and the values are stored as integers giving the level number.

A toy example

I'll generate some simulated data for a logistic regression model (as that generalises to a lot of our other models). We have four players with different degrees of skill and a continuous covariate which does nothing.

Royle-Nichols models for occupancy

In the simplest model, the number available for detection at site \( i \), \( N_i \), is modelled as being drawn from a Poisson distribution with parameter \( \lambda \). This is the biological model. The observation model assumes that detection of each individual is a Bernoulli trial, individuals being detected with probability \( r \). The data do not show which individuals are detected, just whether the species was detected, and that is recorded if at least one individual is observed.

More on overflow and underflow

In the course of applying these ideas to maximum likelihood estimation in the wiqid package, a couple of new issues came up. We will revisit logSumExp and develop new functions to add together two vectors and to do matrix multiplication.

How long do animal signs remain visible?

If each animal produces p signs per day and signs remain visible for t days, then sign density will be:

S = D x p x t

where D is the density of animals. If we can estimate p and t , we can calculate D from S . In this post I want to provide code for estimating the persistence time of signs for retrospective studies of decay.

Probabilities and computer limitations

The solution is to work with logarithms of probabilities instead of the actual values [log(p) instead of p], eg, we routinely work with log-likelihoods. Multiplying probabilities is then simply a matter of adding up the logs. But sometimes we need to add up probabilities or calculate the complement, 1 - p, and we need to do that without falling in the 0 abyss or smashing into the 1 wall.

Arranging Arrays

Here's a simple example: we have 3 sites, visited 4 times per year for 2 years. This is usually shoehorned into a table with 8 columns for the visits, like this:

    occasion
site 1.1 1.2 1.3 1.4 2.1 2.2 2.3 2.4
   A   0   1   1   3   2   0   1   1
   B   1   0   1   0   3   0   2   2
   C   5   2   1   1   3   4   2   1

These look like counts, but the data could be detection/nondetection (0/1) data, wind speed at each visit, or the name of the observer.

SECR with unpaired cameras

This was based on simulations with just one set of parameter values, but it does suggest that projects with limited resources should consider using unpaired cameras, as this allows more locations to be sampled.

Correlated covariates

Burn... or adapt?

Viewing previous graphs in R for Windows

The wiqid package is now on CRAN

Making a habitat mask for SECR in JAGS

library(githubinstall)
githubinstall("makeJAGSmask")

Installing R and JAGS on Windows

Installing R and JAGS on Apple Mac

People seem to run into problems with different versions of the Mac OS, R, JAGS and the rjags package. The only way to stay sane is to use recent versions of all four.

Check your Mac OS version!

From the Apple menu, choose About This Mac; the version number appears below the name. Note whether you have v. 10. 11 ( El Capitan ) or later. If you have an earlier version, upgrade your OS before going further.

Installing R and JAGS on Ubuntu OS

I already had R and JAGS 3 installed, together with the rjags package version 3. Installing the rjags package within R (or updating it with update.packages() ) installs the new version of rjags , v.4, which requires JAGS 4 and throws an error if it isn't found. But the Ubuntu repository still has JAGS 3, so you cannot update JAGS with Ubuntu Software Center.

Bayesian estimation with a random walk sampler

The idea for the sampler was developed by Nicholas Metropolis and colleagues in a paper in 1953. This was before the Gibbs sampler was proposed, but it uses the same idea of updating the parameters one by one. A better name would be "componentwise random walk Metropolis sampler". The rules for the random walk ensure that a large number of samples will be a good description of the posterior distribution.

Bayesian estimation with a simple Gibbs sampler

Gibbs sampling works if we can describe the posterior for each parameter if we know all the other parameters in the model.

Bayesian estimation with conjugate priors

As our example, we'll use estimation of detection probability from data for repeat visits to a site which is known to be occupied by our target species. First, we'll describe the beta distribution, then see how that can be combined with our data. A discussion of priors will follow, and we'll finish with brief descriptions of conjugate priors for other types of data.

Likelihood and maximum likelihood estimation

We'll use a simple occupancy model. It has just two parameters and both must between 0 and 1. That means that we can plot all possible combinations of the two parameters in a simple two-dimensional graph. As we'll see we need to add a third dimension, but three is still manageable.

Introducing the wiqid package

Currently it has functions for estimating occupancy, abundance from closed captures, density from spatial capture-recaptures, and survival from mark-recapture data, plus a slew of functions for species richness and alpha and beta diversity.

It is intended to be used for (1) simulations and bootstraps, (2) teaching, and (3) introducing Bayesian methods. And it should work on all platforms: Windows, Linux, and Mac.

The jackknife estimator

• mark-recapture estimation of number of animals in a closed population;
• species richness estimation for a defined assemblage.

SECR in BUGS/JAGS with patchy habitat

Faced with patches of suitable habitat surrounded by inhospitable terrain, or a large extent of habitat punctuated with patches of non-habitat, we had the choice of ML methods or one of the packages designed specifically for Bayesian SECR analysis, such as SPACECAP or SCRbayes. But then we are limited to the range of models provided by package authors: we don't have the flexibility to specify our own models that comes with WinBUGS, OpenBUGS or JAGS.

Here I present a way to incorporate patchy habitat into a BUGS/JAGS model specification.

Probability densities and spinners

We start off with simple spinners representing a uniform distribution over a range from, say, 0 to 0.5. We discuss the problems of attaching a probability to an exact value, which leads to probability of a range of values and hence probability density.

Camera-trap layout for SECR

Before the advent of SECR methods, putting all your traps into a single cluster with minimal perimeter length made sense, as you needed to estimate the area trapped animals came from to get a density. SECR estimates density directly, without needing to estimate area, so a single, large cluster may no longer be advantageous.

SECR and circular home ranges

I've seen this asserted a couple of times, in particular in Tobler and Powell (2013, p.110), and I've myself drawn circular home ranges when discussing the interpretation of the capture parameters, but I don't think it is a necessary assumption.

SECR : spatially explicit capture recapture

Capture-recapture methods (also know as mark-recapture or capture-mark-recapture) have been used to estimate the size of animal populations for many years: the first software package for analysis of this kind of data, CAPTURE (Otis et al 1978), is now 35 years old. Early methods did not use the spatial component in the data, the capture locations, and spatially explicit capture-recapture models (SECR, or just spatial capture-recapture, SCR) first appeared in 2004 (Efford 2004).

BEST - Bayesian Estimation Supersedes the t-Test

The idea is to provide an R function which is as easy to use as t.test but which gives not a mere p -value but the kind of output Bayesians are used to - posterior probability distributions. John's BESTmcmc function uses JAGS, but handles all the preliminaries automatically and produces a result in a simple format.

Animal activity patterns and overlap

As soon as cameras with "data backs" came along in the early 90s, biologists realised that they could harvest data on the activity patterns of rare, secretive forest animals. Were they diurnal, nocturnal, crepuscular, or maybe cathemeral (active all around the clock)? More recently, people have tried to get clues about how species interact - competition or prey-predator interactions - from activity patterns, by examining the extent of overlap.

In our corner of the biological world, Martin Ridout and Matt Linkie published a paper (2009) on the activity patterns of tigers, clouded leopards and golden cats in Sumatra, with a lot of technical detail on how overlap could be quantified and confidence intervals estimated. They followed up (2011) with a paper on tigers and their prey, also in Sumatra. ...

Comparing confidence intervals

This is often a silly question: the means of real populations are almost always different, even if the difference is microscopic. More useful would be to estimate the difference and the probability that it is big enough to be of practical importance. See the BEST software for a way to do this in R.

Sometimes we are presented with confidence intervals for each of the means. This happens in particular with the standard packages we use for wildlife data analysis, where the output includes confidence intervals for each coefficient or real value. Can we infer evidence of a difference from confidence intervals in the same way as for a p -value from a test of significance?

What if my data aren't normal?

Sometimes people I talk to are worried because their data aren't normally distributed, and they believe that they can't use the usual techniques such as t-tests or ANOVA without first transforming the data to be normal, or they must resort to non-parametric methods.

There are many good reasons for transforming data or NOT using t tests or F tests, but non-normal data is not usually one of them!

Installing JAGS with AVG anti-virus software

A couple of people on a recent workshop had trouble with their AVG anti-virus software when installing JAGS 3.3.0.

This appears to be due to AVG's paranoia: see Martyn Plummer's comment . No malware is detected by McAfee AntiVirus Plus or Trend Micro Office Scan. See also information on false positives at the AVG forum.

Displaying rasters in QGIS

In ecology and wildlife studies, a lot of our spatial data takes the form of rasters rather than vector files. When you first add a raster in QGIS, you usually get a plain grey rectangle, or maybe just a grey outline on a white background, as most raster file formats have no styling information. To make sense of a raster, you need to change the style.

Here I'll give some hints for "quick-and-dirty" styling to display the contents of a raster. For a more detailed tutorial, see here .

Creating a GIS layer for "Distance from..." in QGIS

In a recent post , I showed how to deal with "distance from..." data in GIS layers using the R packages for handling spatial information. The example I used there involved

1. producing a layer with distance-from-nearest-road as a habitat layer so that we can calculate a probability of occupancy layer, and

2. extracting distance-from-nearest-road for each of our cameras in order to model our data.

Here we will see how to do the same thing in QGIS.

Importing data into R for home range analysis

At our recent workshop on Geographical Information Systems (GIS) using Quantum GIS we had a number of people interested in working with radio telemetry or GPS data to model animal home ranges. The home range plugin for QGIS doesn't work with current versions, at least with Windows.

It is designed to pass data to R and get the adehabitat package to do the home range estimation and pass the result back to QGIS. QGIS uses Python code, and to get it to talk to R requires a bit of software called "RPy2". This was always difficult to set up on Windows, but Python has been upgraded and RPy2 no longer works. In any case, the adehabitat package has been replaced by new packages with a wider ranger of options.

So now it's better to prepare spatial data in QGIS, read the files into R, process with adehabitatHR, write the results to new files, and load into QGIS.

Creating a GIS layer for "Distance from..." in R

We recently ran a workshop on Geographical Information Systems (GIS) using Quantum GIS for ecologists and wildlife researchers. For many species, distance from water, a road, forest edge, or a settlement may be an important habitat variable.

For example, we may be using automatic cameras to investigate occupancy of sites by leopards. Probability of occupancy may depend on distance from the nearest road. Given vector layers with roads and camera locations, we want to do two things:

1. produce a layer with distance-from-nearest-road as a habitat layer so that we can calculate a probability of occupancy layer, and

2. extract distance-from-nearest-road for each of our cameras in order to model our data.

Mathematical formulae with MathJax

I sometimes need to put formulae into my web pages, and I've been exploring the use of MathJax .

In the past I've inserted the formula into MS Word with MS Equation 3.0, doing a screen capture, cropping the image to the formula I want, saving as a .GIF file, and then displaying it on the web page as an image. So I get something like this for the Poisson distribution:

That's not ideal. If I want to change anything, I have to start all over again from Word. It's also messy if I want to put something like into the text; for a start it doesn't line up properly. MathJax allows me to type the formula in LaTeX style directly into the HTML code for my web page.

Format for data files

I have a collection of data sets for use during workshops or just to play with when trying out new statistical techniques or computer code.

A big question is what format to use, and I've changed my mind on this several times already!

After looking at this blog post by John Mount I've decided to try using tab-separated files with a .tsv extension.

Learning R can feel overwhelming at first, mainly because the internet offers so much material that finding a sensible starting point becomes a task in itself. Many guides dive straight into advanced details, assume prior experience with other programming languages, or rely on abstract examples that are hard to relate to real work. A gentler approach is to begin with the basics of reading data, inspecting it, and producing simple summaries before attempting anything ambitious. Once those fundamentals feel comfortable, the rest of the language becomes far less intimidating, and searching for help becomes a matter of answering a specific question rather than wandering through endless pages.

Reading data into R is the first practical skill worth practising, and it pays to understand how files are structured before worrying about analysis. Spreadsheets, plain text tables, and exported outputs from other software all arrive in slightly different shapes, and knowing how to inspect the imported result quickly saves a great deal of confusion later. Simple checks on the number of rows, the names of columns, and the type of each variable catch most problems early. Getting this step right means every later analysis rests on data you actually understand rather than data you merely hope is correct.

Bayesian estimation is a recurring theme for anyone working with wildlife data, because it offers a natural way to combine prior knowledge with the evidence contained in a sample. Rather than producing a single best guess, the approach describes a range of plausible values for each parameter, along with how much confidence should be placed in each. This is often more honest about uncertainty than a single point estimate, especially when samples are small or detection is imperfect. Software such as JAGS and its R interface makes the mechanics accessible, though the ideas matter more than the particular tool.

Capture-recapture methods underpin much of the estimation of animal abundance and density, and the logic is easier to grasp than the terminology suggests. Individuals are detected imperfectly, so raw counts understate how many animals are present; models separate the probability of detection from the size of the population itself. Spatially explicit versions go further, using the locations of detections to estimate density across a habitat rather than a single number for an area. Camera traps and other passive detectors generate exactly this kind of data, which is why the two topics so often appear together.

Occupancy modelling addresses a related question: whether a species is present at a site at all, given that a visit without a detection proves very little. Repeated surveys allow the probability of detection to be estimated alongside the probability that the site is occupied, and the two can then be disentangled. This matters greatly for rare or elusive species, where absence is ambiguous and naive interpretation of survey results can be misleading. Habitat covariates can be added to explain variation in both processes, linking the statistical model directly to questions about where animals live and why.

Activity patterns and the timing of detections are a further source of insight from camera data, revealing when species are active across the day and how their schedules overlap. Estimating overlap between species or between seasons requires care, since the number of photographs is not the same as the amount of time spent active. Simple summaries of clock times can be informative, but formal estimation of the underlying distributions gives a clearer picture. These methods connect neatly with broader questions about competition, predation, and how animals share a habitat across the cycle of day and night.

Spatial data handling is central to modern ecological analysis, and mapping tools help turn coordinates and layers into something interpretable. Working with a geographic information system alongside R allows habitat covariates, such as distance to water or vegetation type, to be extracted for each site and fed into a model. Understanding coordinate systems, projections, and the difference between raster and vector formats prevents many silent errors. A little time spent learning how layers align and how attributes are attached repays itself whenever results are mapped or compared across studies.

Statistical models are only as good as the checking that follows them, and this applies equally to Bayesian and likelihood-based approaches. Inspecting chains for convergence, examining posterior distributions, and comparing predictions against observed data are all part of judging whether a fit can be trusted. It is tempting to stop once the software reports success, but the numbers it returns may still describe a model that fits poorly or a parameter that the data cannot inform. Careful diagnosis, and a willingness to revise the model, is what separates a routine output from a defensible conclusion.

Software installation and version management cause more frustration than most analytical problems, particularly when packages depend on one another or on a particular version of the underlying engine. Keeping notes on what was installed and how it was configured saves hours when something later breaks or when an analysis must be reproduced. Working across different operating systems adds its own wrinkles, since paths, permissions, and available tools vary. A modest amount of discipline here, such as recording the versions used for a project, makes collaboration and revisiting old work considerably easier.

Teaching and sharing these methods is part of the same effort as using them, since workshops and written guidance help others avoid the same dead ends. Good explanations favour concrete examples over abstract theory and acknowledge that most learners arrive with a specific problem rather than a desire for completeness. Pointing people towards a few reliable sources is often more useful than listing everything available, and encouraging them to ask precise questions makes searching far more productive. The aim throughout is practical competence, so that the tools become a means of answering ecological questions rather than an end in themselves.