Low-level scattermore API

library(scattermore)
set.seed(2023)

Low-level interface of Scattermore

This vignette gives an overview and comprehensive examples of using the low-level API of scattermore. Using the API you can do many more things than just plotting points; mainly scattermore supports plotting both 2D-histograms (density plots) and image rasters from both points and lines (and possibly other shapes in the future), modifying and expanding both histograms and image rasters with kernels, and blending all of these using ordering-independent blending methods.

The low-level API operates mostly on plain R matrices or arrays; these are formatted in:

One typically starts with either generating the densities or RGBWT rasters, and successively moves lower in the list towards the usual integer RGBA, which may be e.g. converted to a normal R raster.

RGBWT format is specific to scattermore, standing for Red Green Blue Weight Transparency. It is similar to a floating-point RGBA with premultiplied alpha: channels R, G, B behave just like in RGBA, channel T is equivalent to (1-A), and channel W (initially equivalent to A) collect the total amount of “paint” that the given pixel has accumulated. With this, blending of layers is guaranteed to be order independent: values in channels RGBW are added together, and values in T are multiplied. To convert RGBWT back to RGBA, one computes A=(1-T), and divides the channels R,G,B by the value of W.

Plotting data as densities

Let us first generate a small random dataset of points and directions:

n <- 10000
pts <- matrix(rnorm(n*2), n, 2)
pts2 <- cbind(5+rnorm(n), -5*rexp(n))

We can convert the density data to an integer density map:

pdens <- scattermore::scatter_points_histogram(pts, out_size=c(128,128))

par(mar = c(0,0,0,0), bg='white')
image(pdens)

The same can be done for plotting the densities over the lines:

ldens <- scattermore::scatter_lines_histogram(cbind(pts, pts2), out_size=c(256,256))

par(mar = c(0,0,0,0), bg='white')
image(ldens)

Densities are simple matrices, so in order to improve the visualization you can apply the usual R math functions to their contents:

ldens <- scattermore::scatter_lines_histogram(cbind(pts, pts2), out_size=c(256,256))

par(mar = c(0,0,0,0), bg='white')
image(log1p(ldens))

Plotting data as colored pixels

Colored stuff is usually plotted to a 5-channel RGBWT raster format, which eliminates the typical overplotting artifacts. This is later converted to either RGBA or a standard R rasters for plotting.

We can plot the points with a single color to the RGBWT format using scatter_points_rgbwt. We also show how to fix the precise displayed area using xlim and ylim.

# TODO xlim docs
rgbwt <- scatter_points_rgbwt(pts, out_size=c(512,512), xlim=c(-3,3), ylim=c(-3,3))
rstr <- rgba_int_to_raster(rgbwt_to_rgba_int(rgbwt))

par(mar=c(0,0,0,0), bg='white')
plot(rstr)

You can specify a single color (possibly with alpha channel) for plotting; here we force a lower resolution and turn off pixel interpolation to show the transparency effect more precisely:

rgbwt <- scatter_points_rgbwt(pts, out_size=c(128,128), RGBA=col2rgb('#8010f010', alpha=T))
rstr <- rgba_int_to_raster(rgbwt_to_rgba_int(rgbwt))

par(mar=c(0,0,0,0), bg='white')
plot(rstr, interpolate=F)

The RGBA parameter may also assign individual colors to each pixel:

rgbwt <- scatter_points_rgbwt(pts, out_size=c(128,128), RGBA=col2rgb(rainbow(n, alpha=0.3), alpha=T))
rstr <- rgba_int_to_raster(rgbwt_to_rgba_int(rgbwt))

par(mar=c(0,0,0,0), bg='white')
plot(rstr, interpolate=F)

For plotting large clusters that share the colors, arguments map and palette allow for simpler and more efficient specification of the plotting:

clusters <- 1 + (pts[,1] < 0) + 2 * (pts[,2] < 0)
rgbwt <- scatter_points_rgbwt(pts, out_size=c(128,128), palette=col2rgb(rainbow(4, alpha=0.2), alpha=T), map=clusters)
rstr <- rgba_int_to_raster(rgbwt_to_rgba_int(rgbwt))

par(mar=c(0,0,0,0), bg='white')
plot(rstr, interpolate=F)

Colored RGBWT lines are plotted similarly:

#TODO as.vector
rgbwt <- scatter_lines_rgbwt(cbind(pts, pts2), out_size=c(512,512),
    xlim=c(-3,7), ylim=c(-5,2),
    RGBA=as.vector(col2rgb('#8010f005', alpha=T)))
rstr <- rgba_int_to_raster(rgbwt_to_rgba_int(rgbwt))

par(mar=c(0,0,0,0), bg='white')
plot(rstr)

Expanding densities and RGBWT plots

Scattermore takes a distinctive approach to plotting points that are larger than a single pixel – these are not plotted immediately as large, but instead start as single-pixel “centers” and are later expanded using a kernel function. The expansion is faster for plots that contain huge values of individual points, because the “expansion” operations is aggregated for multiple pixels, and the operation is much more computationally regular, giving additional speedups.

First, let’s increase the resolution of the example from above:

rgbwt <- scatter_points_rgbwt(pts, out_size=c(512,512), palette=col2rgb(rainbow(4, alpha=0.5), alpha=T), map=clusters)
rstr <- rgba_int_to_raster(rgbwt_to_rgba_int(rgbwt))

par(mar=c(0,0,0,0), bg='white')
plot(rstr)

This is sub-optimal because the pixels are too small; to fix that we can expand them:

rgbwt <- scatter_points_rgbwt(pts, out_size=c(512,512), palette=col2rgb(rainbow(4, alpha=0.1), alpha=T), map=clusters)
rgbwt <- apply_kernel_rgbwt(rgbwt, 'circle', radius=10)
rstr <- rgba_int_to_raster(rgbwt_to_rgba_int(rgbwt))

par(mar=c(0,0,0,0), bg='white')
plot(rstr)

There are also other expansion options (such as gauss and square). You can also supply your own kernel in order to plot different shapes of pixels:

rgbwt <- scatter_points_rgbwt(pts, out_size=c(512,512), palette=col2rgb(rainbow(4, alpha=1), alpha=T), map=clusters)
rgbwt <- apply_kernel_rgbwt(rgbwt, 'own',
    mask=outer(1:11, 1:11, Vectorize(function(x,y) 1/(x*y))))
rstr <- rgba_int_to_raster(rgbwt_to_rgba_int(rgbwt))

par(mar=c(0,0,0,0), bg='white')
plot(rstr)

The same can be applied to line data to get thicker lines (we limit the number of lines for simplicity here):

rgbwt <- scatter_lines_rgbwt(cbind(pts, pts2)[1:30,], out_size=c(512,512),
    xlim=c(-3,7), ylim=c(-5,2),
    RGBA=as.vector(col2rgb('#8010f010', alpha=T)))
rgbwt <- apply_kernel_rgbwt(rgbwt, 'circle', radius=5)
rstr <- rgba_int_to_raster(rgbwt_to_rgba_int(rgbwt))

par(mar=c(0,0,0,0), bg='white')
plot(rstr)

Gaussian kernels are excellent for smoothing out point and line densities. As the main change, we need to use the histogram-specific kernel function:

pdens <- scattermore::scatter_points_histogram(pts, out_size=c(256,256), xlim=c(-3,3), ylim=c(-3,3))
pdens <- apply_kernel_histogram(pdens, 'gauss', radius=10)

par(mar = c(0,0,0,0), bg='white')
image(pdens, col=rainbow(100))

The calculated densities can be trivially used for overlaying the images with contours. (N. B. that the default gaussian kernels are not “balanced” and increase the total weight present in the graphics; you may still supply a unit weight kernel using the own parameter.)

par(mar = c(0,0,0,0), bg='white')
image(pdens, col=topo.colors(100)[20:100])
contour(pdens-15,