===================================================================
vspline - generic C++ code to create and evaluate uniform B-splines
===================================================================

---------------------
vspline in a nutshell
---------------------

Impatient? Here's a quick example of code to set up a 1D cubic b-spline over three knot points, and to evaluate it at several coordinates:

::

  #include <iostream>
  #include "vspline/vspline.h"

  int main ( int argc , char * argv[] )
  {
    auto spline = vspline::bspline < double , 1 > ( 3 ) ;
    auto coefficients = spline.core ;

    coefficients [ 0 ] = 0.0 ;
    coefficients [ 1 ] = 1.0 ;
    coefficients [ 2 ] = 0.0 ;

    spline.prefilter() ;

    auto evaluator = vspline::make_safe_evaluator ( spline ) ;

    for ( double x = -1.0 ; x < 3.3 ; x += .3 )
      std::cout << x << " -> " << evaluator ( x ) << std::endl ;
  }


This only scratches the surface, but it demonstrates a few points:

- a vspline::bspline object holds the spline's coefficients
- prefiltering the spline has to be done explicitly
- to evaluate the spline, a specific evaluation functor is created

Beyond the single-value operation in this simple example, vspline offers sophisticated processing of data in multidimensional arrays of arbitrary dimensionality, shape and strides. Splines can hold fundamentals or 'xel' data, like pixels, and evaluation code has SIMD overloads, which optionally use an 'explicit SIMD' backend like Vc, highway or sd::simd. Spline degree can be up to 45. The code to process nD arrays of 'xel' data with multithreaded SIMD code is not limited to b-splines - it's perfectly general and you can feed it your own functions. There's also support for functional composition of such functions.

------------
Introduction
------------

vspline provides a free, comprehensive and fast library for uniform (aka cardinal) B-splines and their use with n-dimensional raster data, using multithreading and SIMD processing. It's header-only C++ template metacode, there are no shared or static libraries, only a few dependencies on other libraries, namely vigra and pthreads, and optionally Vc or highway.

Uniform B-splines are a method to provide a 'smooth' interpolation over a set of uniformly sampled data points. They are commonly used in signal processing as they have several 'nice' qualities - an in-depth treatment and comparison to other interpolation methods can be found in the paper 'Interpolation Revisited' [CIT2000]_ by Philippe Thévenaz, Member, IEEE, Thierry Blu, Member, IEEE, and Michael Unser, Fellow, IEEE.

While there are several freely available packets of B-spline code, I failed to find one which is comprehensive, efficient and generic at once. vspline attempts to be all that, making use of generic programming in C++11, and of common, but often underused hardware features in modern processors (namely, multiple CPU cores and SIMD). Overall, there is an emphasis on speed, even if this makes the code more complex. I tried to eke as much performance out of the hardware at my disposal as possible, only compromising when the other design goals would have been compromised.

Some of the code is quite low-level, but there are high-level mechanisms to interface with vspline, allowing easy access to it's functionality without requiring users to familiarize themselves with the internal workings. High-level approach is provided via class 'bspline' defined in bspline.h, 'evaluator' objects from eval.h and via the remap/transform functions defined in transform.h. This high-level code allows using fast, high-quality interpolation with just a few lines of code, automatically using all the multithreading and SIMD capabilities to get the task done. It should be especially attractive to vigra users, since the data handling and parametrization is done with vigra data types (vigra::MultiArrayView and vigra::TinyVector).

While I made an attempt to write code which is portable, vspline is only tested thoroughly with g++ and clang++. An installation of Vigra_ is needed to compile, installation of the SIMD libraries Vc_, highway_ or using std::simd is optional but recommended. Most Linux distros now offer both vigra and Vc versions suitable for vspline, alternatively you can build either from source. highway is, as of this writing, less common, but some distros have a package already - look for 'hwy'. A vspline debian package_ is now available.

vspline is designed to be platform-independent; it should run on a wide variety of hardware and OS. when using Vc, highway or no specific SIMD backend, it requires the C++11 standard, use of std::simd requires C++17. Vc's scope is mainly intel/AMD processors and it's now in 'maintenance mode', whereas highway is actively developed and supports processor architectures beyond intel/AMD, notably ARM which makes it a good backend when targeting e.g. newer macs with 'apple silicon'. std::simd is a good idea but lacks essential features (like gather/scatter), and the current state of the implementation shipping with gcc is missing some vectorized math functions (e.g. atan2), so it's often outperformed by Vc and highway.

vspline has been in use for a few years now, and the code seems robust enough to warrant a 1.X version number. I think it's fit for production use. The driving force behind it's continuing development is now lux_, where I integrate and test new features, to take them into the vspline code base once they've proven useful and stable.

I have made efforts to cover 'reasonable' use cases, but there may be corner cases and unexpected scenarios where my code fails. The code is not well shielded against inappropriate parameters. The intended audience is developers rather than end users.

vspline is useful for sound and image processing, volume slicing, and other tasks needing efficient, precise interpolation. It comes with example code, and if you want to see vspline used in a 'real world' program, have a look at lux_, my image and panorama viewer, which uses vspline for rendering.

vspline goes beyond the 'standard' features one might expect from a b-spline library, namely coefficient generation and evaluation at some coordinate: it also provides code to process n-dimensional arrays of data with multithreaded code and to use hardware vectorization with SIMD code. This makes the code base large, and for maximum benefit you'll have to get familiar with these 'additional' features. I hope you'll find this is worth your while.

I have started to factor out this latter aspect - the processing of raster data with multithreaded SIMD code - to a new library, zimt_.

-----
Scope
-----

There are (at least) two different approaches to tackle B-splines as a mathematical problem. The first one is to look at them as a linear algebra problem. Calculating the B-spline coefficients is done by solving a set of equations, which can be codified as banded diagonal matrices with slight disturbances at the top and bottom, resulting from boundary conditions. The mathematics are reasonably straightforward and can be efficiently coded (at least for lower-degree splines), but I found it hard to generalize to B-splines of higher order.

The second approach to B-splines comes from signal processing, and it's the one which I found most commonly used in the other implementations I studied. It generates the B-spline coefficients by applying a forward-backward recursive digital filter to the (periodized) data and implements boundary conditions by picking appropriate initial causal and anticausal coefficients. Once I had understood the process, I found it elegant and beautiful - and perfectly general, lending itself to the implementation of a body of generic code with the scope I envisioned.

Evaluation of a b-spline requires picking a small subset - or window - of the coefficients and forming a weighted sum of this subset, where the weights depend on the precise interpolation locus and a 'basis function' - for b-splines, one of the b-spline basis functions. So there are two components involved: getting the coefficients from memory and the formation of the weighted sum. vspline is designed to do these tasks as quickly as possible, and a large part of it's code is dedicated to the evaluation process.

As it turns out, merely having a fast evaluation function is not enough to efficiently process large sets of raster data, and a good part of vspline deals with moving data around, parceling them so that a set of independent threads can process them concurrently with SIMD operations, and putting the results back to memory. The actual evaluation is done by a functor, which I think of as a tool. I think of applying this tool to data as 'wielding' it, so I call the code to 'roll out' the functors 'wielding code'. This code goes beyond what one might usually expect in a b-spline package and might even be put into a separate library. For now it's part of vspline, and vspline's remap, transform and apply functions use it extensively - and provide an iterface to the code which does not expose the users to it, so you can pass arrays and functors to these functions without having to worry about the inner workings.

Note that there is now an effort in progress to factor out the 'wielding' code into a separate library. This is work in progress, the code is already usable and available on github. It's called zimt_. zimt tries to do away with a few traits of vspline's wielding code which I am not happy with, and eventually I intend to delegate activity in zimt's scope to zimt, and to remove these parts from vspline. zimt does not rely on vigra and provides it's own types for nD arrays and small aggregates, but there is some glue code in zimt, and as the 'zimt' branch of lux demonstrates, the delegation to zimt is already working.

vspline can handle

-  splines over real and integer data types and their aggregates:
-  all 'xel' data, arbitrary number of channels (template argument)
-  single, double precision and long doubles supported (template argument)
-  splines using integral coefficients
-  a reasonable selection of boundary conditions
-  arbitrary dimensionality of the spline (template argument)
-  spline degree up to 45 (runtime argument)
-  there is specialized code for 1D data
-  'transform' family functions use multithreaded code (pthread)
-  optional use of the CPU's vector units (like AVX/2, AVX512f, ARM NEON)
-  with choice of SIMD back-end (Vc, highway, std::simd, 'goading')

On the evaluation side it provides

-  evaluation of the spline at point locations in the defined range
-  evaluation of vectorized arguments
-  evaluation of the spline's derivatives
-  factory functions to create evaluation functors
-  specialized code for degrees 0 and 1 (nearest neighbour and n-linear)
-  mapping of arbitrary coordinates into the defined range
-  evaluation of nD arrays of coordinates ('remap' function)
-  discrete-coordinate-fed remap function ('index_remap')
-  generalized functor-based 'apply' and 'transform' functions
-  restoration of the original data from the spline coefficients

To produce maximum performance, vspline has a fair amount of collateral code,
and some of this code may be helpful beyond vspline:

-  multithreading with a thread pool
-  efficient processing of nD arrays with multiple threads
-  functional constructs using vspline::unary_functor
-  nD forward-backward n-pole recursive filtering
-  nD separable convolution
-  efficient access to b-spline basis functions and their derivatives
-  precise precalculated constants (made with GNU GSL, BLAS and GNU GMP)
-  use of alternative basis functions
-  common interface to several SIMD back-ends
-  vigra type traits for the SIMD data types
-  many examples, ample comments

The code at the very core of my B-spline prefiltering code evolved from the code by Philippe Thévenaz which he published here_, with some of the boundary condition treatment code derived from formulae which he communicated to me. Next I needed code to handle multidimensional arrays in a generic fashion in C++. I chose to use Vigra_. Since my work has a focus on signal (and, more specifically image) processing, it's an excellent choice, as it provides a large body of code with precisely this focus and has well-thought-out, reliable support for multidimensional arrays and small zero-overhead aggregates. vigra's MultiArray and MultiArrayView types are similar to Numpy_ arrays. vspline uses vigra mainly for it's data handling code - vigra has b-spline code as well, but that's not used in vspline. Sadly, vigra is not very common and it's evolution seems to have slowed down, so it can be cumbersome as a dependency. My zimt_ library makes an attempt to do without vigra, but so far this only covers a fraction of the code using vigra in vspline.

Once I had a prototype up and running, I looked out for further improvements to it's speed. While using GPUs is tempting, I chose not to go down this path. Instead I chose to stick with CPU code and use vectorization. Again, I did some research and found Vc_. Vc allowed me to write generic code for vectorization for a reasonably large sets of targets. I found the performance gain for some data types so enticing that I chose to make my code optionally use Vc. The use of Vc has to be activated by a compile-time flag (USE_VC). When Vc (or other SIMD backends) can't be used - or for data types Vc can't vectorize - vspline uses SIMD emulation: data are aggregated to small vector-friendly parcels and the expectation is that processing of such aggregates will trigger the compiler's autovectorization. I call this technique 'goading' and found it to work surprisingly well. This technique works especially well for vspline's digital filters, while the b-spline evaluation code with it's more complex arithmetics and frequent data-driven memory access does benefit less. Vc is now in 'maintenance mode', and there is no support for newer ISAs. I am fond of it's SimdArray data type, which is well-designed and easy to use, so I modelled alternative SIMD back-ends to provide similar APIs. The most evolved of my alternative back-ends is the one using highway, and in the lux project, it's now the default. highway is actively developed and covers a wide range of ISAs. Vc still has slight advantages for some ISAs (like plain AVX), but for newer ISAs like AVX512 highway is better, and for non-intel/AMD platforms, Vc is not a good choice because it does not produce native binary for them. std::simd is very similar to Vc, but it's lacking some crucial components. So, all in all, I'd say highway is the way to go. Choosing SIMD backends is as simple as #defining USE_VC, USE_HWY or USE_STDSIMD globally.

With the 'core' b-spline functionality established, I wrote the 'wielding' code and extended the code base to handle 'corner cases' like 1D data, which need some trickery to be efficiently processed with multithreading and SIMD instructions, or integer-based splines, which need auxilliary code to exploit the integer dynamic range. Using vspline with other projects, I established a small set of useful helper types, methods for functor composition and factory functions to facilitate using vspline without having to be too concerned about template arguments. This is easy due to 'auto' variables in C++11, allowing you to call a factory function with a set of parameters and receive an appropriately typed result into an 'auto' variable without needing to declare it's type explicitly. Oftentimes the types returned by vspline's factory functions are 'quite a mouthful', so this is a great simplification.

I did all my programming on a Linux system, running on an intel(R) Core (TM) i5-4570 CPU, and used GNU gcc_ and clang_ to compile the code. Due to the growing distribution of lux, vspline has now come to run on other hardware like intel processors using AVX512f, and various ARM processors (including 'apple silicon'). I use it on Microsoft Windows with msys2/mingw and on a mac with apple's version of clang++. It's also in use on freeBSD by the freeBSD port
of lux.

.. _here: http://bigwww.epfl.ch/thevenaz/interpolation/
.. _Vigra: http://ukoethe.github.io/vigra/
.. _Vc: https://github.com/VcDevel/Vc
.. _highway: https://github.com/google/highway
.. _gcc: https://gcc.gnu.org/
.. _clang: http://http://clang.llvm.org/
.. _package: https://tracker.debian.org/pkg/vspline

.. [CIT2000] Interpolation Revisited by Philippe Thévenaz, Member,IEEE, Thierry Blu, Member, IEEE, and Michael Unser, Fellow, IEEE in IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 19, NO. 7, JULY 2000, which is available online_.

.. _online: http://bigwww.epfl.ch/publications/thevenaz0002.pdf

.. [1] I use 'aggregate' here to mean a collection of identical elements, in contrast to what C++ defines as an aggregate type. So aggregates would be pixels, vigra TinyVectors, and, also, complex types.

-------------
Documentation
-------------

vspline uses doxygen to create documentation, but admittedly the doxygen documentation is not very weill maintained. You can access this documentation online here:

documentation_

I have made an effort to comment my code very extensively. There are probably more comments in vspline than actual code, and this is appropriate, since the code is in large parts complex template metacode, in an attempt to make it as generic as possible. The doxygen documentation can at times be quite confusing, and if you're trying to find your way into vspline's code base, you may be better off simply reading through the code with the comments explaining what's going on. Please note that I use a bottom-up coding style, avoiding forward declarations, so the higher-level functions are further down in the code. My formatting is 'old school' with white-space-separated tokens and lower-case identifiers structured with underscores, which I find best for readability.

vspline comes with a fair amount of examples, which at times are a bit rough-and-ready, and less well-groomed than the library code. Nevertheless the examples should help in understanding how vspline can be put to use. I've also published my bootstrapping code, producing precalculated b-spline prefilter pole values and basis function values in high-precision arithmetic using GSL, BLAS and GNU GMP and presenting them with 64 significant decimals. Some of the examples have self-testing code to establish that an installation of vspline runs correctly. The precalculated values and specialized basis function code allow vspline to provide very fast access to the b-spline basis functions and their derivatives, and creation of evaluation functors is near-instantaneous even for high degree splines. For high-degree splines, this is a significant speedup vs. using the Cox-deBoor recursion.

-----
Speed
-----

While performance will vary widely from system to system and between different compiles, I'll quote some measurements from my own system. I include benchmarking code (roundtrip.cc in the examples folder). Here are some measurements done with "roundtrip", working on a full HD (1920*1080) RGB image, using single precision floats internally - the figures are averages of several runs, using a binary compiled with clang++ v.10:

::

    testing bc code MIRROR    spline degree 3 using SIMD emulation
    avg 32 x prefilter:............................ 11.0 ms
    avg 32 x transform with ready-made bspline:.... 27.2 ms
    avg 32 x classic remap:........................ 39.9 ms
    avg 32 x restore original data: ............... 10.8 ms

    testing bc code MIRROR    spline degree 3 using Vc
    avg 32 x prefilter:............................ 9.2 ms
    avg 32 x transform with ready-made bspline:.... 19.0 ms
    avg 32 x classic remap:........................ 32.2 ms
    avg 32 x restore original data: ............... 10.7 ms
    difference original data/restored data:

As can be seen from these test results, using Vc on my system speeds evaluation up a good deal. When it comes to prefiltering, a lot of time is spent buffering data to make them available for fast vector processing. The time spent on actual calculations is much less. Therefore prefiltering for higher-degree splines is not much different:

::

    testing bc code MIRROR spline degree 5 using Vc
    avg 32 x prefilter:............................ 10.3 ms

    testing bc code MIRROR spline degree 7 using Vc
    avg 32 x prefilter:............................ 10.7 ms

Results with highway and std::simd are not too different, you can try out each of the variants and see how it goes. Using double precision arithmetics, vectorization doesn't help so much, and prefiltering is actually slower on my system when using Vc. Doing a complete roundtrip run on your system should give you an idea about which mode of operation best suits your needs. These speed measurements are a bit of a moving target, since the code base is still under development - some modifications make some things faster and other things slower - and compilers improve, producing faster binary code with improving optimization. Also, the SIMD libraries evolve. But the figures above should give you an idea what to expect.

Since vspline can make heavy demands on memory bandwidth and CPU usage, it also lends itself to benchmarking a given system, running it full throttle on all cores with SIMD instructions.

vspline is written to auto-vectorize well, so even if Vc is not used, if the relevant compiler switches are set (-march=native, -mavx2, -O3 or -Ofast etc) the code will use SIMD instructions if the compiler autovectorizes it, so vspline can also serve to check how well different compilers autovectorize. The compilers' autovectorization capabilites differ, but seem to gradually improve, so the need for a dedicated SIMD backend lessens. When using highway, you may need additional ISA-specific compiler flags to get maximal performance, please refer to their documentation, or have a look at how lux does it - the CMakeLists.txt in lux gives the complete set of compiler flag which highway needs for optimal performance on different ISAs. Some mathematical functions like atan2 do not autovectorize well - probably due to the need for conditionals, which, in SIMD code, need to be handled with masking instead, and some SIMD back-ends do not actually provide 'proper' SIMD code but simply use a loop internally (e.g., at the time of this writing, atan2 in gcc's std::simd). This can slow down programs relying on these functions significantly.

----------
History
----------

Some years ago, I needed uniform B-splines for a project in python. I looked for code in C which I could easily wrap with cffi_, as I intended to use it with pypy_, and found K. P. Esler's libeinspline_. I proceeded to code the wrapper, which also contained a layer to process Numpy_ nD-arrays, but at the time I did not touch the C code in libeinspline. The result of my efforts is still available from the repository_ I set up at the time. I did not use the code much and occupied myself with other projects, until my interest in B-splines was rekindled sometime in late 2015. I had a few ideas I wanted to try out which would require working with the B-spline C code itself. I started out modifying code in libeinspline, but after some initial progress I felt constricted by the code base in C, the linear algebra approach, the limitation to cubic splines, etc. Being a C++ programmer with a fondness for generic programming, I first re-coded the core of libeinspline in C++ (to continue using my python interface), but then, finally, I decided to start afresh in C++, abandon the link to libeinspline and not have a python interface. vspline is the result of my work. I have chosen the name vspline, because I relied on two libraries staring with 'V', namely Vigra_ and Vc_. You can also think of it as a shorthand for 'vectorized splines'.

The code has been under development for severeal years now. Once the initial design turned out to be workable I went on to go through all the components cyclically, trying to make everything rock solid and precise.

vspline is in constant use via lux_, my image and panorama viewer, where it is used for interpolation and for efficient handling of rendering pipelines. Interaction with this program drove vspline development and helped in keeping it well-groomed, and by now I tend to introduce new vspline code in lux, where it comes into constant use and can show it's usefulness. After some time I merge these novelties back to the vspline code base.

While my initial way to use b-splines in python used 'classical' python tools like CFFI, I have recently coded a python interface to vspline which provides most of it's functionality to python. This interface is based on cppyy_, a novel way to integrate python and C++ without the need for interface definitions. My new b-spline python module is there_. It's still quite rough-and-ready, but it sports a pythonization layer and can handle NumPy ndarrays.

I found that I could use this python module in julia to good effect, but I haven't gone beyond the dabbling stage with that - have a look at this thread_ on julia's discussion forum.

Maintaining and evolving the code for vspline and lux is a large task, and offering a debian package for vspline (which is updated only infrequently) and distributing some lux binaries takes up a lot of time already. I'd welcome some help, especially with packaging and distribution.

.. _cffi: https://cffi.readthedocs.org/en/latest/
.. _pypy: http://pypy.org/
.. _libeinspline: http://einspline.sourceforge.net/
.. _Numpy: http://www.numpy.org/
.. _repository: https://bitbucket.org/kfj/python-bspline
.. _documentation: https://kfj.bitbucket.io
.. _lux: https://bitbucket.org/kfj/pv
.. _cppyy: https://bitbucket.org/wlav/cppyy
.. _there: https://bitbucket.org/kfj/python-vspline
.. _thread: https://discourse.julialang.org/t/using-cppyy-for-julia-c-interoperability/63979
.. _zimt: https://github.com/kfjahnke/zimt



