Numeric Tensor Framework: Toward a New Paradigm in Technical Computing

Abstract

Technical computing is a cornerstone of modern scientific practice. Within technical computing, the matrix-vector (MV) framework, composed of MV algebra and MV software, dominates the discipline in representing and manipulating linear mappings applied to vectors. Indeed, prominent technical computing packages, e.g., MATLAB, revolve around the MV framework. Applying Thomas S. Kuhn's theory of paradigms, the MV framework is technical computing's paradigm. One may then reasonably ask whether the MV paradigm imposes significant restrictions on technical computing's practice. This question may be answered by synthesising the literature on widespread and disparate research efforts on frameworks beyond the MV paradigm. Two categories of anomalous practice emerge, namely special linear mappings, i.e., high-dimensional and entrywise linear mappings, and mappings beyond linear, i.e., polynomial and multilinear mappings. To tackle these anomalies, a framework for numeric tensors (NTs), i.e., high-dimensional data invested with arithmetic operations, proves well-equipped. The proposed NT framework uses an NT algebra that exploits and extends the storied Einstein notation, offering unmatched capabilities, e.g., N-dimensional operators, associativity, commutativity, entrywise products, and linear invertibility, complemented by distinct ease-of-use. This expressiveness is comprehensively supported by innovative NT software, embodied by open-source C++ and MATLAB libraries. Novelties include a lattice data structure, which can execute or invert any NT product, of any dimensions, using optimised algorithms. Regarding sparse NT computations, which are essential to address the curse of dimensionality, the software takes new approaches for data storage, rearrangement, and multiplication. Moreover, the software performs competitively on representative benchmarks, matching or surpassing leading competitors, including the MATLAB Tensor Toolbox, NumPy, FTensor, and Blitz++, while providing a more general set of arithmetic operations. To illustrate these contributions, two original problems from computer vision are solved using the NT framework. The selected exemplars, concerning image segmentation and depth-map estimation, involve high-dimensional differential operators, linking them to the partial-differential equations found in countless other disciplines. Returning to Kuhn, the contributions of this thesis, literature review included, help make a case that technical computing is experiencing a revisionary period. As such, the NT framework, with its expressive algebra and innovative software, represents a timely and significant contribution to the evolution of technical computing's paradigm.