Packages

•  package root

  Definition Classes

  root

•  package lamp

  Lamp provides utilities to build state of the art machine learning applications

  Lamp provides utilities to build state of the art machine learning applications

  Overview

  Notable types and packages:

  • lamp.STen is a memory managed wrapper around aten.ATen, an off the heap, native n-dimensionl array backed by libtorch.
  • lamp.autograd implements reverse mode automatic differentiation.
  • lamp.nn contains neural network building blocks, see e.g. lamp.nn.Linear.
  • lamp.data.IOLoops implements a training loop and other data related abstractions.
  • lamp.knn implements k-nearest neighbor search on the CPU and GPU
  • lamp.umap.Umap implements the UMAP dimension reduction algorithm
  • lamp.onnx implements serialization of computation graphs into ONNX format
  • lamp.io contains CSV and NPY readers

  How to get data into lamp

  Use one of the file readers in lamp.io or one of the factories in lamp.STen$.

  How to define a custom neural network layer

  See the documentation on lamp.nn.GenericModule

  How to compose neural network layers

  See the documentation on lamp.nn

  How to train models

  See the training loops in lamp.data.IOLoops

  Definition Classes

  root

•  package autograd

  Implements reverse mode automatic differentiaton

  Implements reverse mode automatic differentiaton

  The main types in this package are lamp.autograd.Variable and lamp.autograd.Op. The computational graph built by this package consists of vertices representing values (as lamp.autograd.Variable) and vertices representing operations (as lamp.autograd.Op).

  Variables contain the value of a Rn => Rm function. Variables may also contain the partial derivative of their argument with respect to a single scalar. A Variable whose value is a scalar (m=1) can trigger the computation of partial derivatives of all the intermediate upstream Variables. Computing partial derivatives with respect to non-scalar variables is not supported.

  A constant Variable may be created with the const or param factory method in this package. const may be used for constants which do not need their partial derivatives to be computed. param on the other hand create Variables which will fill in their partial derivatives. Further variables may be created by the methods in this class, eventually expressing more complex Rn => Rm functions.

  Example

  lamp.Scope.root{ implicit scope =>
    // x is constant (depends on no other variables) and won't compute a partial derivative
    val x = lamp.autograd.const(STen.eye(3, STenOptions.d))
    // y is constant but will compute a partial derivative
    val y = lamp.autograd.param(STen.ones(List(3,3), STenOptions.d))
  
    // z is a Variable with x and y dependencies
    val z = x+y
  
    // w is a Variable with z as a direct and x, y as transient dependencies
    val w = z.sum
    // w is a scalar (number of elements is 1), thus we can call backprop() on it.
    // calling backprop will fill out the partial derivatives of the upstream variables
    w.backprop()
  
    // partialDerivative is empty since we created `x` with `const`
    assert(x.partialDerivative.isEmpty)
  
    // `y`'s partial derivatie is defined and is computed
    // it holds `y`'s partial derivative with respect to `w`, the scalar which we called backprop() on
    assert(y.partialDerivative.isDefined)
  
  }

  This package may be used to compute the derivative of any function, provided the function can be composed out of the provided methods. A particular use case is gradient based optimization.

  Definition Classes

  lamp

  See also

  https://arxiv.org/pdf/1811.05031.pdf for a review of the algorithm

  lamp.autograd.Op for how to implement a new operation

• Add
• ArcTan
• ArgMax
• Assign
• Autograd
• AvgPool2D
• BatchNorm
• BatchNorm2D
• BatchedMatMul
• BinaryCrossEntropyWithLogitsLoss
• CappedShiftedNegativeExponential
• CastToPrecision
• Cholesky
• CholeskySolve
• Concatenate
• ConstAdd
• ConstMult
• Constant
• ConstantWithGrad
• ConstantWithoutGrad
• Convolution
• Cos
• Cross
• Debug
• Diag
• Div
• Dropout
• ElementWiseMaximum
• ElementWiseMinimum
• Embedding
• EqWhere
• EuclideanDistance
• Exp
• Expand
• ExpandAs
• Flatten
• Gelu
• GraphMemoryAllocationReport
• HardSwish
• IndexAdd
• IndexAddToTarget
• IndexFill
• IndexSelect
• Inv
• LayerNormOp
• LeakyRelu
• Log
• Log1p
• LogDet
• LogSoftMax
• MaskFill
• MaskSelect
• MatMul
• MaxPool1D
• MaxPool2D
• Mean
• Minus
• MseLoss
• Mult
• NllLoss
• NoReduction
• Norm2
• OneHot
• Op
• PInv
• Pow
• PowConst
• Reduction
• Relu
• RepeatInterleave
• Reshape
• ScaledDotProductAttention
• ScatterAdd
• Select
• Sigmoid
• Sin
• Slice
• SmoothL1Loss
• Softplus
• SparseFromValueAndIndex
• SquaredFrobeniusMatrixNorm
• Stack
• Sum
• Tan
• Tanh
• ToDense
• Transpose
• Variable
• VariableNonConstant
• Variance
• View
• WeightNorm
• Where

lamp.autograd

ConstantWithGrad