forked from kscience/kmath
Compare commits
86 Commits
dev
...
feature/no
Author | SHA1 | Date | |
---|---|---|---|
|
0817efbc4b | ||
|
a35bfad44a | ||
|
3c92bfda59 | ||
|
b288c4ce59 | ||
|
c53bdd38f8 | ||
|
c2c8d80b40 | ||
|
f3a411f0e2 | ||
|
08f7af6d41 | ||
|
4beabb3a5d | ||
|
9ca896b608 | ||
|
705b27aa4b | ||
|
1e7ee53c82 | ||
|
4216a43470 | ||
|
a9da25875b | ||
|
40b9066dd0 | ||
|
3b6e80f5b6 | ||
|
84570549e2 | ||
|
79de6700e7 | ||
|
3660c7f217 | ||
|
927916a01f | ||
|
633eb7ad4e | ||
|
371674c9d3 | ||
|
bc43afe93b | ||
|
3d1a3e3b69 | ||
|
c6acedf9e0 | ||
|
36bc127260 | ||
|
686baa6517 | ||
|
3b0032cb2f | ||
|
cc4da94646 | ||
|
046d26b17a | ||
|
b948ca57cd | ||
|
cccff29378 | ||
|
0bc2c12a05 | ||
|
28fac22f12 | ||
|
ea6cd01b89 | ||
|
face60824d | ||
|
d303c912d6 | ||
|
a33af9ec94 | ||
|
93768ed2a7 | ||
|
c09da54cc9 | ||
|
183d34e01e | ||
|
6b3b8aa6ae | ||
|
3dd915e2fb | ||
|
1029871047 | ||
|
09923a6c22 | ||
|
e4300d0530 | ||
|
1ad20cb143 | ||
|
06bc8fecf6 | ||
|
2a29e66daa | ||
|
c7de0bc4ee | ||
|
a0b72f519b | ||
|
6d5e4a5776 | ||
|
e6e117f694 | ||
|
b8ff5938ff | ||
|
68d0e9958f | ||
|
00a04a1931 | ||
|
7744880ce7 | ||
|
6384182593 | ||
|
1af6dbbb78 | ||
|
d8c4b84ddc | ||
|
80879d3736 | ||
|
bea6ed4d65 | ||
|
280c4e97e2 | ||
|
62b3ccd111 | ||
|
5da017ec2e | ||
|
f328c7f266 | ||
|
0088be99f5 | ||
|
803a88ac2c | ||
|
773ff10dd1 | ||
|
b2b063196d | ||
|
e80e1dcf62 | ||
|
623c96e4bb | ||
|
2a97aca9b6 | ||
|
8f32397f50 | ||
|
40d8a05bb0 | ||
|
78b1cd41da | ||
|
cadcb9916f | ||
|
e17fe32ae2 | ||
|
cd362c749e | ||
|
8e4f7ffce6 | ||
|
675ad089fa | ||
|
f4f5d65bd8 | ||
|
8872263f45 | ||
|
95d2b5b8d9 | ||
|
84937ecaca | ||
|
adfb541fff |
264
kmath-noa/README.md
Normal file
264
kmath-noa/README.md
Normal file
@ -0,0 +1,264 @@
|
||||
# Module kmath-noa
|
||||
|
||||
A general purpose differentiable programming library over
|
||||
[NOA](https://github.com/grinisrit/noa.git)
|
||||
together with relevant functionality from
|
||||
[LibTorch](https://pytorch.org/cppdocs).
|
||||
|
||||
Our aim is to cover a wide set of applications
|
||||
from bayesian computation and deep learning to particle physics
|
||||
simulations. In fact, we support any
|
||||
differentiable program written on top of
|
||||
`AutoGrad` & `ATen`.
|
||||
|
||||
## Installation from source
|
||||
|
||||
Currently, we support only the linux platform for the native artifacts.
|
||||
For `GPU` kernels, we require a compatible
|
||||
[CUDA](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html)
|
||||
installation. If you are on Windows, we recommend setting up
|
||||
everything on [WSL](https://docs.nvidia.com/cuda/wsl-user-guide/index.html).
|
||||
|
||||
To install the library, you can simply publish `KMath` to the local
|
||||
Maven repository:
|
||||
```
|
||||
$ ./gradlew -Dorg.gradle.java.home=/path/to/local/jdk -q publishToMavenLocal
|
||||
```
|
||||
This will fetch and build the `JNI` wrapper `jnoa`.
|
||||
|
||||
The library has been tested with
|
||||
[graalvm-ce-java11-linux-amd64-22.0.0.2.](https://github.com/graalvm/graalvm-ce-builds/releases/tag/vm-22.0.0.2)
|
||||
|
||||
In your own application add the local dependency:
|
||||
```kotlin
|
||||
repositories {
|
||||
mavenCentral()
|
||||
mavenLocal()
|
||||
}
|
||||
|
||||
dependencies {
|
||||
implementation("space.kscience:kmath-noa:0.3.0-dev-17")
|
||||
}
|
||||
```
|
||||
To load the native library you will need to add to the VM options:
|
||||
```
|
||||
-Djava.library.path=${HOME}/.kmath/third-party/noa-v0.0.1/cpp-build/jnoa
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
The library is under active development. Many more features
|
||||
will be available soon.
|
||||
|
||||
### Tensors and Linear Algebra
|
||||
|
||||
We implement the tensor algebra interfaces
|
||||
from [kmath-tensors](../kmath-tensors):
|
||||
```kotlin
|
||||
NoaFloat {
|
||||
val tensor =
|
||||
randNormal(
|
||||
shape = intArrayOf(7, 5, 3),
|
||||
device = Device.CPU) // or Device.CUDA(0) for GPU
|
||||
|
||||
// Compute SVD
|
||||
val (tensorU, tensorS, tensorV) = tensor.svd()
|
||||
|
||||
// Reconstruct tensor
|
||||
val tensorReg =
|
||||
tensorU dot (diagonalEmbedding(tensorS) dot tensorV.transpose(-2, -1))
|
||||
|
||||
// Serialise tensor for later
|
||||
tensorReg.save("tensorReg.pt")
|
||||
}
|
||||
```
|
||||
|
||||
The saved tensor can be loaded in `C++` or in `python`:
|
||||
```python
|
||||
import torch
|
||||
tensor_reg = list(torch.jit.load('tensorReg.pt').parameters())[0]
|
||||
```
|
||||
|
||||
The most efficient way passing data between the `JVM` and the native backend
|
||||
is to rely on primitive arrays:
|
||||
```kotlin
|
||||
val array = (1..8).map { 100f * it }.toFloatArray()
|
||||
val updateArray = floatArrayOf(15f, 20f)
|
||||
val resArray = NoaFloat {
|
||||
val tensor = copyFromArray(array, intArrayOf(2, 2, 2))
|
||||
NoaFloat {
|
||||
// The call `tensor[0]` creates a native tensor instance pointing to a slice of `tensor`
|
||||
// The second call `[1]` is a setter call and does not create any new instances
|
||||
tensor[0][1] = updateArray
|
||||
// The instance `tensor[0]` is destroyed as we move out of the scope
|
||||
}!! // if the computation fails the result fill be null
|
||||
tensor.copyToArray()
|
||||
// the instance `tensor` is destroyed here
|
||||
}!!
|
||||
|
||||
```
|
||||
|
||||
### Automatic Differentiation
|
||||
The [AutoGrad](https://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html)
|
||||
engine is exposed:
|
||||
```kotlin
|
||||
NoaFloat {
|
||||
// Create a quadratic function
|
||||
val dim = 3
|
||||
val tensorX = randNormal(shape = intArrayOf(dim))
|
||||
val randFeatures = randNormal(shape = intArrayOf(dim, dim))
|
||||
val tensorSigma = randFeatures + randFeatures.transpose(0, 1)
|
||||
val tensorMu = randNormal(shape = intArrayOf(dim))
|
||||
|
||||
// Create a differentiable expression
|
||||
val expressionAtX = withGradAt(tensorX) { x ->
|
||||
0.5f * (x dot (tensorSigma dot x)) + (tensorMu dot x) + 25.9f
|
||||
}
|
||||
|
||||
// Evaluate the gradient at tensorX
|
||||
// retaining the graph for the hessian computation
|
||||
val gradientAtX = expressionAtX.autoGradient(tensorX, retainGraph = true)
|
||||
|
||||
// Compute the hessian at tensorX
|
||||
val hessianAtX = expressionAtX.autoHessian(tensorX)
|
||||
}
|
||||
```
|
||||
### Deep Learning
|
||||
You can train any [TorchScript](https://pytorch.org/docs/stable/jit.html) model.
|
||||
For example, you can build in `python` the following neural network
|
||||
and prepare the training data:
|
||||
|
||||
```python
|
||||
import torch
|
||||
|
||||
n_tr = 7
|
||||
n_val = 300
|
||||
x_val = torch.linspace(-5, 5, n_val).view(-1, 1)
|
||||
y_val = torch.sin(x_val)
|
||||
x_train = torch.linspace(-3.14, 3.14, n_tr).view(-1, 1)
|
||||
y_train = torch.sin(x_train) + torch.randn_like(x_train) * 0.1
|
||||
|
||||
class Data(torch.nn.Module):
|
||||
def __init__(self):
|
||||
super(Data, self).__init__()
|
||||
self.register_buffer('x_val', x_val)
|
||||
self.register_buffer('y_val', y_val)
|
||||
self.register_buffer('x_train', x_train)
|
||||
self.register_buffer('y_train', y_train)
|
||||
|
||||
class Net(torch.nn.Module):
|
||||
def __init__(self):
|
||||
super(Net, self).__init__()
|
||||
self.l1 = torch.nn.Linear(1, 10, bias = True)
|
||||
self.l2 = torch.nn.Linear(10, 10, bias = True)
|
||||
self.l3 = torch.nn.Linear(10, 1, bias = True)
|
||||
|
||||
def forward(self, x):
|
||||
x = self.l1(x)
|
||||
x = torch.relu(x)
|
||||
x = self.l2(x)
|
||||
x = torch.relu(x)
|
||||
x = self.l3(x)
|
||||
return x
|
||||
|
||||
class Loss(torch.nn.Module):
|
||||
def __init__(self, target):
|
||||
super(Loss, self).__init__()
|
||||
self.register_buffer('target', target)
|
||||
self.loss = torch.nn.MSELoss()
|
||||
|
||||
def forward(self, x):
|
||||
return self.loss(x, self.target)
|
||||
|
||||
# Generate TorchScript modules and serialise them
|
||||
torch.jit.script(Data()).save('data.pt')
|
||||
torch.jit.script(Net()).save('net.pt')
|
||||
torch.jit.script(Loss(y_train)).save('loss.pt')
|
||||
```
|
||||
|
||||
You can then load the modules into `kotlin` and train them:
|
||||
```kotlin
|
||||
NoaFloat {
|
||||
|
||||
// Load the serialised JIT modules
|
||||
// The training data
|
||||
val dataModule = loadJitModule("data.pt")
|
||||
// The DL model
|
||||
val netModule = loadJitModule("net.pt")
|
||||
// The loss function
|
||||
val lossModule = loadJitModule("loss.pt")
|
||||
|
||||
// Get the tensors from the module
|
||||
val xTrain = dataModule.getBuffer("x_train")
|
||||
val yTrain = dataModule.getBuffer("y_train")
|
||||
val xVal = dataModule.getBuffer("x_val")
|
||||
val yVal = dataModule.getBuffer("y_val")
|
||||
|
||||
// Set the model in training mode
|
||||
netModule.train(true)
|
||||
// Loss function for training
|
||||
lossModule.setBuffer("target", yTrain)
|
||||
|
||||
// Compute the predictions
|
||||
val yPred = netModule.forward(xTrain)
|
||||
// Compute the training loss
|
||||
val loss = lossModule.forward(yPred)
|
||||
println(loss)
|
||||
|
||||
// Set-up the Adam optimiser with learning rate 0.005
|
||||
val optimiser = netModule.adamOptimiser(0.005)
|
||||
|
||||
// Train for 250 epochs
|
||||
repeat(250){
|
||||
// Clean gradients
|
||||
optimiser.zeroGrad()
|
||||
// Use forwardAssign to for better memory management
|
||||
netModule.forwardAssign(xTrain, yPred)
|
||||
lossModule.forwardAssign(yPred, loss)
|
||||
// Backward pass
|
||||
loss.backward()
|
||||
// Update model parameters
|
||||
optimiser.step()
|
||||
if(it % 50 == 0)
|
||||
println("Training loss: $loss")
|
||||
}
|
||||
|
||||
// Finally validate the model
|
||||
// Compute the predictions for the validation features
|
||||
netModule.forwardAssign(xVal, yPred)
|
||||
// Set the loss for validation
|
||||
lossModule.setBuffer("target", yVal)
|
||||
// Compute the loss on validation dataset
|
||||
lossModule.forwardAssign(yPred, loss)
|
||||
println("Validation loss: $loss")
|
||||
|
||||
// The model can be serialised in its current state
|
||||
netModule.save("trained_net.pt")
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
### Custom memory management
|
||||
Native memory management relies on scoping
|
||||
with [NoaScope](src/main/kotlin/space/kscience/kmath/noa/memory/NoaScope.kt)
|
||||
which is readily available within an algebra context.
|
||||
Manual management is also possible:
|
||||
```kotlin
|
||||
// Create a scope
|
||||
val scope = NoaScope()
|
||||
|
||||
val tensor = NoaFloat(scope){
|
||||
full(5f, intArrayOf(1))
|
||||
}!! // the result might be null
|
||||
|
||||
// If the computation fails resources will be freed automatically
|
||||
// Otherwise it's your responsibility:
|
||||
scope.disposeAll()
|
||||
|
||||
// Attempts to use tensor here is undefined behaviour
|
||||
```
|
||||
|
||||
For more examples have a look at
|
||||
[NOA](https://github.com/grinisrit/noa) docs.
|
||||
|
||||
Contributed by [Roland Grinis](https://github.com/grinisrit)
|
200
kmath-noa/build.gradle.kts
Normal file
200
kmath-noa/build.gradle.kts
Normal file
@ -0,0 +1,200 @@
|
||||
/*
|
||||
* Copyright 2018-2021 KMath contributors.
|
||||
* Use of this source code is governed by the Apache 2.0 license that can be found in the license/LICENSE.txt file.
|
||||
*/
|
||||
|
||||
import de.undercouch.gradle.tasks.download.Download
|
||||
|
||||
|
||||
plugins {
|
||||
kotlin("jvm")
|
||||
id("ru.mipt.npm.gradle.common")
|
||||
id("de.undercouch.download")
|
||||
}
|
||||
|
||||
description = "Wrapper for the Differentiable Computation library NOA on top of LibTorch"
|
||||
|
||||
dependencies {
|
||||
implementation(project(":kmath-tensors"))
|
||||
}
|
||||
|
||||
val home: String = System.getProperty("user.home")
|
||||
val javaHome: String = System.getProperty("java.home")
|
||||
val thirdPartyDir = "$home/.kmath/third-party/noa-v0.0.1"
|
||||
val cppBuildDir = "$thirdPartyDir/cpp-build"
|
||||
val jNoaDir = "$thirdPartyDir/jnoa/noa-kmath" //"$home/devspace/noa"
|
||||
|
||||
val cudaHome: String? = System.getenv("CUDA_HOME")
|
||||
val cudaDefault = file("/usr/local/cuda").exists()
|
||||
val cudaFound = (cudaHome?.isNotEmpty() ?: false) or cudaDefault //false
|
||||
|
||||
val cmakeArchive = "cmake-3.20.5-linux-x86_64"
|
||||
val torchArchive = "libtorch"
|
||||
val clangArchive = "clang+llvm-12.0.1-x86_64-linux-gnu-ubuntu-16.04"
|
||||
|
||||
val cmakeCmd = "$thirdPartyDir/cmake/$cmakeArchive/bin/cmake"
|
||||
val ninjaCmd = "$thirdPartyDir/ninja/ninja"
|
||||
val clangRootDir = "$thirdPartyDir/clang/$clangArchive"
|
||||
val clangCmd = "$clangRootDir/bin/clang"
|
||||
val clangxxCmd = "$clangRootDir/bin/clang++"
|
||||
|
||||
val generateJNIHeader by tasks.registering {
|
||||
doLast {
|
||||
exec {
|
||||
workingDir(projectDir.resolve("src/main/java/space/kscience/kmath/noa"))
|
||||
commandLine(
|
||||
"$javaHome/bin/javac", "-h",
|
||||
projectDir.resolve("src/main/resources"), "JNoa.java"
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
val downloadCMake by tasks.registering(Download::class) {
|
||||
val tarFile = "$cmakeArchive.tar.gz"
|
||||
src("https://github.com/Kitware/CMake/releases/download/v3.20.5/$tarFile")
|
||||
dest(File("$thirdPartyDir/cmake", tarFile))
|
||||
overwrite(false)
|
||||
}
|
||||
|
||||
|
||||
val downloadNinja by tasks.registering(Download::class) {
|
||||
src("https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip")
|
||||
dest(File("$thirdPartyDir/ninja", "ninja-linux.zip"))
|
||||
overwrite(false)
|
||||
}
|
||||
|
||||
val downloadClang by tasks.registering(Download::class) {
|
||||
val tarFile = "$clangArchive.tar.xz"
|
||||
src("https://github.com/llvm/llvm-project/releases/download/llvmorg-12.0.1/$tarFile")
|
||||
dest(File("$thirdPartyDir/clang", tarFile))
|
||||
overwrite(false)
|
||||
}
|
||||
|
||||
val downloadTorch by tasks.registering(Download::class) {
|
||||
val torchVersion = "$torchArchive-shared-with-deps-1.10.2%2B"
|
||||
val cudaUrl = "https://download.pytorch.org/libtorch/cu113/${torchVersion}cu113.zip"
|
||||
val cpuUrl = "https://download.pytorch.org/libtorch/cpu/${torchVersion}cpu.zip"
|
||||
val url = if (cudaFound) cudaUrl else cpuUrl
|
||||
src(url)
|
||||
dest(File("$thirdPartyDir/torch", "$torchArchive.zip"))
|
||||
overwrite(false)
|
||||
}
|
||||
|
||||
fun downloadJNoaHelper(update: Boolean) = tasks.registering(Download::class) {
|
||||
src("https://github.com/grinisrit/noa/archive/refs/heads/kmath.zip")
|
||||
dest(File("$thirdPartyDir/jnoa", "kmath.zip"))
|
||||
overwrite(update)
|
||||
}
|
||||
|
||||
val downloadJNoa by downloadJNoaHelper(false)
|
||||
|
||||
val reDownloadJNoa by downloadJNoaHelper(true)
|
||||
|
||||
|
||||
val extractCMake by tasks.registering(Copy::class) {
|
||||
dependsOn(downloadCMake)
|
||||
from(tarTree(resources.gzip(downloadCMake.get().dest)))
|
||||
into("$thirdPartyDir/cmake")
|
||||
}
|
||||
|
||||
val extractNinja by tasks.registering(Copy::class) {
|
||||
dependsOn(downloadNinja)
|
||||
from(zipTree(downloadNinja.get().dest))
|
||||
into("$thirdPartyDir/ninja")
|
||||
}
|
||||
|
||||
val extractClang by tasks.registering {
|
||||
dependsOn(downloadClang)
|
||||
onlyIf { !file(clangRootDir).exists() }
|
||||
doLast {
|
||||
exec {
|
||||
workingDir("$thirdPartyDir/clang")
|
||||
commandLine("mkdir", clangArchive)
|
||||
}
|
||||
exec {
|
||||
workingDir("$thirdPartyDir/clang")
|
||||
commandLine("tar", "-xf", "$clangArchive.tar.xz",
|
||||
"-C", clangArchive, "--strip-components", "1")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
val extractTorch by tasks.registering(Copy::class) {
|
||||
dependsOn(downloadTorch)
|
||||
from(zipTree(downloadTorch.get().dest))
|
||||
into("$thirdPartyDir/torch")
|
||||
}
|
||||
|
||||
val extractJNoa by tasks.registering(Copy::class) {
|
||||
dependsOn(downloadJNoa)
|
||||
from(zipTree(downloadJNoa.get().dest))
|
||||
into("$thirdPartyDir/jnoa")
|
||||
}
|
||||
|
||||
val configureCpp by tasks.registering {
|
||||
dependsOn(extractCMake)
|
||||
dependsOn(extractNinja)
|
||||
dependsOn(extractClang)
|
||||
dependsOn(extractTorch)
|
||||
dependsOn(extractJNoa)
|
||||
onlyIf { !file(cppBuildDir).exists() }
|
||||
doLast {
|
||||
exec {
|
||||
workingDir(thirdPartyDir)
|
||||
commandLine("mkdir", "-p", cppBuildDir)
|
||||
}
|
||||
exec {
|
||||
workingDir(cppBuildDir)
|
||||
commandLine(
|
||||
cmakeCmd,
|
||||
jNoaDir,
|
||||
"-GNinja",
|
||||
"-DCMAKE_MAKE_PROGRAM=$ninjaCmd",
|
||||
"-DCMAKE_C_COMPILER=$clangCmd",
|
||||
"-DCMAKE_CXX_COMPILER=$clangxxCmd",
|
||||
"-DCMAKE_PREFIX_PATH=$thirdPartyDir/torch/$torchArchive",
|
||||
"-DJAVA_HOME=$javaHome",
|
||||
"-DBUILD_JNOA=ON",
|
||||
"-DCMAKE_BUILD_TYPE=Release",
|
||||
"-DBUILD_NOA_TESTS=OFF",
|
||||
"-DBUILD_NOA_BENCHMARKS=OFF",
|
||||
"-DINSTALL_NOA=OFF"
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
val cleanCppBuild by tasks.registering {
|
||||
onlyIf { file(cppBuildDir).exists() }
|
||||
doLast {
|
||||
exec {
|
||||
workingDir(thirdPartyDir)
|
||||
commandLine("rm", "-rf", cppBuildDir)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
val buildCpp by tasks.registering {
|
||||
dependsOn(configureCpp)
|
||||
doLast {
|
||||
exec {
|
||||
workingDir(cppBuildDir)
|
||||
commandLine(cmakeCmd, "--build", ".", "--config", "Release", "--target", "jnoa")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
tasks["compileJava"].dependsOn(buildCpp)
|
||||
|
||||
tasks {
|
||||
withType<Test>{
|
||||
systemProperty("java.library.path", "$cppBuildDir/jnoa")
|
||||
//systemProperty("java.library.path",
|
||||
// "${System.getProperty("user.home")}/devspace/noa/cmake-build-release/jnoa")
|
||||
}
|
||||
}
|
||||
|
||||
readme {
|
||||
maturity = ru.mipt.npm.gradle.Maturity.PROTOTYPE
|
||||
}
|
407
kmath-noa/src/main/java/space/kscience/kmath/noa/JNoa.java
Normal file
407
kmath-noa/src/main/java/space/kscience/kmath/noa/JNoa.java
Normal file
@ -0,0 +1,407 @@
|
||||
/*
|
||||
* Copyright 2018-2021 KMath contributors.
|
||||
* Use of this source code is governed by the Apache 2.0 license that can be found in the license/LICENSE.txt file.
|
||||
*/
|
||||
|
||||
package space.kscience.kmath.noa;
|
||||
|
||||
class JNoa {
|
||||
|
||||
static {
|
||||
try {
|
||||
System.loadLibrary("jnoa");
|
||||
} catch (UnsatisfiedLinkError e) {
|
||||
System.err.println(
|
||||
"Failed to load the native library NOA:\n" +
|
||||
" - Follow the installation instructions from\n" +
|
||||
" https://github.com/grinisrit/noa \n" +
|
||||
" - Set java.library.path to the location of libjnoa.so");
|
||||
System.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
public static native int testException(int seed);
|
||||
|
||||
public static native boolean cudaIsAvailable();
|
||||
|
||||
public static native int getNumThreads();
|
||||
|
||||
public static native void setNumThreads(int numThreads);
|
||||
|
||||
public static native void setSeed(int seed);
|
||||
|
||||
public static native void disposeTensor(long tensorHandle);
|
||||
|
||||
public static native long emptyTensor();
|
||||
|
||||
public static native long fromBlobDouble(double[] data, int[] shape, int device);
|
||||
|
||||
public static native long fromBlobFloat(float[] data, int[] shape, int device);
|
||||
|
||||
public static native long fromBlobLong(long[] data, int[] shape, int device);
|
||||
|
||||
public static native long fromBlobInt(int[] data, int[] shape, int device);
|
||||
|
||||
public static native long copyTensor(long tensorHandle);
|
||||
|
||||
public static native long copyToDevice(long tensorHandle, int device);
|
||||
|
||||
public static native long copyToDouble(long tensorHandle);
|
||||
|
||||
public static native long copyToFloat(long tensorHandle);
|
||||
|
||||
public static native long copyToLong(long tensorHandle);
|
||||
|
||||
public static native long copyToInt(long tensorHandle);
|
||||
|
||||
public static native long viewTensor(long tensorHandle, int[] shape);
|
||||
|
||||
public static native long viewAsTensor(long tensorHandle, long asTensorHandle);
|
||||
|
||||
public static native String tensorToString(long tensorHandle);
|
||||
|
||||
public static native int getDim(long tensorHandle);
|
||||
|
||||
public static native int getNumel(long tensorHandle);
|
||||
|
||||
public static native int getShapeAt(long tensorHandle, int d);
|
||||
|
||||
public static native int getStrideAt(long tensorHandle, int d);
|
||||
|
||||
public static native int getDevice(long tensorHandle);
|
||||
|
||||
public static native double getItemDouble(long tensorHandle);
|
||||
|
||||
public static native float getItemFloat(long tensorHandle);
|
||||
|
||||
public static native long getItemLong(long tensorHandle);
|
||||
|
||||
public static native int getItemInt(long tensorHandle);
|
||||
|
||||
public static native long getIndex(long tensorHandle, int index);
|
||||
|
||||
public static native double getDouble(long tensorHandle, int[] index);
|
||||
|
||||
public static native float getFloat(long tensorHandle, int[] index);
|
||||
|
||||
public static native long getLong(long tensorHandle, int[] index);
|
||||
|
||||
public static native int getInt(long tensorHandle, int[] index);
|
||||
|
||||
public static native void setDouble(long tensorHandle, int[] index, double value);
|
||||
|
||||
public static native void setFloat(long tensorHandle, int[] index, float value);
|
||||
|
||||
public static native void setLong(long tensorHandle, int[] index, long value);
|
||||
|
||||
public static native void setInt(long tensorHandle, int[] index, int value);
|
||||
|
||||
public static native long randDouble(int[] shape, int device);
|
||||
|
||||
public static native long randnDouble(int[] shape, int device);
|
||||
|
||||
public static native long randFloat(int[] shape, int device);
|
||||
|
||||
public static native long randnFloat(int[] shape, int device);
|
||||
|
||||
public static native long randintDouble(long low, long high, int[] shape, int device);
|
||||
|
||||
public static native long randintFloat(long low, long high, int[] shape, int device);
|
||||
|
||||
public static native long randintLong(long low, long high, int[] shape, int device);
|
||||
|
||||
public static native long randintInt(long low, long high, int[] shape, int device);
|
||||
|
||||
public static native long randLike(long tensorHandle);
|
||||
|
||||
public static native void randLikeAssign(long tensorHandle);
|
||||
|
||||
public static native long randnLike(long tensorHandle);
|
||||
|
||||
public static native void randnLikeAssign(long tensorHandle);
|
||||
|
||||
public static native long randintLike(long tensorHandle, long low, long high);
|
||||
|
||||
public static native void randintLikeAssign(long tensorHandle, long low, long high);
|
||||
|
||||
public static native long fullDouble(double value, int[] shape, int device);
|
||||
|
||||
public static native long fullFloat(float value, int[] shape, int device);
|
||||
|
||||
public static native long fullLong(long value, int[] shape, int device);
|
||||
|
||||
public static native long fullInt(int value, int[] shape, int device);
|
||||
|
||||
public static native long timesDouble(double value, long other);
|
||||
|
||||
public static native long timesFloat(float value, long other);
|
||||
|
||||
public static native long timesLong(long value, long other);
|
||||
|
||||
public static native long timesInt(int value, long other);
|
||||
|
||||
public static native void timesDoubleAssign(double value, long other);
|
||||
|
||||
public static native void timesFloatAssign(float value, long other);
|
||||
|
||||
public static native void timesLongAssign(long value, long other);
|
||||
|
||||
public static native void timesIntAssign(int value, long other);
|
||||
|
||||
public static native long plusDouble(double value, long other);
|
||||
|
||||
public static native long plusFloat(float value, long other);
|
||||
|
||||
public static native long plusLong(long value, long other);
|
||||
|
||||
public static native long plusInt(int value, long other);
|
||||
|
||||
public static native void plusDoubleAssign(double value, long other);
|
||||
|
||||
public static native void plusFloatAssign(float value, long other);
|
||||
|
||||
public static native void plusLongAssign(long value, long other);
|
||||
|
||||
public static native void plusIntAssign(int value, long other);
|
||||
|
||||
public static native long timesTensor(long lhs, long rhs);
|
||||
|
||||
public static native void timesTensorAssign(long lhs, long rhs);
|
||||
|
||||
public static native long divTensor(long lhs, long rhs);
|
||||
|
||||
public static native void divTensorAssign(long lhs, long rhs);
|
||||
|
||||
public static native long plusTensor(long lhs, long rhs);
|
||||
|
||||
public static native void plusTensorAssign(long lhs, long rhs);
|
||||
|
||||
public static native long minusTensor(long lhs, long rhs);
|
||||
|
||||
public static native void minusTensorAssign(long lhs, long rhs);
|
||||
|
||||
public static native long unaryMinus(long tensorHandle);
|
||||
|
||||
public static native long transposeTensor(long tensorHandle, int i, int j);
|
||||
|
||||
public static native long absTensor(long tensorHandle);
|
||||
|
||||
public static native long expTensor(long tensorHandle);
|
||||
|
||||
public static native long lnTensor(long tensorHandle);
|
||||
|
||||
public static native long sqrtTensor(long tensorHandle);
|
||||
|
||||
public static native long cosTensor(long tensorHandle);
|
||||
|
||||
public static native long acosTensor(long tensorHandle);
|
||||
|
||||
public static native long coshTensor(long tensorHandle);
|
||||
|
||||
public static native long acoshTensor(long tensorHandle);
|
||||
|
||||
public static native long sinTensor(long tensorHandle);
|
||||
|
||||
public static native long asinTensor(long tensorHandle);
|
||||
|
||||
public static native long sinhTensor(long tensorHandle);
|
||||
|
||||
public static native long asinhTensor(long tensorHandle);
|
||||
|
||||
public static native long tanTensor(long tensorHandle);
|
||||
|
||||
public static native long atanTensor(long tensorHandle);
|
||||
|
||||
public static native long tanhTensor(long tensorHandle);
|
||||
|
||||
public static native long atanhTensor(long tensorHandle);
|
||||
|
||||
public static native long ceilTensor(long tensorHandle);
|
||||
|
||||
public static native long floorTensor(long tensorHandle);
|
||||
|
||||
public static native long sumTensor(long tensorHandle);
|
||||
|
||||
public static native long sumDimTensor(long tensorHandle, int dim, boolean keepDim);
|
||||
|
||||
public static native long minTensor(long tensorHandle);
|
||||
|
||||
public static native long minDimTensor(long tensorHandle, int dim, boolean keepDim);
|
||||
|
||||
public static native long maxTensor(long tensorHandle);
|
||||
|
||||
public static native long maxDimTensor(long tensorHandle, int dim, boolean keepDim);
|
||||
|
||||
public static native long meanTensor(long tensorHandle);
|
||||
|
||||
public static native long meanDimTensor(long tensorHandle, int dim, boolean keepDim);
|
||||
|
||||
public static native long stdTensor(long tensorHandle);
|
||||
|
||||
public static native long stdDimTensor(long tensorHandle, int dim, boolean keepDim);
|
||||
|
||||
public static native long varTensor(long tensorHandle);
|
||||
|
||||
public static native long varDimTensor(long tensorHandle, int dim, boolean keepDim);
|
||||
|
||||
public static native long argMaxTensor(long tensorHandle, int dim, boolean keepDim);
|
||||
|
||||
public static native long flattenTensor(long tensorHandle, int startDim, int endDim);
|
||||
|
||||
public static native long matmul(long lhs, long rhs);
|
||||
|
||||
public static native void matmulAssign(long lhs, long rhs);
|
||||
|
||||
public static native void matmulRightAssign(long lhs, long rhs);
|
||||
|
||||
public static native long diagEmbed(long diagsHandle, int offset, int dim1, int dim2);
|
||||
|
||||
public static native long detTensor(long tensorHandle);
|
||||
|
||||
public static native long invTensor(long tensorHandle);
|
||||
|
||||
public static native long choleskyTensor(long tensorHandle);
|
||||
|
||||
public static native void qrTensor(long tensorHandle, long Qhandle, long Rhandle);
|
||||
|
||||
public static native void luTensor(long tensorHandle, long Phandle, long Lhandle, long Uhandle);
|
||||
|
||||
public static native void svdTensor(long tensorHandle, long Uhandle, long Shandle, long Vhandle);
|
||||
|
||||
public static native void symEigTensor(long tensorHandle, long Shandle, long Vhandle);
|
||||
|
||||
public static native boolean requiresGrad(long tensorHandle);
|
||||
|
||||
public static native void setRequiresGrad(long tensorHandle, boolean status);
|
||||
|
||||
public static native long detachFromGraph(long tensorHandle);
|
||||
|
||||
public static native long autoGradTensor(long value, long variable, boolean retainGraph);
|
||||
|
||||
public static native long autoHessTensor(long value, long variable);
|
||||
|
||||
public static native void backwardPass(long tensorHandle);
|
||||
|
||||
public static native long tensorGrad(long tensorHandle);
|
||||
|
||||
public static native void disposeJitModule(long jitModuleHandle);
|
||||
|
||||
public static native void trainMode(long jitModuleHandle, boolean status);
|
||||
|
||||
public static native long loadJitModuleDouble(String path, int device);
|
||||
|
||||
public static native long loadJitModuleFloat(String path, int device);
|
||||
|
||||
public static native long loadJitModuleLong(String path, int device);
|
||||
|
||||
public static native long loadJitModuleInt(String path, int device);
|
||||
|
||||
public static native long forwardPass(long jitModuleHandle, long tensorHandle);
|
||||
|
||||
public static native void forwardPassAssign(long jitModuleHandle, long featuresHandle, long predsHandle);
|
||||
|
||||
public static native long getModuleParameter(long jitModuleHandle, String name);
|
||||
|
||||
public static native void setModuleParameter(long jitModuleHandle, String name, long tensorHandle);
|
||||
|
||||
public static native long getModuleBuffer(long jitModuleHandle, String name);
|
||||
|
||||
public static native void setModuleBuffer(long jitModuleHandle, String name, long tensorHandle);
|
||||
|
||||
public static native long adamOptim(long jitModuleHandle, double learningRate);
|
||||
|
||||
public static native void disposeAdamOptim(long adamOptHandle);
|
||||
|
||||
public static native void stepAdamOptim(long adamOptHandle);
|
||||
|
||||
public static native void zeroGradAdamOptim(long adamOptHandle);
|
||||
|
||||
public static native long rmsOptim(long jitModuleHandle, double learningRate, double alpha,
|
||||
double eps, double weight_decay, double momentum, boolean centered);
|
||||
|
||||
public static native void disposeRmsOptim(long rmsOptHandle);
|
||||
|
||||
public static native void stepRmsOptim(long rmsOptHandle);
|
||||
|
||||
public static native void zeroGradRmsOptim(long rmsOptHandle);
|
||||
|
||||
public static native long adamWOptim(long jitModuleHandle, double learningRate, double beta1,
|
||||
double beta2, double eps, double weight_decay, boolean amsgrad);
|
||||
|
||||
public static native void disposeAdamWOptim(long adamWOptHandle);
|
||||
|
||||
public static native void stepAdamWOptim(long adamWOptHandle);
|
||||
|
||||
public static native void zeroGradAdamWOptim(long adamWOptHandle);
|
||||
|
||||
public static native long adagradOptim(long jitModuleHandle, double learningRate, double weight_decay,
|
||||
double lr_decay, double initial_accumulator_value, double eps);
|
||||
|
||||
public static native void disposeAdagradOptim(long adagradOptHandle);
|
||||
|
||||
public static native void stepAdagradOptim(long adagradOptHandle);
|
||||
|
||||
public static native void zeroGradAdagradOptim(long adagradOptHandle);
|
||||
|
||||
public static native long sgdOptim(long jitModuleHandle, double learningRate, double momentum,
|
||||
double dampening, double weight_decay, boolean nesterov);
|
||||
|
||||
public static native void disposeSgdOptim(long sgdOptHandle);
|
||||
|
||||
public static native void stepSgdOptim(long sgdOptHandle);
|
||||
|
||||
public static native void zeroGradSgdOptim(long sgdOptHandle);
|
||||
|
||||
public static native void swapTensors(long lhsHandle, long rhsHandle);
|
||||
|
||||
public static native long loadTensorDouble(String path, int device);
|
||||
|
||||
public static native long loadTensorFloat(String path, int device);
|
||||
|
||||
public static native long loadTensorLong(String path, int device);
|
||||
|
||||
public static native long loadTensorInt(String path, int device);
|
||||
|
||||
public static native void saveTensor(long tensorHandle, String path);
|
||||
|
||||
public static native void saveJitModule(long jitModuleHandle, String path);
|
||||
|
||||
public static native void assignBlobDouble(long tensorHandle, double[] data);
|
||||
|
||||
public static native void assignBlobFloat(long tensorHandle, float[] data);
|
||||
|
||||
public static native void assignBlobLong(long tensorHandle, long[] data);
|
||||
|
||||
public static native void assignBlobInt(long tensorHandle, int[] data);
|
||||
|
||||
public static native void setBlobDouble(long tensorHandle, int i, double[] data);
|
||||
|
||||
public static native void setBlobFloat(long tensorHandle, int i, float[] data);
|
||||
|
||||
public static native void setBlobLong(long tensorHandle, int i, long[] data);
|
||||
|
||||
public static native void setBlobInt(long tensorHandle, int i, int[] data);
|
||||
|
||||
public static native void getBlobDouble(long tensorHandle, double[] data);
|
||||
|
||||
public static native void getBlobFloat(long tensorHandle, float[] data);
|
||||
|
||||
public static native void getBlobLong(long tensorHandle, long[] data);
|
||||
|
||||
public static native void getBlobInt(long tensorHandle, int[] data);
|
||||
|
||||
public static native void setTensor(long tensorHandle, int i, long tensorValue);
|
||||
|
||||
public static native long getSliceTensor(long tensorHandle, int dim, int start, int end);
|
||||
|
||||
public static native void setSliceTensor(long tensorHandle, int dim, int start, int end, long tensorValue);
|
||||
|
||||
public static native void setSliceBlobDouble(long tensorHandle, int dim, int start, int end, double[] data);
|
||||
|
||||
public static native void setSliceBlobFloat(long tensorHandle, int dim, int start, int end, float[] data);
|
||||
|
||||
public static native void setSliceBlobLong(long tensorHandle, int dim, int start, int end, long[] data);
|
||||
|
||||
public static native void setSliceBlobInt(long tensorHandle, int dim, int start, int end, int[] data);
|
||||
|
||||
}
|
@ -0,0 +1,12 @@
|
||||
/*
|
||||
* Copyright 2018-2021 KMath contributors.
|
||||
* Use of this source code is governed by the Apache 2.0 license that can be found in the license/LICENSE.txt file.
|
||||
*/
|
||||
|
||||
package space.kscience.kmath.noa;
|
||||
|
||||
public class NoaException extends Exception {
|
||||
public NoaException(String errorMessage) {
|
||||
super(errorMessage);
|
||||
}
|
||||
}
|
778
kmath-noa/src/main/kotlin/space/kscience/kmath/noa/algebras.kt
Normal file
778
kmath-noa/src/main/kotlin/space/kscience/kmath/noa/algebras.kt
Normal file
@ -0,0 +1,778 @@
|
||||
/*
|
||||
* Copyright 2018-2021 KMath contributors.
|
||||
* Use of tensor source code is governed by the Apache 2.0 license that can be found in the license/LICENSE.txt file.
|
||||
*/
|
||||
|
||||
package space.kscience.kmath.noa
|
||||
|
||||
import space.kscience.kmath.misc.PerformancePitfall
|
||||
import space.kscience.kmath.nd.StructureND
|
||||
import space.kscience.kmath.noa.memory.NoaScope
|
||||
import space.kscience.kmath.operations.*
|
||||
import space.kscience.kmath.tensors.api.AnalyticTensorAlgebra
|
||||
import space.kscience.kmath.tensors.api.LinearOpsTensorAlgebra
|
||||
import space.kscience.kmath.tensors.api.Tensor
|
||||
import space.kscience.kmath.tensors.api.TensorAlgebra
|
||||
import space.kscience.kmath.tensors.core.TensorLinearStructure
|
||||
|
||||
internal typealias Slice = Pair<Int, Int>
|
||||
|
||||
public sealed class NoaAlgebra<T, A : Ring<T>, PrimitiveArray, TensorType : NoaTensor<T>>
|
||||
protected constructor(protected val scope: NoaScope) :
|
||||
TensorAlgebra<T, A> {
|
||||
|
||||
protected abstract val StructureND<T>.tensor: TensorType
|
||||
|
||||
protected abstract fun wrap(tensorHandle: TensorHandle): TensorType
|
||||
|
||||
@PerformancePitfall
|
||||
public fun Tensor<T>.cast(): TensorType = tensor
|
||||
|
||||
/**
|
||||
* A scalar tensor must have empty shape
|
||||
*/
|
||||
override fun StructureND<T>.valueOrNull(): T? =
|
||||
try {
|
||||
tensor.item()
|
||||
} catch (e: NoaException) {
|
||||
null
|
||||
}
|
||||
|
||||
override fun StructureND<T>.value(): T = tensor.item()
|
||||
|
||||
public abstract fun randDiscrete(low: Long, high: Long, shape: IntArray, device: Device = Device.CPU): TensorType
|
||||
|
||||
public abstract fun TensorType.copyToArray(): PrimitiveArray
|
||||
|
||||
public abstract fun copyFromArray(array: PrimitiveArray, shape: IntArray, device: Device = Device.CPU): TensorType
|
||||
|
||||
public abstract fun full(value: T, shape: IntArray, device: Device = Device.CPU): TensorType
|
||||
|
||||
override operator fun StructureND<T>.times(arg: StructureND<T>): TensorType {
|
||||
return wrap(JNoa.timesTensor(tensor.tensorHandle, arg.tensor.tensorHandle))
|
||||
}
|
||||
|
||||
override operator fun Tensor<T>.timesAssign(arg: StructureND<T>): Unit {
|
||||
JNoa.timesTensorAssign(tensor.tensorHandle, arg.tensor.tensorHandle)
|
||||
}
|
||||
|
||||
override operator fun StructureND<T>.plus(arg: StructureND<T>): TensorType {
|
||||
return wrap(JNoa.plusTensor(tensor.tensorHandle, arg.tensor.tensorHandle))
|
||||
}
|
||||
|
||||
override operator fun Tensor<T>.plusAssign(arg: StructureND<T>): Unit {
|
||||
JNoa.plusTensorAssign(tensor.tensorHandle, arg.tensor.tensorHandle)
|
||||
}
|
||||
|
||||
override operator fun StructureND<T>.minus(arg: StructureND<T>): TensorType {
|
||||
return wrap(JNoa.minusTensor(tensor.tensorHandle, arg.tensor.tensorHandle))
|
||||
}
|
||||
|
||||
override operator fun Tensor<T>.minusAssign(arg: StructureND<T>): Unit {
|
||||
JNoa.minusTensorAssign(tensor.tensorHandle, arg.tensor.tensorHandle)
|
||||
}
|
||||
|
||||
override operator fun StructureND<T>.unaryMinus(): TensorType =
|
||||
wrap(JNoa.unaryMinus(tensor.tensorHandle))
|
||||
|
||||
override infix fun StructureND<T>.dot(other: StructureND<T>): TensorType {
|
||||
return wrap(JNoa.matmul(tensor.tensorHandle, other.tensor.tensorHandle))
|
||||
}
|
||||
|
||||
public infix fun Tensor<T>.dotAssign(arg: StructureND<T>): Unit {
|
||||
JNoa.matmulAssign(tensor.tensorHandle, arg.tensor.tensorHandle)
|
||||
}
|
||||
|
||||
public infix fun StructureND<T>.dotRightAssign(arg: Tensor<T>): Unit {
|
||||
JNoa.matmulRightAssign(tensor.tensorHandle, arg.tensor.tensorHandle)
|
||||
}
|
||||
|
||||
override operator fun Tensor<T>.get(i: Int): TensorType =
|
||||
wrap(JNoa.getIndex(tensor.tensorHandle, i))
|
||||
|
||||
public operator fun TensorType.set(i: Int, value: Tensor<T>): Unit =
|
||||
JNoa.setTensor(tensorHandle, i, value.tensor.tensorHandle)
|
||||
|
||||
public abstract operator fun TensorType.set(i: Int, array: PrimitiveArray): Unit
|
||||
|
||||
public operator fun Tensor<T>.get(dim: Int, slice: Slice): TensorType =
|
||||
wrap(JNoa.getSliceTensor(tensor.tensorHandle, dim, slice.first, slice.second))
|
||||
|
||||
public operator fun TensorType.set(dim: Int, slice: Slice, value: Tensor<T>): Unit =
|
||||
JNoa.setSliceTensor(tensorHandle, dim, slice.first, slice.second, value.tensor.tensorHandle)
|
||||
|
||||
public abstract operator fun TensorType.set(dim: Int, slice: Slice, array: PrimitiveArray): Unit
|
||||
|
||||
override fun diagonalEmbedding(
|
||||
diagonalEntries: Tensor<T>, offset: Int, dim1: Int, dim2: Int
|
||||
): TensorType =
|
||||
wrap(JNoa.diagEmbed(diagonalEntries.tensor.tensorHandle, offset, dim1, dim2))
|
||||
|
||||
override fun Tensor<T>.transpose(i: Int, j: Int): TensorType {
|
||||
return wrap(JNoa.transposeTensor(tensor.tensorHandle, i, j))
|
||||
}
|
||||
|
||||
override fun Tensor<T>.view(shape: IntArray): TensorType {
|
||||
return wrap(JNoa.viewTensor(tensor.tensorHandle, shape))
|
||||
}
|
||||
|
||||
override fun Tensor<T>.viewAs(other: StructureND<T>): TensorType {
|
||||
return wrap(JNoa.viewAsTensor(tensor.tensorHandle, other.tensor.tensorHandle))
|
||||
}
|
||||
|
||||
public fun StructureND<T>.abs(): TensorType = wrap(JNoa.absTensor(tensor.tensorHandle))
|
||||
|
||||
public fun StructureND<T>.sumAll(): TensorType = wrap(JNoa.sumTensor(tensor.tensorHandle))
|
||||
override fun StructureND<T>.sum(): T = sumAll().item()
|
||||
override fun StructureND<T>.sum(dim: Int, keepDim: Boolean): TensorType =
|
||||
wrap(JNoa.sumDimTensor(tensor.tensorHandle, dim, keepDim))
|
||||
|
||||
public fun StructureND<T>.minAll(): TensorType = wrap(JNoa.minTensor(tensor.tensorHandle))
|
||||
override fun StructureND<T>.min(): T = minAll().item()
|
||||
override fun StructureND<T>.min(dim: Int, keepDim: Boolean): TensorType =
|
||||
wrap(JNoa.minDimTensor(tensor.tensorHandle, dim, keepDim))
|
||||
|
||||
public fun StructureND<T>.maxAll(): TensorType = wrap(JNoa.maxTensor(tensor.tensorHandle))
|
||||
override fun StructureND<T>.max(): T = maxAll().item()
|
||||
override fun StructureND<T>.max(dim: Int, keepDim: Boolean): TensorType =
|
||||
wrap(JNoa.maxDimTensor(tensor.tensorHandle, dim, keepDim))
|
||||
|
||||
override fun StructureND<T>.argMax(dim: Int, keepDim: Boolean): NoaIntTensor =
|
||||
NoaIntTensor(scope, JNoa.argMaxTensor(tensor.tensorHandle, dim, keepDim))
|
||||
|
||||
public fun Tensor<T>.flatten(startDim: Int, endDim: Int): TensorType =
|
||||
wrap(JNoa.flattenTensor(tensor.tensorHandle, startDim, endDim))
|
||||
|
||||
public fun Tensor<T>.randDiscrete(low: Long, high: Long): TensorType =
|
||||
wrap(JNoa.randintLike(tensor.tensorHandle, low, high))
|
||||
|
||||
public fun Tensor<T>.randDiscreteAssign(low: Long, high: Long): Unit =
|
||||
JNoa.randintLikeAssign(tensor.tensorHandle, low, high)
|
||||
|
||||
public fun Tensor<T>.copy(): TensorType =
|
||||
wrap(JNoa.copyTensor(tensor.tensorHandle))
|
||||
|
||||
public fun Tensor<T>.copyToDevice(device: Device = Device.CPU): TensorType =
|
||||
wrap(JNoa.copyToDevice(tensor.tensorHandle, device.toInt()))
|
||||
|
||||
public abstract fun loadJitModule(path: String, device: Device = Device.CPU): NoaJitModule
|
||||
|
||||
public abstract fun loadTensor(path: String, device: Device = Device.CPU): TensorType
|
||||
|
||||
public fun NoaJitModule.forward(features: Tensor<T>): TensorType =
|
||||
wrap(JNoa.forwardPass(jitModuleHandle, features.tensor.tensorHandle))
|
||||
|
||||
public fun NoaJitModule.forwardAssign(features: TensorType, predictions: TensorType): Unit =
|
||||
JNoa.forwardPassAssign(jitModuleHandle, features.tensorHandle, predictions.tensorHandle)
|
||||
|
||||
public fun NoaJitModule.getParameter(name: String): TensorType =
|
||||
wrap(JNoa.getModuleParameter(jitModuleHandle, name))
|
||||
|
||||
public fun NoaJitModule.setParameter(name: String, parameter: Tensor<T>): Unit =
|
||||
JNoa.setModuleParameter(jitModuleHandle, name, parameter.tensor.tensorHandle)
|
||||
|
||||
public fun NoaJitModule.getBuffer(name: String): TensorType =
|
||||
wrap(JNoa.getModuleBuffer(jitModuleHandle, name))
|
||||
|
||||
public fun NoaJitModule.setBuffer(name: String, buffer: Tensor<T>): Unit =
|
||||
JNoa.setModuleBuffer(jitModuleHandle, name, buffer.tensor.tensorHandle)
|
||||
|
||||
public infix fun TensorType.swap(arg: TensorType): Unit =
|
||||
JNoa.swapTensors(tensorHandle, arg.tensorHandle)
|
||||
|
||||
public abstract fun TensorType.assignFromArray(array: PrimitiveArray): Unit
|
||||
|
||||
}
|
||||
|
||||
public sealed class NoaPartialDivisionAlgebra<T, A : Field<T>, PrimitiveArray, TensorType : NoaTensor<T>>
|
||||
protected constructor(scope: NoaScope) :
|
||||
NoaAlgebra<T, A, PrimitiveArray, TensorType>(scope),
|
||||
LinearOpsTensorAlgebra<T, A>,
|
||||
AnalyticTensorAlgebra<T, A> {
|
||||
|
||||
override operator fun StructureND<T>.div(arg: StructureND<T>): TensorType {
|
||||
return wrap(JNoa.divTensor(tensor.tensorHandle, arg.tensor.tensorHandle))
|
||||
}
|
||||
|
||||
override operator fun Tensor<T>.divAssign(arg: StructureND<T>): Unit {
|
||||
JNoa.divTensorAssign(tensor.tensorHandle, arg.tensor.tensorHandle)
|
||||
}
|
||||
|
||||
public fun StructureND<T>.meanAll(): TensorType = wrap(JNoa.meanTensor(tensor.tensorHandle))
|
||||
override fun StructureND<T>.mean(): T = meanAll().item()
|
||||
override fun StructureND<T>.mean(dim: Int, keepDim: Boolean): TensorType =
|
||||
wrap(JNoa.meanDimTensor(tensor.tensorHandle, dim, keepDim))
|
||||
|
||||
public fun StructureND<T>.stdAll(): TensorType = wrap(JNoa.stdTensor(tensor.tensorHandle))
|
||||
override fun StructureND<T>.std(): T = stdAll().item()
|
||||
override fun StructureND<T>.std(dim: Int, keepDim: Boolean): TensorType =
|
||||
wrap(JNoa.stdDimTensor(tensor.tensorHandle, dim, keepDim))
|
||||
|
||||
public fun StructureND<T>.varAll(): TensorType = wrap(JNoa.varTensor(tensor.tensorHandle))
|
||||
override fun StructureND<T>.variance(): T = varAll().item()
|
||||
override fun StructureND<T>.variance(dim: Int, keepDim: Boolean): TensorType =
|
||||
wrap(JNoa.varDimTensor(tensor.tensorHandle, dim, keepDim))
|
||||
|
||||
public abstract fun randNormal(shape: IntArray, device: Device = Device.CPU): TensorType
|
||||
|
||||
public abstract fun randUniform(shape: IntArray, device: Device = Device.CPU): TensorType
|
||||
|
||||
public fun StructureND<T>.randUniform(): TensorType =
|
||||
wrap(JNoa.randLike(tensor.tensorHandle))
|
||||
|
||||
public fun StructureND<T>.randUniformAssign(): Unit =
|
||||
JNoa.randLikeAssign(tensor.tensorHandle)
|
||||
|
||||
public fun StructureND<T>.randNormal(): TensorType =
|
||||
wrap(JNoa.randnLike(tensor.tensorHandle))
|
||||
|
||||
public fun StructureND<T>.randNormalAssign(): Unit =
|
||||
JNoa.randnLikeAssign(tensor.tensorHandle)
|
||||
|
||||
override fun StructureND<T>.exp(): TensorType =
|
||||
wrap(JNoa.expTensor(tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<T>.ln(): TensorType =
|
||||
wrap(JNoa.lnTensor(tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<T>.sqrt(): TensorType =
|
||||
wrap(JNoa.sqrtTensor(tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<T>.cos(): TensorType =
|
||||
wrap(JNoa.cosTensor(tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<T>.acos(): TensorType =
|
||||
wrap(JNoa.acosTensor(tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<T>.cosh(): TensorType =
|
||||
wrap(JNoa.coshTensor(tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<T>.acosh(): TensorType =
|
||||
wrap(JNoa.acoshTensor(tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<T>.sin(): TensorType =
|
||||
wrap(JNoa.sinTensor(tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<T>.asin(): TensorType =
|
||||
wrap(JNoa.asinTensor(tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<T>.sinh(): TensorType =
|
||||
wrap(JNoa.sinhTensor(tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<T>.asinh(): TensorType =
|
||||
wrap(JNoa.asinhTensor(tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<T>.tan(): TensorType =
|
||||
wrap(JNoa.tanTensor(tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<T>.atan(): TensorType =
|
||||
wrap(JNoa.atanTensor(tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<T>.tanh(): TensorType =
|
||||
wrap(JNoa.tanhTensor(tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<T>.atanh(): TensorType =
|
||||
wrap(JNoa.atanhTensor(tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<T>.ceil(): TensorType =
|
||||
wrap(JNoa.ceilTensor(tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<T>.floor(): TensorType =
|
||||
wrap(JNoa.floorTensor(tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<T>.det(): Tensor<T> =
|
||||
wrap(JNoa.detTensor(tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<T>.inv(): Tensor<T> =
|
||||
wrap(JNoa.invTensor(tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<T>.cholesky(): Tensor<T> =
|
||||
wrap(JNoa.choleskyTensor(tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<T>.qr(): Pair<TensorType, TensorType> {
|
||||
val Q = JNoa.emptyTensor()
|
||||
val R = JNoa.emptyTensor()
|
||||
JNoa.qrTensor(tensor.tensorHandle, Q, R)
|
||||
return Pair(wrap(Q), wrap(R))
|
||||
}
|
||||
|
||||
/**
|
||||
* this implementation satisfies `tensor = P dot L dot U`
|
||||
*/
|
||||
override fun StructureND<T>.lu(): Triple<TensorType, TensorType, TensorType> {
|
||||
val P = JNoa.emptyTensor()
|
||||
val L = JNoa.emptyTensor()
|
||||
val U = JNoa.emptyTensor()
|
||||
JNoa.luTensor(tensor.tensorHandle, P, L, U)
|
||||
return Triple(wrap(P), wrap(L), wrap(U))
|
||||
}
|
||||
|
||||
override fun StructureND<T>.svd(): Triple<TensorType, TensorType, TensorType> {
|
||||
val U = JNoa.emptyTensor()
|
||||
val V = JNoa.emptyTensor()
|
||||
val S = JNoa.emptyTensor()
|
||||
JNoa.svdTensor(tensor.tensorHandle, U, S, V)
|
||||
return Triple(wrap(U), wrap(S), wrap(V))
|
||||
}
|
||||
|
||||
override fun StructureND<T>.symEig(): Pair<TensorType, TensorType> {
|
||||
val V = JNoa.emptyTensor()
|
||||
val S = JNoa.emptyTensor()
|
||||
JNoa.symEigTensor(tensor.tensorHandle, S, V)
|
||||
return Pair(wrap(S), wrap(V))
|
||||
}
|
||||
|
||||
public fun TensorType.autoGradient(variable: TensorType, retainGraph: Boolean = false): TensorType =
|
||||
wrap(JNoa.autoGradTensor(tensorHandle, variable.tensorHandle, retainGraph))
|
||||
|
||||
public fun TensorType.autoHessian(variable: TensorType): TensorType =
|
||||
wrap(JNoa.autoHessTensor(tensorHandle, variable.tensorHandle))
|
||||
|
||||
public fun TensorType.detachFromGraph(): TensorType =
|
||||
wrap(JNoa.detachFromGraph(tensorHandle))
|
||||
|
||||
public fun TensorType.backward(): Unit =
|
||||
JNoa.backwardPass(tensorHandle)
|
||||
|
||||
public fun TensorType.grad(): TensorType =
|
||||
wrap(JNoa.tensorGrad(tensorHandle))
|
||||
|
||||
public fun NoaJitModule.train(status: Boolean): Unit =
|
||||
JNoa.trainMode(jitModuleHandle, status)
|
||||
|
||||
public fun NoaJitModule.adamOptimiser(learningRate: Double): AdamOptimiser =
|
||||
AdamOptimiser(scope, JNoa.adamOptim(jitModuleHandle, learningRate))
|
||||
|
||||
/**
|
||||
* Implements RMSprop algorithm. Receive `learning rate`, `alpha` (smoothing constant),
|
||||
* `eps` (term added to the denominator to improve numerical stability), `weight_decay`,
|
||||
* `momentum` factor, `centered` (if True, compute the centered RMSProp).
|
||||
* For more information: https://pytorch.org/docs/stable/generated/torch.optim.RMSprop.html
|
||||
*
|
||||
* @receiver the `learning rate`, `alpha`, `eps`, `weight_decay`, `momentum`, `centered`.
|
||||
* @return RMSpropOptimiser.
|
||||
*/
|
||||
public fun NoaJitModule.rmsOptimiser(learningRate: Double, alpha: Double,
|
||||
eps: Double, weightDecay: Double, momentum: Double, centered: Boolean): RMSpropOptimiser =
|
||||
RMSpropOptimiser(scope, JNoa.rmsOptim(jitModuleHandle, learningRate, alpha,
|
||||
eps, weightDecay, momentum, centered))
|
||||
|
||||
/**
|
||||
* Implements AdamW algorithm. Receive `learning rate`, `beta1` and `beta2` (coefficients used
|
||||
* for computing running averages of gradient and its square), `eps` (term added to the denominator
|
||||
* to improve numerical stability), `weight_decay`, `amsgrad`.
|
||||
* For more information: https://pytorch.org/docs/stable/generated/torch.optim.AdamW.html
|
||||
*
|
||||
* @receiver the `learning rate`, `beta1`, `beta2`, `eps`, `weight_decay`, `amsgrad`.
|
||||
* @return AdamWOptimiser.
|
||||
*/
|
||||
public fun NoaJitModule.adamWOptimiser(learningRate: Double, beta1: Double,
|
||||
beta2: Double, eps: Double, weightDecay: Double, amsgrad: Boolean): AdamWOptimiser =
|
||||
AdamWOptimiser(scope, JNoa.adamWOptim(jitModuleHandle, learningRate, beta1,
|
||||
beta2, eps, weightDecay, amsgrad))
|
||||
|
||||
/**
|
||||
* Implements Adagrad algorithm. Receive `learning rate`, `weight_decay`,
|
||||
* `learning rate decay`, `initial accumulator value`, `eps`.
|
||||
* For more information: https://pytorch.org/docs/stable/generated/torch.optim.Adagrad.html
|
||||
*
|
||||
* @receiver the `learning rate`, `weight_decay`, `learning rate decay`, `initial accumulator value`, `eps`.
|
||||
* @return AdagradOptimiser.
|
||||
*/
|
||||
public fun NoaJitModule.adagradOptimiser(learningRate: Double, weightDecay: Double,
|
||||
lrDecay: Double, initialAccumulatorValue: Double, eps: Double): AdagradOptimiser =
|
||||
AdagradOptimiser(scope, JNoa.adagradOptim(jitModuleHandle, learningRate, weightDecay,
|
||||
lrDecay, initialAccumulatorValue, eps))
|
||||
|
||||
/**
|
||||
* Implements stochastic gradient descent. Receive `learning rate`, `momentum` factor,
|
||||
* `dampening` for momentum, `weight_decay`, `nesterov` (enables Nesterov momentum).
|
||||
* For more information: https://pytorch.org/docs/stable/generated/torch.optim.SGD.html
|
||||
*
|
||||
* @receiver the `learning rate`, `momentum`, `dampening`, `weight_decay`, `nesterov`.
|
||||
* @return SgdOptimiser.
|
||||
*/
|
||||
public fun NoaJitModule.sgdOptimiser(learningRate: Double, momentum: Double,
|
||||
dampening: Double, weightDecay: Double, nesterov: Boolean): SgdOptimiser =
|
||||
SgdOptimiser(scope, JNoa.sgdOptim(jitModuleHandle, learningRate, momentum,
|
||||
dampening, weightDecay, nesterov))
|
||||
}
|
||||
|
||||
public sealed class NoaDoubleAlgebra
|
||||
protected constructor(scope: NoaScope) :
|
||||
NoaPartialDivisionAlgebra<Double, DoubleField, DoubleArray, NoaDoubleTensor>(scope) {
|
||||
|
||||
override val elementAlgebra: DoubleField
|
||||
get() = DoubleField
|
||||
|
||||
override fun structureND(shape: IntArray, initializer: DoubleField.(IntArray) -> Double): NoaDoubleTensor =
|
||||
copyFromArray(
|
||||
TensorLinearStructure(shape).asSequence().map { DoubleField.initializer(it) }.toMutableList()
|
||||
.toDoubleArray(),
|
||||
shape, Device.CPU
|
||||
)
|
||||
|
||||
private fun StructureND<Double>.castHelper(): NoaDoubleTensor =
|
||||
copyFromArray(
|
||||
TensorLinearStructure(this.shape).asSequence().map(this::get).toMutableList().toDoubleArray(),
|
||||
this.shape, Device.CPU
|
||||
)
|
||||
|
||||
override val StructureND<Double>.tensor: NoaDoubleTensor
|
||||
get() = when (this) {
|
||||
is NoaDoubleTensor -> this
|
||||
else -> castHelper()
|
||||
}
|
||||
|
||||
override fun wrap(tensorHandle: TensorHandle): NoaDoubleTensor =
|
||||
NoaDoubleTensor(scope = scope, tensorHandle = tensorHandle)
|
||||
|
||||
override fun NoaDoubleTensor.copyToArray(): DoubleArray {
|
||||
val array = DoubleArray(numElements)
|
||||
JNoa.getBlobDouble(tensorHandle, array)
|
||||
return array
|
||||
}
|
||||
|
||||
override fun copyFromArray(array: DoubleArray, shape: IntArray, device: Device): NoaDoubleTensor =
|
||||
wrap(JNoa.fromBlobDouble(array, shape, device.toInt()))
|
||||
|
||||
override fun randNormal(shape: IntArray, device: Device): NoaDoubleTensor =
|
||||
wrap(JNoa.randnDouble(shape, device.toInt()))
|
||||
|
||||
override fun randUniform(shape: IntArray, device: Device): NoaDoubleTensor =
|
||||
wrap(JNoa.randDouble(shape, device.toInt()))
|
||||
|
||||
override fun randDiscrete(low: Long, high: Long, shape: IntArray, device: Device): NoaDoubleTensor =
|
||||
wrap(JNoa.randintDouble(low, high, shape, device.toInt()))
|
||||
|
||||
override operator fun Double.plus(arg: StructureND<Double>): NoaDoubleTensor =
|
||||
wrap(JNoa.plusDouble(this, arg.tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<Double>.plus(value: Double): NoaDoubleTensor =
|
||||
wrap(JNoa.plusDouble(value, tensor.tensorHandle))
|
||||
|
||||
override fun Tensor<Double>.plusAssign(value: Double): Unit =
|
||||
JNoa.plusDoubleAssign(value, tensor.tensorHandle)
|
||||
|
||||
override operator fun Double.minus(arg: StructureND<Double>): NoaDoubleTensor =
|
||||
wrap(JNoa.plusDouble(-this, arg.tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<Double>.minus(value: Double): NoaDoubleTensor =
|
||||
wrap(JNoa.plusDouble(-value, tensor.tensorHandle))
|
||||
|
||||
override fun Tensor<Double>.minusAssign(value: Double): Unit =
|
||||
JNoa.plusDoubleAssign(-value, tensor.tensorHandle)
|
||||
|
||||
override operator fun Double.times(arg: StructureND<Double>): NoaDoubleTensor =
|
||||
wrap(JNoa.timesDouble(this, arg.tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<Double>.times(value: Double): NoaDoubleTensor =
|
||||
wrap(JNoa.timesDouble(value, tensor.tensorHandle))
|
||||
|
||||
override fun Tensor<Double>.timesAssign(value: Double): Unit =
|
||||
JNoa.timesDoubleAssign(value, tensor.tensorHandle)
|
||||
|
||||
override fun Double.div(arg: StructureND<Double>): NoaDoubleTensor =
|
||||
arg.tensor * (1 / this)
|
||||
|
||||
override fun StructureND<Double>.div(value: Double): NoaDoubleTensor =
|
||||
tensor * (1 / value)
|
||||
|
||||
override fun Tensor<Double>.divAssign(value: Double): Unit =
|
||||
tensor.timesAssign(1 / value)
|
||||
|
||||
override fun full(value: Double, shape: IntArray, device: Device): NoaDoubleTensor =
|
||||
wrap(JNoa.fullDouble(value, shape, device.toInt()))
|
||||
|
||||
override fun loadJitModule(path: String, device: Device): NoaJitModule =
|
||||
NoaJitModule(scope, JNoa.loadJitModuleDouble(path, device.toInt()))
|
||||
|
||||
override fun loadTensor(path: String, device: Device): NoaDoubleTensor =
|
||||
wrap(JNoa.loadTensorDouble(path, device.toInt()))
|
||||
|
||||
override fun NoaDoubleTensor.assignFromArray(array: DoubleArray): Unit =
|
||||
JNoa.assignBlobDouble(tensorHandle, array)
|
||||
|
||||
override fun NoaDoubleTensor.set(i: Int, array: DoubleArray): Unit =
|
||||
JNoa.setBlobDouble(tensorHandle, i, array)
|
||||
|
||||
override fun NoaDoubleTensor.set(dim: Int, slice: Slice, array: DoubleArray): Unit =
|
||||
JNoa.setSliceBlobDouble(tensorHandle, dim, slice.first, slice.second, array)
|
||||
}
|
||||
|
||||
public sealed class NoaFloatAlgebra
|
||||
protected constructor(scope: NoaScope) :
|
||||
NoaPartialDivisionAlgebra<Float, FloatField, FloatArray, NoaFloatTensor>(scope) {
|
||||
|
||||
override val elementAlgebra: FloatField
|
||||
get() = FloatField
|
||||
|
||||
override fun structureND(shape: IntArray, initializer: FloatField.(IntArray) -> Float): NoaFloatTensor =
|
||||
copyFromArray(
|
||||
TensorLinearStructure(shape).asSequence().map { FloatField.initializer(it) }.toMutableList()
|
||||
.toFloatArray(),
|
||||
shape, Device.CPU
|
||||
)
|
||||
|
||||
private fun StructureND<Float>.castHelper(): NoaFloatTensor =
|
||||
copyFromArray(
|
||||
TensorLinearStructure(this.shape).asSequence().map(this::get).toMutableList().toFloatArray(),
|
||||
this.shape, Device.CPU
|
||||
)
|
||||
|
||||
override val StructureND<Float>.tensor: NoaFloatTensor
|
||||
get() = when (this) {
|
||||
is NoaFloatTensor -> this
|
||||
else -> castHelper()
|
||||
}
|
||||
|
||||
override fun wrap(tensorHandle: TensorHandle): NoaFloatTensor =
|
||||
NoaFloatTensor(scope = scope, tensorHandle = tensorHandle)
|
||||
|
||||
override fun NoaFloatTensor.copyToArray(): FloatArray {
|
||||
val res = FloatArray(numElements)
|
||||
JNoa.getBlobFloat(tensorHandle, res)
|
||||
return res
|
||||
}
|
||||
|
||||
override fun copyFromArray(array: FloatArray, shape: IntArray, device: Device): NoaFloatTensor =
|
||||
wrap(JNoa.fromBlobFloat(array, shape, device.toInt()))
|
||||
|
||||
override fun randNormal(shape: IntArray, device: Device): NoaFloatTensor =
|
||||
wrap(JNoa.randnFloat(shape, device.toInt()))
|
||||
|
||||
override fun randUniform(shape: IntArray, device: Device): NoaFloatTensor =
|
||||
wrap(JNoa.randFloat(shape, device.toInt()))
|
||||
|
||||
override fun randDiscrete(low: Long, high: Long, shape: IntArray, device: Device): NoaFloatTensor =
|
||||
wrap(JNoa.randintFloat(low, high, shape, device.toInt()))
|
||||
|
||||
override operator fun Float.plus(arg: StructureND<Float>): NoaFloatTensor =
|
||||
wrap(JNoa.plusFloat(this, arg.tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<Float>.plus(value: Float): NoaFloatTensor =
|
||||
wrap(JNoa.plusFloat(value, tensor.tensorHandle))
|
||||
|
||||
override fun Tensor<Float>.plusAssign(value: Float): Unit =
|
||||
JNoa.plusFloatAssign(value, tensor.tensorHandle)
|
||||
|
||||
override operator fun Float.minus(arg: StructureND<Float>): NoaFloatTensor =
|
||||
wrap(JNoa.plusFloat(-this, arg.tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<Float>.minus(value: Float): NoaFloatTensor =
|
||||
wrap(JNoa.plusFloat(-value, tensor.tensorHandle))
|
||||
|
||||
override fun Tensor<Float>.minusAssign(value: Float): Unit =
|
||||
JNoa.plusFloatAssign(-value, tensor.tensorHandle)
|
||||
|
||||
override operator fun Float.times(arg: StructureND<Float>): NoaFloatTensor =
|
||||
wrap(JNoa.timesFloat(this, arg.tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<Float>.times(value: Float): NoaFloatTensor =
|
||||
wrap(JNoa.timesFloat(value, tensor.tensorHandle))
|
||||
|
||||
override fun Tensor<Float>.timesAssign(value: Float): Unit =
|
||||
JNoa.timesFloatAssign(value, tensor.tensorHandle)
|
||||
|
||||
override fun Float.div(arg: StructureND<Float>): NoaFloatTensor =
|
||||
arg.tensor * (1 / this)
|
||||
|
||||
override fun StructureND<Float>.div(value: Float): NoaFloatTensor =
|
||||
tensor * (1 / value)
|
||||
|
||||
override fun Tensor<Float>.divAssign(value: Float): Unit =
|
||||
tensor.timesAssign(1 / value)
|
||||
|
||||
override fun full(value: Float, shape: IntArray, device: Device): NoaFloatTensor =
|
||||
wrap(JNoa.fullFloat(value, shape, device.toInt()))
|
||||
|
||||
override fun loadJitModule(path: String, device: Device): NoaJitModule =
|
||||
NoaJitModule(scope, JNoa.loadJitModuleFloat(path, device.toInt()))
|
||||
|
||||
override fun loadTensor(path: String, device: Device): NoaFloatTensor =
|
||||
wrap(JNoa.loadTensorFloat(path, device.toInt()))
|
||||
|
||||
override fun NoaFloatTensor.assignFromArray(array: FloatArray): Unit =
|
||||
JNoa.assignBlobFloat(tensorHandle, array)
|
||||
|
||||
override fun NoaFloatTensor.set(i: Int, array: FloatArray): Unit =
|
||||
JNoa.setBlobFloat(tensorHandle, i, array)
|
||||
|
||||
override fun NoaFloatTensor.set(dim: Int, slice: Slice, array: FloatArray): Unit =
|
||||
JNoa.setSliceBlobFloat(tensorHandle, dim, slice.first, slice.second, array)
|
||||
|
||||
}
|
||||
|
||||
public sealed class NoaLongAlgebra
|
||||
protected constructor(scope: NoaScope) :
|
||||
NoaAlgebra<Long, LongRing, LongArray, NoaLongTensor>(scope) {
|
||||
|
||||
override val elementAlgebra: LongRing
|
||||
get() = LongRing
|
||||
|
||||
override fun structureND(shape: IntArray, initializer: LongRing.(IntArray) -> Long): NoaLongTensor =
|
||||
copyFromArray(
|
||||
TensorLinearStructure(shape).asSequence().map { LongRing.initializer(it) }.toMutableList()
|
||||
.toLongArray(),
|
||||
shape, Device.CPU
|
||||
)
|
||||
|
||||
private fun StructureND<Long>.castHelper(): NoaLongTensor =
|
||||
copyFromArray(
|
||||
TensorLinearStructure(this.shape).asSequence().map(this::get).toMutableList().toLongArray(),
|
||||
this.shape, Device.CPU
|
||||
)
|
||||
|
||||
override val StructureND<Long>.tensor: NoaLongTensor
|
||||
get() = when (this) {
|
||||
is NoaLongTensor -> this
|
||||
else -> castHelper()
|
||||
}
|
||||
|
||||
override fun wrap(tensorHandle: TensorHandle): NoaLongTensor =
|
||||
NoaLongTensor(scope = scope, tensorHandle = tensorHandle)
|
||||
|
||||
override fun NoaLongTensor.copyToArray(): LongArray {
|
||||
val array = LongArray(numElements)
|
||||
JNoa.getBlobLong(tensorHandle, array)
|
||||
return array
|
||||
}
|
||||
|
||||
override fun copyFromArray(array: LongArray, shape: IntArray, device: Device): NoaLongTensor =
|
||||
wrap(JNoa.fromBlobLong(array, shape, device.toInt()))
|
||||
|
||||
override fun randDiscrete(low: Long, high: Long, shape: IntArray, device: Device): NoaLongTensor =
|
||||
wrap(JNoa.randintLong(low, high, shape, device.toInt()))
|
||||
|
||||
override operator fun Long.plus(arg: StructureND<Long>): NoaLongTensor =
|
||||
wrap(JNoa.plusLong(this, arg.tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<Long>.plus(value: Long): NoaLongTensor =
|
||||
wrap(JNoa.plusLong(value, tensor.tensorHandle))
|
||||
|
||||
override fun Tensor<Long>.plusAssign(value: Long): Unit =
|
||||
JNoa.plusLongAssign(value, tensor.tensorHandle)
|
||||
|
||||
override operator fun Long.minus(arg: StructureND<Long>): NoaLongTensor =
|
||||
wrap(JNoa.plusLong(-this, arg.tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<Long>.minus(value: Long): NoaLongTensor =
|
||||
wrap(JNoa.plusLong(-value, tensor.tensorHandle))
|
||||
|
||||
override fun Tensor<Long>.minusAssign(value: Long): Unit =
|
||||
JNoa.plusLongAssign(-value, tensor.tensorHandle)
|
||||
|
||||
override operator fun Long.times(arg: StructureND<Long>): NoaLongTensor =
|
||||
wrap(JNoa.timesLong(this, arg.tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<Long>.times(value: Long): NoaLongTensor =
|
||||
wrap(JNoa.timesLong(value, tensor.tensorHandle))
|
||||
|
||||
override fun Tensor<Long>.timesAssign(value: Long): Unit =
|
||||
JNoa.timesLongAssign(value, tensor.tensorHandle)
|
||||
|
||||
override fun full(value: Long, shape: IntArray, device: Device): NoaLongTensor =
|
||||
wrap(JNoa.fullLong(value, shape, device.toInt()))
|
||||
|
||||
override fun loadJitModule(path: String, device: Device): NoaJitModule =
|
||||
NoaJitModule(scope, JNoa.loadJitModuleLong(path, device.toInt()))
|
||||
|
||||
override fun loadTensor(path: String, device: Device): NoaLongTensor =
|
||||
wrap(JNoa.loadTensorLong(path, device.toInt()))
|
||||
|
||||
override fun NoaLongTensor.assignFromArray(array: LongArray): Unit =
|
||||
JNoa.assignBlobLong(tensorHandle, array)
|
||||
|
||||
override fun NoaLongTensor.set(i: Int, array: LongArray): Unit =
|
||||
JNoa.setBlobLong(tensorHandle, i, array)
|
||||
|
||||
override fun NoaLongTensor.set(dim: Int, slice: Slice, array: LongArray): Unit =
|
||||
JNoa.setSliceBlobLong(tensorHandle, dim, slice.first, slice.second, array)
|
||||
}
|
||||
|
||||
public sealed class NoaIntAlgebra
|
||||
protected constructor(scope: NoaScope) :
|
||||
NoaAlgebra<Int, IntRing, IntArray, NoaIntTensor>(scope) {
|
||||
|
||||
override val elementAlgebra: IntRing
|
||||
get() = IntRing
|
||||
|
||||
override fun structureND(shape: IntArray, initializer: IntRing.(IntArray) -> Int): NoaIntTensor =
|
||||
copyFromArray(
|
||||
TensorLinearStructure(shape).asSequence().map { IntRing.initializer(it) }.toMutableList()
|
||||
.toIntArray(),
|
||||
shape, Device.CPU
|
||||
)
|
||||
|
||||
private fun StructureND<Int>.castHelper(): NoaIntTensor =
|
||||
copyFromArray(
|
||||
TensorLinearStructure(this.shape).asSequence().map(this::get).toMutableList().toIntArray(),
|
||||
this.shape, Device.CPU
|
||||
)
|
||||
|
||||
override val StructureND<Int>.tensor: NoaIntTensor
|
||||
get() = when (this) {
|
||||
is NoaIntTensor -> this
|
||||
else -> castHelper()
|
||||
}
|
||||
|
||||
override fun wrap(tensorHandle: TensorHandle): NoaIntTensor =
|
||||
NoaIntTensor(scope = scope, tensorHandle = tensorHandle)
|
||||
|
||||
override fun NoaIntTensor.copyToArray(): IntArray {
|
||||
val array = IntArray(numElements)
|
||||
JNoa.getBlobInt(tensorHandle, array)
|
||||
return array
|
||||
}
|
||||
|
||||
override fun copyFromArray(array: IntArray, shape: IntArray, device: Device): NoaIntTensor =
|
||||
wrap(JNoa.fromBlobInt(array, shape, device.toInt()))
|
||||
|
||||
override fun randDiscrete(low: Long, high: Long, shape: IntArray, device: Device): NoaIntTensor =
|
||||
wrap(JNoa.randintInt(low, high, shape, device.toInt()))
|
||||
|
||||
override operator fun Int.plus(arg: StructureND<Int>): NoaIntTensor =
|
||||
wrap(JNoa.plusInt(this, arg.tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<Int>.plus(value: Int): NoaIntTensor =
|
||||
wrap(JNoa.plusInt(value, tensor.tensorHandle))
|
||||
|
||||
override fun Tensor<Int>.plusAssign(value: Int): Unit =
|
||||
JNoa.plusIntAssign(value, tensor.tensorHandle)
|
||||
|
||||
override operator fun Int.minus(arg: StructureND<Int>): NoaIntTensor =
|
||||
wrap(JNoa.plusInt(-this, arg.tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<Int>.minus(value: Int): NoaIntTensor =
|
||||
wrap(JNoa.plusInt(-value, tensor.tensorHandle))
|
||||
|
||||
override fun Tensor<Int>.minusAssign(value: Int): Unit =
|
||||
JNoa.plusIntAssign(-value, tensor.tensorHandle)
|
||||
|
||||
override operator fun Int.times(arg: StructureND<Int>): NoaIntTensor =
|
||||
wrap(JNoa.timesInt(this, arg.tensor.tensorHandle))
|
||||
|
||||
override fun StructureND<Int>.times(value: Int): NoaIntTensor =
|
||||
wrap(JNoa.timesInt(value, tensor.tensorHandle))
|
||||
|
||||
override fun Tensor<Int>.timesAssign(value: Int): Unit =
|
||||
JNoa.timesIntAssign(value, tensor.tensorHandle)
|
||||
|
||||
override fun full(value: Int, shape: IntArray, device: Device): NoaIntTensor =
|
||||
wrap(JNoa.fullInt(value, shape, device.toInt()))
|
||||
|
||||
override fun loadJitModule(path: String, device: Device): NoaJitModule =
|
||||
NoaJitModule(scope, JNoa.loadJitModuleInt(path, device.toInt()))
|
||||
|
||||
override fun loadTensor(path: String, device: Device): NoaIntTensor =
|
||||
wrap(JNoa.loadTensorInt(path, device.toInt()))
|
||||
|
||||
override fun NoaIntTensor.assignFromArray(array: IntArray): Unit =
|
||||
JNoa.assignBlobInt(tensorHandle, array)
|
||||
|
||||
override fun NoaIntTensor.set(i: Int, array: IntArray): Unit =
|
||||
JNoa.setBlobInt(tensorHandle, i, array)
|
||||
|
||||
override fun NoaIntTensor.set(dim: Int, slice: Slice, array: IntArray): Unit =
|
||||
JNoa.setSliceBlobInt(tensorHandle, dim, slice.first, slice.second, array)
|
||||
}
|
49
kmath-noa/src/main/kotlin/space/kscience/kmath/noa/api.kt
Normal file
49
kmath-noa/src/main/kotlin/space/kscience/kmath/noa/api.kt
Normal file
@ -0,0 +1,49 @@
|
||||
/*
|
||||
* Copyright 2018-2021 KMath contributors.
|
||||
* Use of this source code is governed by the Apache 2.0 license that can be found in the license/LICENSE.txt file.
|
||||
*/
|
||||
|
||||
package space.kscience.kmath.noa
|
||||
|
||||
import space.kscience.kmath.noa.memory.NoaScope
|
||||
import space.kscience.kmath.noa.memory.withNoaScope
|
||||
|
||||
public class NoaDouble
|
||||
internal constructor(scope: NoaScope) :
|
||||
NoaDoubleAlgebra(scope)
|
||||
|
||||
public fun <R> NoaDouble(block: NoaDouble.() -> R): R? =
|
||||
withNoaScope { NoaDouble(this).block() }
|
||||
|
||||
public fun <R> NoaDouble(scope: NoaScope, block: NoaDouble.() -> R): R? =
|
||||
withNoaScope(scope) { NoaDouble(this).block() }
|
||||
|
||||
public class NoaFloat
|
||||
internal constructor(scope: NoaScope) :
|
||||
NoaFloatAlgebra(scope)
|
||||
|
||||
public fun <R> NoaFloat(block: NoaFloat.() -> R): R? =
|
||||
withNoaScope { NoaFloat(this).block() }
|
||||
|
||||
public fun <R> NoaFloat(scope: NoaScope, block: NoaFloat.() -> R): R? =
|
||||
withNoaScope(scope) { NoaFloat(this).block() }
|
||||
|
||||
public class NoaLong
|
||||
internal constructor(scope: NoaScope) :
|
||||
NoaLongAlgebra(scope)
|
||||
|
||||
public fun <R> NoaLong(block: NoaLong.() -> R): R? =
|
||||
withNoaScope { NoaLong(this).block() }
|
||||
|
||||
public fun <R> NoaLong(scope: NoaScope, block: NoaLong.() -> R): R? =
|
||||
withNoaScope(scope) { NoaLong(this).block() }
|
||||
|
||||
public class NoaInt
|
||||
internal constructor(scope: NoaScope) :
|
||||
NoaIntAlgebra(scope)
|
||||
|
||||
public fun <R> NoaInt(block: NoaInt.() -> R): R? =
|
||||
withNoaScope { NoaInt(this).block() }
|
||||
|
||||
public fun <R> NoaInt(scope: NoaScope, block: NoaInt.() -> R): R? =
|
||||
withNoaScope(scope) { NoaInt(this).block() }
|
19
kmath-noa/src/main/kotlin/space/kscience/kmath/noa/jit.kt
Normal file
19
kmath-noa/src/main/kotlin/space/kscience/kmath/noa/jit.kt
Normal file
@ -0,0 +1,19 @@
|
||||
/*
|
||||
* Copyright 2018-2021 KMath contributors.
|
||||
* Use of this source code is governed by the Apache 2.0 license that can be found in the license/LICENSE.txt file.
|
||||
*/
|
||||
|
||||
package space.kscience.kmath.noa
|
||||
|
||||
import space.kscience.kmath.noa.memory.NoaResource
|
||||
import space.kscience.kmath.noa.memory.NoaScope
|
||||
|
||||
internal typealias JitModuleHandle = Long
|
||||
|
||||
public class NoaJitModule
|
||||
internal constructor(scope: NoaScope, internal val jitModuleHandle: JitModuleHandle)
|
||||
: NoaResource(scope){
|
||||
override fun dispose(): Unit = JNoa.disposeJitModule(jitModuleHandle)
|
||||
|
||||
public fun save(path: String): Unit = JNoa.saveJitModule(jitModuleHandle, path)
|
||||
}
|
@ -0,0 +1,15 @@
|
||||
/*
|
||||
* Copyright 2018-2021 KMath contributors.
|
||||
* Use of this source code is governed by the Apache 2.0 license that can be found in the license/LICENSE.txt file.
|
||||
*/
|
||||
|
||||
package space.kscience.kmath.noa.memory
|
||||
|
||||
public abstract class NoaResource
|
||||
internal constructor(internal val scope: NoaScope) {
|
||||
init {
|
||||
scope.add(::dispose)
|
||||
}
|
||||
|
||||
protected abstract fun dispose(): Unit
|
||||
}
|
@ -0,0 +1,49 @@
|
||||
/*
|
||||
* Copyright 2018-2021 KMath contributors.
|
||||
* Use of this source code is governed by the Apache 2.0 license that can be found in the license/LICENSE.txt file.
|
||||
*/
|
||||
|
||||
package space.kscience.kmath.noa.memory
|
||||
|
||||
private typealias Disposable = () -> Unit
|
||||
|
||||
public class NoaScope {
|
||||
|
||||
internal val disposables: ArrayDeque<Disposable> = ArrayDeque(0)
|
||||
|
||||
public fun disposeAll() {
|
||||
disposables.forEach(Disposable::invoke)
|
||||
disposables.clear()
|
||||
}
|
||||
|
||||
internal inline fun add(crossinline disposable: Disposable) {
|
||||
disposables += {
|
||||
try {
|
||||
disposable()
|
||||
} catch (e: Throwable) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
internal fun addAll(scope: NoaScope) {
|
||||
disposables.addAll(scope.disposables)
|
||||
}
|
||||
}
|
||||
|
||||
internal inline fun <R> withNoaScope(block: NoaScope.() -> R): R? {
|
||||
val noaScope = NoaScope()
|
||||
val result = try { noaScope.block() } catch (e: Throwable) { null }
|
||||
noaScope.disposeAll()
|
||||
return result
|
||||
}
|
||||
|
||||
internal inline fun <R> withNoaScope(scope: NoaScope, block: NoaScope.() -> R): R? {
|
||||
val noaScope = NoaScope()
|
||||
val result = try { noaScope.block() } catch (e: Throwable) { null }
|
||||
if (result == null){
|
||||
noaScope.disposeAll()
|
||||
} else {
|
||||
scope.addAll(noaScope)
|
||||
}
|
||||
return result
|
||||
}
|
58
kmath-noa/src/main/kotlin/space/kscience/kmath/noa/optim.kt
Normal file
58
kmath-noa/src/main/kotlin/space/kscience/kmath/noa/optim.kt
Normal file
@ -0,0 +1,58 @@
|
||||
/*
|
||||
* Copyright 2018-2021 KMath contributors.
|
||||
* Use of this source code is governed by the Apache 2.0 license that can be found in the license/LICENSE.txt file.
|
||||
*/
|
||||
|
||||
package space.kscience.kmath.noa
|
||||
|
||||
import space.kscience.kmath.noa.memory.NoaResource
|
||||
import space.kscience.kmath.noa.memory.NoaScope
|
||||
|
||||
internal typealias OptimiserHandle = Long
|
||||
|
||||
public abstract class NoaOptimiser
|
||||
internal constructor(scope: NoaScope) : NoaResource(scope) {
|
||||
public abstract fun step(): Unit
|
||||
public abstract fun zeroGrad(): Unit
|
||||
}
|
||||
|
||||
public class AdamOptimiser
|
||||
internal constructor(scope: NoaScope, internal val optimiserHandle: OptimiserHandle)
|
||||
: NoaOptimiser(scope) {
|
||||
override fun dispose(): Unit = JNoa.disposeAdamOptim(optimiserHandle)
|
||||
override fun step(): Unit = JNoa.stepAdamOptim(optimiserHandle)
|
||||
override fun zeroGrad(): Unit = JNoa.zeroGradAdamOptim(optimiserHandle)
|
||||
}
|
||||
|
||||
public class RMSpropOptimiser
|
||||
internal constructor(scope: NoaScope, internal val optimiserHandle: OptimiserHandle)
|
||||
: NoaOptimiser(scope) {
|
||||
override fun dispose(): Unit = JNoa.disposeRmsOptim(optimiserHandle)
|
||||
override fun step(): Unit = JNoa.stepRmsOptim(optimiserHandle)
|
||||
override fun zeroGrad(): Unit = JNoa.zeroGradRmsOptim(optimiserHandle)
|
||||
}
|
||||
|
||||
|
||||
public class AdamWOptimiser
|
||||
internal constructor(scope: NoaScope, internal val optimiserHandle: OptimiserHandle)
|
||||
: NoaOptimiser(scope) {
|
||||
override fun dispose(): Unit = JNoa.disposeAdamWOptim(optimiserHandle)
|
||||
override fun step(): Unit = JNoa.stepAdamWOptim(optimiserHandle)
|
||||
override fun zeroGrad(): Unit = JNoa.zeroGradAdamWOptim(optimiserHandle)
|
||||
}
|
||||
|
||||
public class AdagradOptimiser
|
||||
internal constructor(scope: NoaScope, internal val optimiserHandle: OptimiserHandle)
|
||||
: NoaOptimiser(scope) {
|
||||
override fun dispose(): Unit = JNoa.disposeAdagradOptim(optimiserHandle)
|
||||
override fun step(): Unit = JNoa.stepAdagradOptim(optimiserHandle)
|
||||
override fun zeroGrad(): Unit = JNoa.zeroGradAdagradOptim(optimiserHandle)
|
||||
}
|
||||
|
||||
public class SgdOptimiser
|
||||
internal constructor(scope: NoaScope, internal val optimiserHandle: OptimiserHandle)
|
||||
: NoaOptimiser(scope) {
|
||||
override fun dispose(): Unit = JNoa.disposeSgdOptim(optimiserHandle)
|
||||
override fun step(): Unit = JNoa.stepSgdOptim(optimiserHandle)
|
||||
override fun zeroGrad(): Unit = JNoa.zeroGradSgdOptim(optimiserHandle)
|
||||
}
|
159
kmath-noa/src/main/kotlin/space/kscience/kmath/noa/tensors.kt
Normal file
159
kmath-noa/src/main/kotlin/space/kscience/kmath/noa/tensors.kt
Normal file
@ -0,0 +1,159 @@
|
||||
/*
|
||||
* Copyright 2018-2021 KMath contributors.
|
||||
* Use of this source code is governed by the Apache 2.0 license that can be found in the license/LICENSE.txt file.
|
||||
*/
|
||||
|
||||
package space.kscience.kmath.noa
|
||||
|
||||
import space.kscience.kmath.misc.PerformancePitfall
|
||||
import space.kscience.kmath.noa.memory.NoaResource
|
||||
import space.kscience.kmath.noa.memory.NoaScope
|
||||
import space.kscience.kmath.tensors.api.Tensor
|
||||
import space.kscience.kmath.tensors.core.TensorLinearStructure
|
||||
|
||||
internal typealias TensorHandle = Long
|
||||
|
||||
public sealed class NoaTensor<T>
|
||||
protected constructor(scope: NoaScope, internal val tensorHandle: TensorHandle) :
|
||||
NoaResource(scope), Tensor<T> {
|
||||
|
||||
override fun dispose(): Unit = JNoa.disposeTensor(tensorHandle)
|
||||
|
||||
internal abstract fun item(): T
|
||||
|
||||
override val dimension: Int get() = JNoa.getDim(tensorHandle)
|
||||
|
||||
override val shape: IntArray
|
||||
get() = (1..dimension).map { JNoa.getShapeAt(tensorHandle, it - 1) }.toIntArray()
|
||||
|
||||
public val strides: IntArray
|
||||
get() = (1..dimension).map { JNoa.getStrideAt(tensorHandle, it - 1) }.toIntArray()
|
||||
|
||||
public val numElements: Int get() = JNoa.getNumel(tensorHandle)
|
||||
|
||||
public val device: Device get() = Device.fromInt(JNoa.getDevice(tensorHandle))
|
||||
|
||||
public fun save(path: String): Unit = JNoa.saveTensor(tensorHandle, path)
|
||||
|
||||
override fun toString(): String = JNoa.tensorToString(tensorHandle)
|
||||
|
||||
@PerformancePitfall
|
||||
override fun elements(): Sequence<Pair<IntArray, T>> {
|
||||
if (dimension == 0) {
|
||||
return emptySequence()
|
||||
}
|
||||
val indices = (1..numElements).asSequence().map {
|
||||
TensorLinearStructure.indexFromOffset(it - 1, strides, dimension)
|
||||
}
|
||||
return indices.map { it to get(it) }
|
||||
}
|
||||
|
||||
|
||||
public fun asDouble(): NoaDoubleTensor = NoaDoubleTensor(
|
||||
scope = scope,
|
||||
tensorHandle = JNoa.copyToDouble(this.tensorHandle)
|
||||
)
|
||||
|
||||
public fun asFloat(): NoaFloatTensor = NoaFloatTensor(
|
||||
scope = scope,
|
||||
tensorHandle = JNoa.copyToFloat(this.tensorHandle)
|
||||
)
|
||||
|
||||
public fun asLong(): NoaLongTensor = NoaLongTensor(
|
||||
scope = scope,
|
||||
tensorHandle = JNoa.copyToLong(this.tensorHandle)
|
||||
)
|
||||
|
||||
public fun asInt(): NoaIntTensor = NoaIntTensor(
|
||||
scope = scope,
|
||||
tensorHandle = JNoa.copyToInt(this.tensorHandle)
|
||||
)
|
||||
}
|
||||
|
||||
public sealed class NoaTensorOverField<T>
|
||||
protected constructor(scope: NoaScope, tensorHandle: Long) :
|
||||
NoaTensor<T>(scope, tensorHandle) {
|
||||
public var requiresGrad: Boolean
|
||||
get() = JNoa.requiresGrad(tensorHandle)
|
||||
set(value) = JNoa.setRequiresGrad(tensorHandle, value)
|
||||
}
|
||||
|
||||
|
||||
public class NoaDoubleTensor
|
||||
internal constructor(scope: NoaScope, tensorHandle: TensorHandle) :
|
||||
NoaTensorOverField<Double>(scope, tensorHandle) {
|
||||
|
||||
override fun item(): Double = JNoa.getItemDouble(tensorHandle)
|
||||
|
||||
override fun get(index: IntArray): Double = JNoa.getDouble(tensorHandle, index)
|
||||
|
||||
override fun set(index: IntArray, value: Double) {
|
||||
JNoa.setDouble(tensorHandle, index, value)
|
||||
}
|
||||
}
|
||||
|
||||
public class NoaFloatTensor
|
||||
internal constructor(scope: NoaScope, tensorHandle: TensorHandle) :
|
||||
NoaTensorOverField<Float>(scope, tensorHandle) {
|
||||
|
||||
override fun item(): Float = JNoa.getItemFloat(tensorHandle)
|
||||
|
||||
|
||||
override fun get(index: IntArray): Float = JNoa.getFloat(tensorHandle, index)
|
||||
|
||||
|
||||
override fun set(index: IntArray, value: Float) {
|
||||
JNoa.setFloat(tensorHandle, index, value)
|
||||
}
|
||||
}
|
||||
|
||||
public class NoaLongTensor
|
||||
internal constructor(scope: NoaScope, tensorHandle: TensorHandle) :
|
||||
NoaTensor<Long>(scope, tensorHandle) {
|
||||
|
||||
override fun item(): Long = JNoa.getItemLong(tensorHandle)
|
||||
|
||||
override fun get(index: IntArray): Long = JNoa.getLong(tensorHandle, index)
|
||||
|
||||
override fun set(index: IntArray, value: Long) {
|
||||
JNoa.setLong(tensorHandle, index, value)
|
||||
}
|
||||
}
|
||||
|
||||
public class NoaIntTensor
|
||||
internal constructor(scope: NoaScope, tensorHandle: TensorHandle) :
|
||||
NoaTensor<Int>(scope, tensorHandle) {
|
||||
|
||||
override fun item(): Int = JNoa.getItemInt(tensorHandle)
|
||||
|
||||
override fun get(index: IntArray): Int = JNoa.getInt(tensorHandle, index)
|
||||
|
||||
override fun set(index: IntArray, value: Int) {
|
||||
JNoa.setInt(tensorHandle, index, value)
|
||||
}
|
||||
}
|
||||
|
||||
public sealed class Device {
|
||||
public object CPU : Device() {
|
||||
override fun toString(): String {
|
||||
return "CPU"
|
||||
}
|
||||
}
|
||||
|
||||
public data class CUDA(val index: Int) : Device()
|
||||
|
||||
public fun toInt(): Int {
|
||||
when (this) {
|
||||
is CPU -> return 0
|
||||
is CUDA -> return this.index + 1
|
||||
}
|
||||
}
|
||||
|
||||
public companion object {
|
||||
public fun fromInt(deviceInt: Int): Device {
|
||||
return if (deviceInt == 0) CPU else CUDA(
|
||||
deviceInt - 1
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
35
kmath-noa/src/main/kotlin/space/kscience/kmath/noa/utils.kt
Normal file
35
kmath-noa/src/main/kotlin/space/kscience/kmath/noa/utils.kt
Normal file
@ -0,0 +1,35 @@
|
||||
/*
|
||||
* Copyright 2018-2021 KMath contributors.
|
||||
* Use of this source code is governed by the Apache 2.0 license that can be found in the license/LICENSE.txt file.
|
||||
*/
|
||||
|
||||
package space.kscience.kmath.noa
|
||||
|
||||
import space.kscience.kmath.operations.Field
|
||||
|
||||
public fun cudaAvailable(): Boolean {
|
||||
return JNoa.cudaIsAvailable()
|
||||
}
|
||||
|
||||
public fun getNumThreads(): Int {
|
||||
return JNoa.getNumThreads()
|
||||
}
|
||||
|
||||
public fun setNumThreads(numThreads: Int): Unit {
|
||||
JNoa.setNumThreads(numThreads)
|
||||
}
|
||||
|
||||
public fun setSeed(seed: Int): Unit {
|
||||
JNoa.setSeed(seed)
|
||||
}
|
||||
|
||||
public inline fun <T, A : Field<T>, ArrayT,
|
||||
GradTensorT : NoaTensorOverField<T>,
|
||||
GradAlgebraT : NoaPartialDivisionAlgebra<T, A, ArrayT, GradTensorT>>
|
||||
GradAlgebraT.withGradAt(
|
||||
tensor: GradTensorT,
|
||||
block: GradAlgebraT.(GradTensorT) -> GradTensorT
|
||||
): GradTensorT {
|
||||
tensor.requiresGrad = true
|
||||
return this.block(tensor)
|
||||
}
|
1533
kmath-noa/src/main/resources/space_kscience_kmath_noa_JNoa.h
Normal file
1533
kmath-noa/src/main/resources/space_kscience_kmath_noa_JNoa.h
Normal file
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,84 @@
|
||||
/*
|
||||
* Copyright 2018-2021 KMath contributors.
|
||||
* Use of this source code is governed by the Apache 2.0 license that can be found in the license/LICENSE.txt file.
|
||||
*/
|
||||
|
||||
package space.kscience.kmath.noa
|
||||
|
||||
import kotlin.test.Test
|
||||
import kotlin.test.assertTrue
|
||||
|
||||
|
||||
internal fun NoaDouble.testingLinearStructure(device: Device = Device.CPU): Unit {
|
||||
|
||||
val shape = intArrayOf(3)
|
||||
val tensorA = full(value = -4.5, shape = shape, device = device)
|
||||
val tensorB = full(value = 10.9, shape = shape, device = device)
|
||||
val tensorC = full(value = 789.3, shape = shape, device = device)
|
||||
val tensorD = full(value = -72.9, shape = shape, device = device)
|
||||
val tensorE = full(value = 553.1, shape = shape, device = device)
|
||||
val result = 15.8 * tensorA - 1.5 * tensorB * (-tensorD) + 0.02 * tensorC / tensorE - 39.4
|
||||
val expected = copyFromArray(
|
||||
array = (1..3).map {
|
||||
15.8 * (-4.5) - 1.5 * 10.9 * 72.9 + 0.02 * 789.3 / 553.1 - 39.4
|
||||
}.toDoubleArray(),
|
||||
shape = shape,
|
||||
device = device
|
||||
)
|
||||
|
||||
val assignResult = full(value = 0.0, shape = shape, device = device)
|
||||
tensorA *= 15.8
|
||||
tensorB *= 1.5
|
||||
tensorB *= -tensorD
|
||||
tensorC *= 0.02
|
||||
tensorC /= tensorE
|
||||
assignResult += tensorA
|
||||
assignResult -= tensorB
|
||||
assignResult += tensorC
|
||||
assignResult += -39.4
|
||||
|
||||
val error = (expected - result).abs().sum() +
|
||||
(expected - assignResult).abs().sum()
|
||||
assertTrue(error < TOLERANCE)
|
||||
|
||||
}
|
||||
|
||||
internal fun NoaDouble.testingBatchedSVD(device: Device = Device.CPU): Unit {
|
||||
val tensor = randNormal(shape = intArrayOf(7, 5, 3), device = device)
|
||||
val (tensorU, tensorS, tensorV) = tensor.svd()
|
||||
val error = tensor - (tensorU dot (diagonalEmbedding(tensorS) dot tensorV.transpose(-2, -1)))
|
||||
assertTrue(error.abs().sum() < TOLERANCE)
|
||||
}
|
||||
|
||||
internal fun NoaDouble.testingBatchedSymEig(device: Device = Device.CPU): Unit {
|
||||
val tensor = randNormal(shape = intArrayOf(5, 5), device = device)
|
||||
val tensorSigma = tensor + tensor.transpose(-2, -1)
|
||||
val (tensorS, tensorV) = tensorSigma.symEig()
|
||||
val error = tensorSigma - (tensorV dot (diagonalEmbedding(tensorS) dot tensorV.transpose(-2, -1)))
|
||||
assertTrue(error.abs().sum() < TOLERANCE)
|
||||
}
|
||||
|
||||
|
||||
class TestAlgebra {
|
||||
|
||||
@Test
|
||||
fun testLinearStructure() = NoaDouble {
|
||||
withCuda { device ->
|
||||
testingLinearStructure(device)
|
||||
}
|
||||
}!!
|
||||
|
||||
@Test
|
||||
fun testBatchedSVD() = NoaDouble {
|
||||
withCuda { device ->
|
||||
testingBatchedSVD(device)
|
||||
}
|
||||
}!!
|
||||
|
||||
@Test
|
||||
fun testBatchedSymEig() = NoaDouble {
|
||||
withCuda { device ->
|
||||
testingBatchedSymEig(device)
|
||||
}
|
||||
}!!
|
||||
}
|
@ -0,0 +1,69 @@
|
||||
/*
|
||||
* Copyright 2018-2021 KMath contributors.
|
||||
* Use of this source code is governed by the Apache 2.0 license that can be found in the license/LICENSE.txt file.
|
||||
*/
|
||||
|
||||
package space.kscience.kmath.noa
|
||||
|
||||
import kotlin.test.Test
|
||||
import kotlin.test.assertTrue
|
||||
|
||||
internal fun NoaFloat.testingAutoGrad(device: Device = Device.CPU): Unit {
|
||||
setSeed(SEED)
|
||||
val dim = 3
|
||||
val tensorX = randNormal(shape = intArrayOf(dim), device = device)
|
||||
val randFeatures = randNormal(shape = intArrayOf(dim, dim), device = device)
|
||||
val tensorSigma = randFeatures + randFeatures.transpose(0, 1)
|
||||
val tensorMu = randNormal(shape = intArrayOf(dim), device = device)
|
||||
|
||||
val expressionAtX = withGradAt(tensorX) { x ->
|
||||
0.5f * (x dot (tensorSigma dot x)) + (tensorMu dot x) + 25.9f
|
||||
}
|
||||
|
||||
val gradientAtX = expressionAtX.autoGradient(tensorX, retainGraph = true)
|
||||
val hessianAtX = expressionAtX.autoHessian(tensorX)
|
||||
val expectedGradientAtX = (tensorSigma dot tensorX) + tensorMu
|
||||
|
||||
val error = (gradientAtX - expectedGradientAtX).abs().sum() +
|
||||
(hessianAtX - tensorSigma).abs().sum()
|
||||
assertTrue(error < TOLERANCE)
|
||||
}
|
||||
|
||||
internal fun NoaFloat.testingBatchedAutoGrad(device: Device = Device.CPU): Unit {
|
||||
setSeed(SEED)
|
||||
val batch = intArrayOf(2)
|
||||
val dim = 2
|
||||
val tensorX = randNormal(shape = batch + intArrayOf(1, dim), device = device)
|
||||
val randFeatures = randNormal(shape = batch + intArrayOf(dim, dim), device = device)
|
||||
val tensorSigma = randFeatures + randFeatures.transpose(-2, -1)
|
||||
val tensorMu = randNormal(shape = batch + intArrayOf(1, dim), device = device)
|
||||
|
||||
val expressionAtX = withGradAt(tensorX) { x ->
|
||||
val xt = x.transpose(-1, -2)
|
||||
(0.5f * (x dot (tensorSigma dot xt)) + (tensorMu dot xt) + 58.2f).sumAll()
|
||||
}
|
||||
|
||||
|
||||
val gradientAtX = expressionAtX.autoGradient(tensorX)
|
||||
val expectedGradientAtX = (tensorX dot tensorSigma) + tensorMu
|
||||
|
||||
val error = (gradientAtX - expectedGradientAtX).abs().sum()
|
||||
assertTrue(error < TOLERANCE)
|
||||
}
|
||||
|
||||
class TestAutoGrad {
|
||||
|
||||
@Test
|
||||
fun testAutoGrad() = NoaFloat {
|
||||
withCuda { device ->
|
||||
testingAutoGrad(device)
|
||||
}
|
||||
}!!
|
||||
|
||||
@Test
|
||||
fun testBatchedAutoGrad() = NoaFloat {
|
||||
withCuda { device ->
|
||||
testingBatchedAutoGrad(device)
|
||||
}
|
||||
}!!
|
||||
}
|
@ -0,0 +1,199 @@
|
||||
/*
|
||||
* Copyright 2018-2021 KMath contributors.
|
||||
* Use of this source code is governed by the Apache 2.0 license that can be found in the license/LICENSE.txt file.
|
||||
*/
|
||||
|
||||
package space.kscience.kmath.noa
|
||||
|
||||
import java.io.File
|
||||
import kotlin.test.Ignore
|
||||
import kotlin.test.Test
|
||||
import kotlin.test.assertTrue
|
||||
|
||||
class TestJitModules {
|
||||
|
||||
private val resources = File("").resolve("src/test/resources")
|
||||
private val dataPath = resources.resolve("data.pt").absolutePath
|
||||
private val netPath = resources.resolve("net.pt").absolutePath
|
||||
private val lossPath = resources.resolve("loss.pt").absolutePath
|
||||
|
||||
@Test
|
||||
fun testOptimisationAdam() = NoaFloat {
|
||||
|
||||
setSeed(SEED)
|
||||
|
||||
val dataModule = loadJitModule(dataPath)
|
||||
val netModule = loadJitModule(netPath)
|
||||
val lossModule = loadJitModule(lossPath)
|
||||
|
||||
val xTrain = dataModule.getBuffer("x_train")
|
||||
val yTrain = dataModule.getBuffer("y_train")
|
||||
val xVal = dataModule.getBuffer("x_val")
|
||||
val yVal = dataModule.getBuffer("y_val")
|
||||
|
||||
netModule.train(true)
|
||||
lossModule.setBuffer("target", yTrain)
|
||||
|
||||
val yPred = netModule.forward(xTrain)
|
||||
val loss = lossModule.forward(yPred)
|
||||
val optimiser = netModule.adamOptimiser(0.005)
|
||||
|
||||
repeat(250){
|
||||
optimiser.zeroGrad()
|
||||
netModule.forwardAssign(xTrain, yPred)
|
||||
lossModule.forwardAssign(yPred, loss)
|
||||
loss.backward()
|
||||
optimiser.step()
|
||||
}
|
||||
|
||||
netModule.forwardAssign(xVal, yPred)
|
||||
lossModule.setBuffer("target", yVal)
|
||||
lossModule.forwardAssign(yPred, loss)
|
||||
|
||||
assertTrue(loss.value() < 0.1)
|
||||
}!!
|
||||
|
||||
@Test
|
||||
fun testOptimisationRms() = NoaFloat {
|
||||
|
||||
setSeed(SEED)
|
||||
|
||||
val dataModule = loadJitModule(dataPath)
|
||||
val netModule = loadJitModule(netPath)
|
||||
val lossModule = loadJitModule(lossPath)
|
||||
|
||||
val xTrain = dataModule.getBuffer("x_train")
|
||||
val yTrain = dataModule.getBuffer("y_train")
|
||||
val xVal = dataModule.getBuffer("x_val")
|
||||
val yVal = dataModule.getBuffer("y_val")
|
||||
|
||||
netModule.train(true)
|
||||
lossModule.setBuffer("target", yTrain)
|
||||
|
||||
val yPred = netModule.forward(xTrain)
|
||||
val loss = lossModule.forward(yPred)
|
||||
val optimiser = netModule.rmsOptimiser(0.005, 0.99, 1e-08, 0.0, 0.0, false)
|
||||
|
||||
repeat(250){
|
||||
optimiser.zeroGrad()
|
||||
netModule.forwardAssign(xTrain, yPred)
|
||||
lossModule.forwardAssign(yPred, loss)
|
||||
loss.backward()
|
||||
optimiser.step()
|
||||
}
|
||||
|
||||
netModule.forwardAssign(xVal, yPred)
|
||||
lossModule.setBuffer("target", yVal)
|
||||
lossModule.forwardAssign(yPred, loss)
|
||||
|
||||
assertTrue(loss.value() < 0.1)
|
||||
}!!
|
||||
|
||||
@Test
|
||||
fun testOptimisationAdamW() = NoaFloat {
|
||||
|
||||
setSeed(SEED)
|
||||
|
||||
val dataModule = loadJitModule(dataPath)
|
||||
val netModule = loadJitModule(netPath)
|
||||
val lossModule = loadJitModule(lossPath)
|
||||
|
||||
val xTrain = dataModule.getBuffer("x_train")
|
||||
val yTrain = dataModule.getBuffer("y_train")
|
||||
val xVal = dataModule.getBuffer("x_val")
|
||||
val yVal = dataModule.getBuffer("y_val")
|
||||
|
||||
netModule.train(true)
|
||||
lossModule.setBuffer("target", yTrain)
|
||||
|
||||
val yPred = netModule.forward(xTrain)
|
||||
val loss = lossModule.forward(yPred)
|
||||
val optimiser = netModule.adamWOptimiser(0.005, 0.9, 0.999, 1e-08, 0.01, false)
|
||||
|
||||
repeat(250){
|
||||
optimiser.zeroGrad()
|
||||
netModule.forwardAssign(xTrain, yPred)
|
||||
lossModule.forwardAssign(yPred, loss)
|
||||
loss.backward()
|
||||
optimiser.step()
|
||||
}
|
||||
|
||||
netModule.forwardAssign(xVal, yPred)
|
||||
lossModule.setBuffer("target", yVal)
|
||||
lossModule.forwardAssign(yPred, loss)
|
||||
|
||||
assertTrue(loss.value() < 0.1)
|
||||
}!!
|
||||
|
||||
@Test
|
||||
fun testOptimisationAdagrad() = NoaFloat {
|
||||
|
||||
setSeed(SEED)
|
||||
|
||||
val dataModule = loadJitModule(dataPath)
|
||||
val netModule = loadJitModule(netPath)
|
||||
val lossModule = loadJitModule(lossPath)
|
||||
|
||||
val xTrain = dataModule.getBuffer("x_train")
|
||||
val yTrain = dataModule.getBuffer("y_train")
|
||||
val xVal = dataModule.getBuffer("x_val")
|
||||
val yVal = dataModule.getBuffer("y_val")
|
||||
|
||||
netModule.train(true)
|
||||
lossModule.setBuffer("target", yTrain)
|
||||
|
||||
val yPred = netModule.forward(xTrain)
|
||||
val loss = lossModule.forward(yPred)
|
||||
val optimiser = netModule.adagradOptimiser(0.05, 0.0, 0.0, 0.0, 1e-10)
|
||||
|
||||
repeat(250){
|
||||
optimiser.zeroGrad()
|
||||
netModule.forwardAssign(xTrain, yPred)
|
||||
lossModule.forwardAssign(yPred, loss)
|
||||
loss.backward()
|
||||
optimiser.step()
|
||||
}
|
||||
|
||||
netModule.forwardAssign(xVal, yPred)
|
||||
lossModule.setBuffer("target", yVal)
|
||||
lossModule.forwardAssign(yPred, loss)
|
||||
|
||||
assertTrue(loss.value() < 0.1)
|
||||
}!!
|
||||
|
||||
@Test
|
||||
fun testOptimisationSgd() = NoaFloat {
|
||||
|
||||
setSeed(SEED)
|
||||
|
||||
val dataModule = loadJitModule(dataPath)
|
||||
val netModule = loadJitModule(netPath)
|
||||
val lossModule = loadJitModule(lossPath)
|
||||
|
||||
val xTrain = dataModule.getBuffer("x_train")
|
||||
val yTrain = dataModule.getBuffer("y_train")
|
||||
val xVal = dataModule.getBuffer("x_val")
|
||||
val yVal = dataModule.getBuffer("y_val")
|
||||
|
||||
netModule.train(true)
|
||||
lossModule.setBuffer("target", yTrain)
|
||||
|
||||
val yPred = netModule.forward(xTrain)
|
||||
val loss = lossModule.forward(yPred)
|
||||
val optimiser = netModule.sgdOptimiser(0.01, 0.9, 0.0, 0.0, false)
|
||||
|
||||
repeat(400){
|
||||
optimiser.zeroGrad()
|
||||
netModule.forwardAssign(xTrain, yPred)
|
||||
lossModule.forwardAssign(yPred, loss)
|
||||
loss.backward()
|
||||
optimiser.step()
|
||||
}
|
||||
|
||||
netModule.forwardAssign(xVal, yPred)
|
||||
lossModule.setBuffer("target", yVal)
|
||||
lossModule.forwardAssign(yPred, loss)
|
||||
|
||||
assertTrue(loss.value() < 0.1)
|
||||
}!!
|
||||
}
|
132
kmath-noa/src/test/kotlin/space/kscience/kmath/noa/TestTensor.kt
Normal file
132
kmath-noa/src/test/kotlin/space/kscience/kmath/noa/TestTensor.kt
Normal file
@ -0,0 +1,132 @@
|
||||
/*
|
||||
* Copyright 2018-2021 KMath contributors.
|
||||
* Use of this source code is governed by the Apache 2.0 license that can be found in the license/LICENSE.txt file.
|
||||
*/
|
||||
|
||||
package space.kscience.kmath.noa
|
||||
|
||||
import java.io.File
|
||||
import kotlin.test.Test
|
||||
import kotlin.test.assertEquals
|
||||
import kotlin.test.assertTrue
|
||||
|
||||
|
||||
internal fun NoaFloat.testingCopying(device: Device = Device.CPU): Unit {
|
||||
val array = (1..24).map { 10f * it * it }.toFloatArray()
|
||||
val shape = intArrayOf(2, 3, 4)
|
||||
val tensor = copyFromArray(array, shape = shape, device = device)
|
||||
val copyOfTensor = tensor.copy()
|
||||
tensor[intArrayOf(1, 2, 3)] = 0.1f
|
||||
assertTrue(copyOfTensor.copyToArray() contentEquals array)
|
||||
assertEquals(0.1f, tensor[intArrayOf(1, 2, 3)])
|
||||
if (device != Device.CPU) {
|
||||
val normalCpu = randNormal(intArrayOf(2, 3))
|
||||
val normalGpu = normalCpu.copyToDevice(device)
|
||||
assertTrue(normalCpu.copyToArray() contentEquals normalGpu.copyToArray())
|
||||
|
||||
val uniformGpu = randUniform(intArrayOf(3, 2), device)
|
||||
val uniformCpu = uniformGpu.copyToDevice(Device.CPU)
|
||||
assertTrue(uniformGpu.copyToArray() contentEquals uniformCpu.copyToArray())
|
||||
}
|
||||
}
|
||||
|
||||
internal fun NoaInt.testingViewWithNoCopy(device: Device = Device.CPU) {
|
||||
val tensor = copyFromArray(intArrayOf(1, 2, 3, 4, 5, 6), shape = intArrayOf(6), device)
|
||||
val viewTensor = tensor.view(intArrayOf(2, 3))
|
||||
assertTrue(viewTensor.shape contentEquals intArrayOf(2, 3))
|
||||
viewTensor[intArrayOf(0, 0)] = 10
|
||||
assertEquals(tensor[intArrayOf(0)], 10)
|
||||
}
|
||||
|
||||
internal fun NoaFloat.testingSerialisation(tensorPath: String, device: Device = Device.CPU) {
|
||||
val tensor = copyFromArray(floatArrayOf(45.5f, 98.6f), intArrayOf(2), device)
|
||||
tensor.save(tensorPath)
|
||||
val loadedTensor = loadTensor(tensorPath, device)
|
||||
assertTrue(tensor.copyToArray() contentEquals loadedTensor.copyToArray())
|
||||
}
|
||||
|
||||
internal fun NoaFloat.testingBatchedGetterSetter(device: Device = Device.CPU) {
|
||||
val array = (1..8).map { 100f * it }.toFloatArray()
|
||||
val tensor = full(0.0f, intArrayOf(2, 2, 2), device)
|
||||
tensor.assignFromArray(array)
|
||||
assertTrue(tensor.copyToArray() contentEquals array)
|
||||
|
||||
val updateArray = floatArrayOf(15f, 20f)
|
||||
val updateTensor = full(5.0f, intArrayOf(4), device)
|
||||
updateTensor[0, Slice(1, 3)] = updateArray
|
||||
|
||||
NoaFloat {
|
||||
tensor[0][1] = updateArray
|
||||
tensor[1] = updateTensor.view(intArrayOf(2, 2))
|
||||
updateTensor[0, Slice(2, 4)] = updateTensor[0, Slice(0, 2)]
|
||||
}!!
|
||||
|
||||
assertTrue(
|
||||
tensor.copyToArray() contentEquals
|
||||
floatArrayOf(100f, 200f, 15f, 20f, 5f, 15f, 20f, 5f)
|
||||
)
|
||||
assertTrue(
|
||||
updateTensor.copyToArray() contentEquals
|
||||
floatArrayOf(5f, 15f, 5f, 15f)
|
||||
)
|
||||
|
||||
}
|
||||
|
||||
class TestTensor {
|
||||
|
||||
private val resources = File("").resolve("src/test/resources")
|
||||
private val tensorPath = resources.resolve("tensor.pt").absolutePath
|
||||
|
||||
@Test
|
||||
fun testCopying() = NoaFloat {
|
||||
withCuda { device ->
|
||||
testingCopying(device)
|
||||
}
|
||||
}!!
|
||||
|
||||
@Test
|
||||
fun testRequiresGrad() = NoaFloat {
|
||||
val tensor = randNormal(intArrayOf(3))
|
||||
assertTrue(!tensor.requiresGrad)
|
||||
tensor.requiresGrad = true
|
||||
assertTrue(tensor.requiresGrad)
|
||||
tensor.requiresGrad = false
|
||||
assertTrue(!tensor.requiresGrad)
|
||||
tensor.requiresGrad = true
|
||||
val detachedTensor = tensor.detachFromGraph()
|
||||
assertTrue(!detachedTensor.requiresGrad)
|
||||
}!!
|
||||
|
||||
@Test
|
||||
fun testTypeMoving() = NoaFloat {
|
||||
val tensorInt = copyFromArray(floatArrayOf(1f, 2f, 3f), intArrayOf(3)).asInt()
|
||||
NoaInt {
|
||||
val temporalTensor = copyFromArray(intArrayOf(4, 5, 6), intArrayOf(3))
|
||||
tensorInt swap temporalTensor
|
||||
assertTrue(temporalTensor.copyToArray() contentEquals intArrayOf(1, 2, 3))
|
||||
}
|
||||
assertTrue(tensorInt.asFloat().copyToArray() contentEquals floatArrayOf(4f, 5f, 6f))
|
||||
}!!
|
||||
|
||||
@Test
|
||||
fun testViewWithNoCopy() = NoaInt {
|
||||
withCuda { device ->
|
||||
testingViewWithNoCopy(device)
|
||||
}
|
||||
}!!
|
||||
|
||||
@Test
|
||||
fun testSerialisation() = NoaFloat {
|
||||
withCuda { device ->
|
||||
testingSerialisation(tensorPath, device)
|
||||
}
|
||||
}!!
|
||||
|
||||
@Test
|
||||
fun testBatchedGetterSetter() = NoaFloat {
|
||||
withCuda { device ->
|
||||
testingBatchedGetterSetter(device)
|
||||
}
|
||||
}!!
|
||||
|
||||
}
|
@ -0,0 +1,76 @@
|
||||
/*
|
||||
* Copyright 2018-2021 KMath contributors.
|
||||
* Use of this source code is governed by the Apache 2.0 license that can be found in the license/LICENSE.txt file.
|
||||
*/
|
||||
|
||||
package space.kscience.kmath.noa
|
||||
|
||||
import space.kscience.kmath.noa.memory.NoaScope
|
||||
import space.kscience.kmath.operations.Ring
|
||||
import kotlin.test.Test
|
||||
import kotlin.test.assertEquals
|
||||
|
||||
|
||||
internal val SEED = 987654
|
||||
internal val TOLERANCE = 1e-6
|
||||
|
||||
internal fun <T, A : Ring<T>, ArrayT, TensorT : NoaTensor<T>, AlgebraT : NoaAlgebra<T, A, ArrayT, TensorT>>
|
||||
AlgebraT.withCuda(block: AlgebraT.(Device) -> Unit): Unit {
|
||||
this.block(Device.CPU)
|
||||
if (cudaAvailable()) this.block(Device.CUDA(0))
|
||||
}
|
||||
|
||||
internal fun NoaFloat.testingSetSeed(device: Device = Device.CPU): Unit {
|
||||
setSeed(SEED)
|
||||
val integral = randDiscrete(0, 100, IntArray(0), device = device).value()
|
||||
val normal = randNormal(IntArray(0), device = device).value()
|
||||
val uniform = randUniform(IntArray(0), device = device).value()
|
||||
setSeed(SEED)
|
||||
val nextIntegral = randDiscrete(0, 100, IntArray(0), device = device).value()
|
||||
val nextNormal = randNormal(IntArray(0), device = device).value()
|
||||
val nextUniform = randUniform(IntArray(0), device = device).value()
|
||||
assertEquals(normal, nextNormal)
|
||||
assertEquals(uniform, nextUniform)
|
||||
assertEquals(integral, nextIntegral)
|
||||
}
|
||||
|
||||
|
||||
class TestUtils {
|
||||
|
||||
@Test
|
||||
fun testException() {
|
||||
val i = try {
|
||||
JNoa.testException(5)
|
||||
} catch (e: NoaException) {
|
||||
10
|
||||
}
|
||||
assertEquals(i, 10)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testSetNumThreads() {
|
||||
val numThreads = 2
|
||||
setNumThreads(numThreads)
|
||||
assertEquals(numThreads, getNumThreads())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testSetSeed() = NoaFloat {
|
||||
withCuda { device ->
|
||||
testingSetSeed(device)
|
||||
}
|
||||
}!!
|
||||
|
||||
@Test
|
||||
fun testScoping(): Unit {
|
||||
val scope = NoaScope()
|
||||
val tensor = NoaFloat(scope){
|
||||
full(5f, intArrayOf(1))
|
||||
}!!
|
||||
assertEquals(tensor.numElements, 1)
|
||||
assertEquals(scope.disposables.size, 1)
|
||||
scope.disposeAll()
|
||||
assertEquals(scope.disposables.size, 0)
|
||||
}
|
||||
|
||||
}
|
BIN
kmath-noa/src/test/resources/data.pt
Normal file
BIN
kmath-noa/src/test/resources/data.pt
Normal file
Binary file not shown.
52
kmath-noa/src/test/resources/jitscripts.py
Normal file
52
kmath-noa/src/test/resources/jitscripts.py
Normal file
@ -0,0 +1,52 @@
|
||||
import torch
|
||||
|
||||
torch.manual_seed(987654)
|
||||
|
||||
n_tr = 7
|
||||
n_val = 300
|
||||
|
||||
x_val = torch.linspace(-5, 5, n_val).view(-1, 1)
|
||||
y_val = torch.sin(x_val)
|
||||
x_train = torch.linspace(-3.14, 3.14, n_tr).view(-1, 1)
|
||||
y_train = torch.sin(x_train) + torch.randn_like(x_train) * 0.1
|
||||
|
||||
|
||||
class Data(torch.nn.Module):
|
||||
def __init__(self):
|
||||
super(Data, self).__init__()
|
||||
self.register_buffer('x_val', x_val)
|
||||
self.register_buffer('y_val', y_val)
|
||||
self.register_buffer('x_train', x_train)
|
||||
self.register_buffer('y_train', y_train)
|
||||
|
||||
|
||||
class Net(torch.nn.Module):
|
||||
def __init__(self):
|
||||
super(Net, self).__init__()
|
||||
self.l1 = torch.nn.Linear(1, 10, bias = True)
|
||||
self.l2 = torch.nn.Linear(10, 10, bias = True)
|
||||
self.l3 = torch.nn.Linear(10, 1, bias = True)
|
||||
|
||||
def forward(self, x):
|
||||
x = self.l1(x)
|
||||
x = torch.relu(x)
|
||||
x = self.l2(x)
|
||||
x = torch.relu(x)
|
||||
x = self.l3(x)
|
||||
return x
|
||||
|
||||
class Loss(torch.nn.Module):
|
||||
def __init__(self, target):
|
||||
super(Loss, self).__init__()
|
||||
self.register_buffer('target', target)
|
||||
self.loss = torch.nn.MSELoss()
|
||||
|
||||
def forward(self, x):
|
||||
return self.loss(x, self.target)
|
||||
|
||||
|
||||
torch.jit.script(Data()).save('data.pt')
|
||||
|
||||
torch.jit.script(Net()).save('net.pt')
|
||||
|
||||
torch.jit.script(Loss(y_train)).save('loss.pt')
|
BIN
kmath-noa/src/test/resources/loss.pt
Normal file
BIN
kmath-noa/src/test/resources/loss.pt
Normal file
Binary file not shown.
BIN
kmath-noa/src/test/resources/net.pt
Normal file
BIN
kmath-noa/src/test/resources/net.pt
Normal file
Binary file not shown.
BIN
kmath-noa/src/test/resources/tensor.pt
Normal file
BIN
kmath-noa/src/test/resources/tensor.pt
Normal file
Binary file not shown.
@ -16,7 +16,7 @@ kotlin.sourceSets {
|
||||
commonMain {
|
||||
dependencies {
|
||||
api(project(":kmath-core"))
|
||||
api(project(":kmath-stat"))
|
||||
implementation(project(":kmath-stat"))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
@ -67,8 +67,10 @@ public interface LinearOpsTensorAlgebra<T, A : Field<T>> : TensorPartialDivision
|
||||
* LUP decomposition
|
||||
*
|
||||
* Computes the LUP decomposition of a matrix or a batch of matrices.
|
||||
* Given a tensor `input`, return tensors (P, L, U) satisfying `P dot input = L dot U`,
|
||||
* with `P` being a permutation matrix or batch of matrices,
|
||||
* Given a tensor `input`, return tensors (P, L, U) satisfying :
|
||||
* `P dot input = L dot U` or `input = P dot L dot U`
|
||||
* depending on the implementation, with :
|
||||
* `P` being a permutation matrix or batch of matrices,
|
||||
* `L` being a lower triangular matrix or batch of matrices,
|
||||
* `U` being an upper triangular matrix or batch of matrices.
|
||||
*
|
||||
|
2005
kotlin-js-store/yarn.lock
Normal file
2005
kotlin-js-store/yarn.lock
Normal file
File diff suppressed because it is too large
Load Diff
@ -30,3 +30,8 @@ include(
|
||||
":examples",
|
||||
":benchmarks",
|
||||
)
|
||||
|
||||
|
||||
if(System.getProperty("os.name") == "Linux"){
|
||||
include(":kmath-noa")
|
||||
}
|
||||
|
Loading…
Reference in New Issue
Block a user