diff --git a/.gitignore b/.gitignore
index 17a319a4..53b55cd4 100644
--- a/.gitignore
+++ b/.gitignore
@@ -5,5 +5,7 @@ out/
.gradle
build/
+.kotlin
+
!gradle-wrapper.jar
\ No newline at end of file
diff --git a/CHANGELOG.md b/CHANGELOG.md
index e32cc6bf..06dfa5bf 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -14,6 +14,94 @@
### Security
+## 0.10.0 - 2025-01-19
+
+### Added
+
+- Coroutine exception logging in context
+- `ObservableMutableMetaSerializer`
+- `MutableMetaView` - a Meta wrapper that creates nodes only when its or its children are changed.
+
+### Changed
+
+- Simplify inheritance logic in `MutableTypedMeta`
+- Full rework of `DataTree` and associated interfaces (`DataSource`, `DataSink`, etc.).
+- Filter data by type is moved from `dataforge-data` to `dataforge-workspace` to avoid reflection dependency.
+
+### Deprecated
+
+- MetaProvider `spec` is replaced by `readable`. `listOfSpec` replaced with `listOfReadable`
+
+### Removed
+
+- Remove implicit io format resolver in `IOPlugin` and `FileWorkspaceCache`. There are no guarantees that only one format is present in the contrxt for each type.
+- Dependencies on `atomicfu` and `kotlin.reflect` from dataforge-data to improve performance.
+
+### Fixed
+
+- Fixed NameToken parsing.
+- Top level string list meta conversion.
+
+## 0.9.0 - 2024-06-04
+
+### Added
+
+- Custom CoroutineContext during `Context` creation.
+
+### Changed
+
+- Kotlin 2.0
+- `MetaSpec` renamed to `MetaReader`. MetaSpec is now reserved for builder-based generation of meta descriptors.
+- Add self-type for Meta. Remove unsafe cast method for meta instances.
+
+### Removed
+
+- Automatic descriptors for schema. It is not possible to implement them without heavy reflection.
+
+## 0.8.2 - 2024-04-27
+
+### Added
+
+- Name index comparator
+- Specialized ByteArrayValue
+
+### Changed
+
+- DataSink `branch` is replaced with `putAll` to avoid confusion with DataTree methods
+- Meta delegate now uses a specific class that has a descriptor
+
+### Fixed
+
+- `listOfScheme` and `listOfConvertable` delegates provides correct items order.
+- Scheme meta setter works with proper sub-branch.
+- NameToken.parse improper work with indices.
+- Proper data handling for cache.
+
+## 0.8.0 - 2024-02-03
+
+### Added
+
+- Wasm artifacts
+- Add automatic MetaConverter for serializeable objects
+- Add Meta and MutableMeta delegates for convertable and serializeable
+- Meta mapping for data.
+
+### Changed
+
+- Descriptor `children` renamed to `nodes`
+- `MetaConverter` now inherits `MetaSpec` (former `Specifiction`). So `MetaConverter` could be used more universally.
+- Meta copy and modification now use lightweight non-observable meta builders.
+- Full refactor of Data API. DataTree now works similar to Meta: contains optional anonymous root element and data items. Updates are available for `ObservaleDataSource` and `ObservableDataTree` variants.
+
+### Deprecated
+
+- `node(key,converter)` in favor of `serializable` delegate
+
+### Fixed
+
+- Partially fixed a bug with `MutableMeta` observable wrappers.
+- `valueSequence` now include root value. So `meta.update` works properly.
+
## 0.7.0 - 2023-11-26
### Added
diff --git a/README.md b/README.md
index da910804..a3dd7b7b 100644
--- a/README.md
+++ b/README.md
@@ -1,7 +1,70 @@
[](https://confluence.jetbrains.com/display/ALL/JetBrains+on+GitHub)
[](https://zenodo.org/badge/latestdoi/148831678)
-
+## Publications
+
+* [A general overview](https://doi.org/10.1051/epjconf/201817705003)
+* [An application in "Troitsk nu-mass" experiment](https://doi.org/10.1088/1742-6596/1525/1/012024)
+
+## Video
+
+* [A presentation on application of DataForge (legacy version) to Troitsk nu-mass analysis.](https://youtu.be/OpWzLXUZnLI?si=3qn7EMruOHMJX3Bc)
+
+## Questions and Answers
+
+In this section, we will try to cover DataForge main ideas in the form of questions and answers.
+
+### General
+
+**Q**: I have a lot of data to analyze. The analysis process is complicated, requires a lot of stages, and data flow is not always obvious. Also, the data size is huge, so I don't want to perform operation I don't need (calculate something I won't need or calculate something twice). I need it to be performed in parallel and probably on remote computer. By the way, I am sick and tired of scripts that modify other scripts that control scripts. Could you help me?
+
+**A**: Yes, that is precisely the problem DataForge was made to solve. It allows performing some automated data manipulations with optimization and parallelization. The important thing that data processing recipes are made in the declarative way, so it is quite easy to perform computations on a remote station. Also, DataForge guarantees reproducibility of analysis results.
+
+**Q**: How does it work?
+
+**A**: At the core of DataForge lies the idea of metadata processor. It utilizes the fact that to analyze something you need data itself and some additional information about what does that data represent and what does user want as a result. This additional information is called metadata and could be organized in a regular structure (a tree of values similar to XML or JSON). The important thing is that this distinction leaves no place for user instructions (or scripts). Indeed, the idea of DataForge logic is that one does not need imperative commands. The framework configures itself according to input meta-data and decides what operations should be performed in the most efficient way.
+
+**Q**: But where does it take algorithms to use?
+
+**A**: Of course algorithms must be written somewhere. No magic here. The logic is written in specialized modules. Some modules are provided out of the box at the system core, some need to be developed for a specific problem.
+
+**Q**: So I still need to write the code? What is the difference then?
+
+**A**: Yes, someone still needs to write the code. But not necessary you. Simple operations could be performed using provided core logic. Also, your group can have one programmer writing the logic and all other using it without any real programming expertise. The framework organized in a such way that one writes some additional logic, they do not need to think about complicated thing like parallel computing, resource handling, logging, caching, etc. Most of the things are done by the DataForge.
+
+### Platform
+
+**Q**: Which platform does DataForge use? Which operating system is it working on?
+
+**A**: The DataForge is mostly written in Kotlin-multiplatform and could be used on JVM, JS and native targets. Some modules and functions are supported only on JVM
+
+**Q**: Can I use my C++/Fortran/Python code in DataForge?
+
+**A**: Yes, as long as the code could be called from Java. Most common languages have a bridge for Java access. There are completely no problems with compiled C/Fortran libraries. Python code could be called via one of existing python-java interfaces. It is also planned to implement remote method invocation for common languages, so your Python, or, say, Julia, code could run in its native environment. The metadata processor paradigm makes it much easier to do so.
+
+### Features
+
+**Q**: What other features does DataForge provide?
+
+**A**: Alongside metadata processing (and a lot of tools for metadata manipulation and layering), DataForge has two additional important concepts:
+
+* **Modularisation**. Contrary to lot other frameworks, DataForge is intrinsically modular. The mandatory part is a rather tiny core module. Everything else could be customized.
+
+* **Context encapsulation**. Every DataForge task is executed in some context. The context isolates environment for the task and also works as dependency injection base and specifies interaction of the task with the external world.
+
+### Misc
+
+**Q**: So everything looks great, can I replace my ROOT / other data analysis framework with DataForge?
+
+**A**: One must note that DataForge is made for analysis, not for visualization. The visualization and user interaction capabilities of DataForge are rather limited compared to frameworks like ROOT, JAS3 or DataMelt. The idea is to provide reliable API and core functionality. [VisionForge](https://git.sciprog.center/kscience/visionforge) project aims to provide tools for both 2D and 3D visualization both locally and remotely.
+
+**Q**: How does DataForge compare to cluster computation frameworks like Apache Spark?
+
+**A**: It is not the purpose of DataForge to replace cluster computing software. DataForge has some internal parallelism mechanics and implementations, but they are most certainly worse than specially developed programs. Still, DataForge is not fixed on one single implementation. Your favourite parallel processing tool could be still used as a back-end for the DataForge. With full benefit of configuration tools, integrations and no performance overhead.
+
+**Q**: Is it possible to use DataForge in notebook mode?
+
+**A**: [Kotlin jupyter](https://github.com/Kotlin/kotlin-jupyter) allows using any JVM program in a notebook mode. The dedicated module for DataForge is work in progress.
### [dataforge-context](dataforge-context)
@@ -14,16 +77,31 @@
> **Maturity**: EXPERIMENTAL
### [dataforge-io](dataforge-io)
-> IO module
+> Serialization foundation for Meta objects and Envelope processing.
>
> **Maturity**: EXPERIMENTAL
+>
+> **Features:**
+> - [IO format](dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/IOFormat.kt) : A generic API for reading something from binary representation and writing it to Binary.
+> - [Binary](dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/Binary.kt) : Multi-read random access binary.
+> - [Envelope](dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/Envelope.kt) : API and implementations for combined data and metadata format.
+> - [Tagged envelope](dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/TaggedEnvelope.kt) : Implementation for binary-friendly envelope format with machine readable tag and forward size declaration.
+> - [Tagged envelope](dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/TaglessEnvelope.kt) : Implementation for text-friendly envelope format with text separators for sections.
+
### [dataforge-meta](dataforge-meta)
-> Meta definition and basic operations on meta
+> Core Meta and Name manipulation module
>
> **Maturity**: DEVELOPMENT
+>
+> **Features:**
+> - [Meta](dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Meta.kt) : **Meta** is the representation of basic DataForge concept: Metadata, but it also could be called meta-value tree.
+> - [Value](dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Value.kt) : **Value** a sum type for different meta values.
+> - [Name](dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/names/Name.kt) : **Name** is an identifier to access tree-like structure.
+
### [dataforge-scripting](dataforge-scripting)
+> Scripting definition fow workspace generation
>
> **Maturity**: PROTOTYPE
@@ -31,6 +109,11 @@
>
> **Maturity**: EXPERIMENTAL
+### [dataforge-io/dataforge-io-proto](dataforge-io/dataforge-io-proto)
+> ProtoBuf Meta representation
+>
+> **Maturity**: PROTOTYPE
+
### [dataforge-io/dataforge-io-yaml](dataforge-io/dataforge-io-yaml)
> YAML meta converters and Front Matter envelope format
>
diff --git a/build.gradle.kts b/build.gradle.kts
index 4b5fe48c..84c1bba7 100644
--- a/build.gradle.kts
+++ b/build.gradle.kts
@@ -3,24 +3,31 @@ import space.kscience.gradle.useApache2Licence
import space.kscience.gradle.useSPCTeam
plugins {
- id("space.kscience.gradle.project")
+ alias(spclibs.plugins.kscience.project)
+ alias(spclibs.plugins.kotlinx.kover)
}
allprojects {
group = "space.kscience"
- version = "0.7.0"
+ version = "0.10.0"
}
subprojects {
apply(plugin = "maven-publish")
tasks.withType<KotlinCompile> {
- kotlinOptions {
- freeCompilerArgs = freeCompilerArgs + "-Xcontext-receivers"
+ compilerOptions {
+ freeCompilerArgs.add("-Xcontext-receivers")
}
}
}
+dependencies{
+ subprojects.forEach {
+ dokka(it)
+ }
+}
+
readme {
readmeTemplate = file("docs/templates/README-TEMPLATE.md")
}
@@ -30,8 +37,8 @@ ksciencePublish {
useApache2Licence()
useSPCTeam()
}
- repository("spc","https://maven.sciprog.center/kscience")
- sonatype()
+ repository("spc", "https://maven.sciprog.center/kscience")
+ central()
}
apiValidation {
diff --git a/dataforge-context/README.md b/dataforge-context/README.md
index 905171b1..894868fa 100644
--- a/dataforge-context/README.md
+++ b/dataforge-context/README.md
@@ -6,18 +6,16 @@ Context and provider definitions
## Artifact:
-The Maven coordinates of this project are `space.kscience:dataforge-context:0.7.0`.
+The Maven coordinates of this project are `space.kscience:dataforge-context:0.10.0`.
**Gradle Kotlin DSL:**
```kotlin
repositories {
maven("https://repo.kotlin.link")
- //uncomment to access development builds
- //maven("https://maven.pkg.jetbrains.space/spc/p/sci/dev")
mavenCentral()
}
dependencies {
- implementation("space.kscience:dataforge-context:0.7.0")
+ implementation("space.kscience:dataforge-context:0.10.0")
}
```
diff --git a/dataforge-context/api/dataforge-context.api b/dataforge-context/api/dataforge-context.api
index eac8e52c..390123b8 100644
--- a/dataforge-context/api/dataforge-context.api
+++ b/dataforge-context/api/dataforge-context.api
@@ -57,6 +57,7 @@ public abstract interface class space/kscience/dataforge/context/ContextAware {
public final class space/kscience/dataforge/context/ContextBuilder {
public final fun build ()Lspace/kscience/dataforge/context/Context;
+ public final fun coroutineContext (Lkotlin/coroutines/CoroutineContext;)V
public final fun getName ()Lspace/kscience/dataforge/names/Name;
public final fun plugin (Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Lkotlin/jvm/functions/Function1;)V
public final fun plugin (Lspace/kscience/dataforge/context/Plugin;)V
@@ -69,9 +70,6 @@ public final class space/kscience/dataforge/context/ContextBuilder {
public final fun properties (Lkotlin/jvm/functions/Function1;)V
}
-public final class space/kscience/dataforge/context/ContextBuilderKt {
-}
-
public final class space/kscience/dataforge/context/DefaultLogManager : space/kscience/dataforge/context/AbstractPlugin, space/kscience/dataforge/context/LogManager {
public static final field Companion Lspace/kscience/dataforge/context/DefaultLogManager$Companion;
public fun <init> ()V
@@ -212,14 +210,14 @@ public final class space/kscience/dataforge/context/PluginTag : space/kscience/d
public fun toString ()Ljava/lang/String;
}
-public final class space/kscience/dataforge/context/PluginTag$$serializer : kotlinx/serialization/internal/GeneratedSerializer {
+public synthetic class space/kscience/dataforge/context/PluginTag$$serializer : kotlinx/serialization/internal/GeneratedSerializer {
public static final field INSTANCE Lspace/kscience/dataforge/context/PluginTag$$serializer;
- public fun childSerializers ()[Lkotlinx/serialization/KSerializer;
+ public final fun childSerializers ()[Lkotlinx/serialization/KSerializer;
public synthetic fun deserialize (Lkotlinx/serialization/encoding/Decoder;)Ljava/lang/Object;
- public fun deserialize (Lkotlinx/serialization/encoding/Decoder;)Lspace/kscience/dataforge/context/PluginTag;
- public fun getDescriptor ()Lkotlinx/serialization/descriptors/SerialDescriptor;
+ public final fun deserialize (Lkotlinx/serialization/encoding/Decoder;)Lspace/kscience/dataforge/context/PluginTag;
+ public final fun getDescriptor ()Lkotlinx/serialization/descriptors/SerialDescriptor;
public synthetic fun serialize (Lkotlinx/serialization/encoding/Encoder;Ljava/lang/Object;)V
- public fun serialize (Lkotlinx/serialization/encoding/Encoder;Lspace/kscience/dataforge/context/PluginTag;)V
+ public final fun serialize (Lkotlinx/serialization/encoding/Encoder;Lspace/kscience/dataforge/context/PluginTag;)V
public fun typeParametersSerializers ()[Lkotlinx/serialization/KSerializer;
}
@@ -249,13 +247,19 @@ public final class space/kscience/dataforge/context/SlfLogManager$Companion : sp
public fun getTag ()Lspace/kscience/dataforge/context/PluginTag;
}
-public final class space/kscience/dataforge/properties/PropertyKt {
+public abstract interface annotation class space/kscience/dataforge/descriptors/Description : java/lang/annotation/Annotation {
+ public abstract fun value ()Ljava/lang/String;
}
-public final class space/kscience/dataforge/properties/SchemePropertyKt {
+public abstract interface annotation class space/kscience/dataforge/descriptors/DescriptorResource : java/lang/annotation/Annotation {
+ public abstract fun resourceName ()Ljava/lang/String;
}
-public final class space/kscience/dataforge/provider/DfTypeKt {
+public abstract interface annotation class space/kscience/dataforge/descriptors/DescriptorUrl : java/lang/annotation/Annotation {
+ public abstract fun url ()Ljava/lang/String;
+}
+
+public abstract interface annotation class space/kscience/dataforge/descriptors/Multiple : java/lang/annotation/Annotation {
}
public final class space/kscience/dataforge/provider/Path : java/lang/Iterable, kotlin/jvm/internal/markers/KMappedMarker {
@@ -278,6 +282,7 @@ public final class space/kscience/dataforge/provider/Path : java/lang/Iterable,
public final class space/kscience/dataforge/provider/Path$Companion {
public final fun parse-X5wN5Vs (Ljava/lang/String;)Ljava/util/List;
+ public final fun serializer ()Lkotlinx/serialization/KSerializer;
}
public final class space/kscience/dataforge/provider/PathKt {
diff --git a/dataforge-context/build.gradle.kts b/dataforge-context/build.gradle.kts
index be9036d0..ad7b76a9 100644
--- a/dataforge-context/build.gradle.kts
+++ b/dataforge-context/build.gradle.kts
@@ -8,14 +8,15 @@ kscience {
jvm()
js()
native()
+ wasm()
useCoroutines()
useSerialization()
- dependencies {
- api(project(":dataforge-meta"))
+ commonMain {
+ api(projects.dataforgeMeta)
}
- dependencies(jvmMain){
- api(kotlin("reflect"))
- api("org.slf4j:slf4j-api:1.7.30")
+ jvmMain{
+ api(spclibs.kotlin.reflect)
+ api(spclibs.slf4j)
}
}
diff --git a/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/context/Context.kt b/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/context/Context.kt
index bb74d605..457c39fc 100644
--- a/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/context/Context.kt
+++ b/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/context/Context.kt
@@ -1,5 +1,6 @@
package space.kscience.dataforge.context
+import kotlinx.coroutines.CoroutineExceptionHandler
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Job
import kotlinx.coroutines.SupervisorJob
@@ -10,6 +11,7 @@ import space.kscience.dataforge.misc.ThreadSafe
import space.kscience.dataforge.names.Name
import space.kscience.dataforge.provider.Provider
import kotlin.coroutines.CoroutineContext
+import kotlin.coroutines.EmptyCoroutineContext
/**
* The local environment for anything being done in DataForge framework. Contexts are organized into tree structure with [Global] at the top.
@@ -26,6 +28,7 @@ public open class Context internal constructor(
public val parent: Context?,
plugins: Set<Plugin>, // set of unattached plugins
meta: Meta,
+ coroutineContext: CoroutineContext = EmptyCoroutineContext,
) : Named, MetaRepr, Provider, CoroutineScope {
/**
@@ -65,7 +68,9 @@ public open class Context internal constructor(
override val coroutineContext: CoroutineContext by lazy {
(parent ?: Global).coroutineContext.let { parenContext ->
- parenContext + SupervisorJob(parenContext[Job])
+ parenContext + coroutineContext + SupervisorJob(parenContext[Job]) + CoroutineExceptionHandler { _, throwable ->
+ logger.error(throwable) { "Exception in context $name" }
+ }
}
}
diff --git a/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/context/ContextBuilder.kt b/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/context/ContextBuilder.kt
index c0db4314..894c5f15 100644
--- a/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/context/ContextBuilder.kt
+++ b/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/context/ContextBuilder.kt
@@ -13,6 +13,8 @@ import space.kscience.dataforge.names.plus
import kotlin.collections.component1
import kotlin.collections.component2
import kotlin.collections.set
+import kotlin.coroutines.CoroutineContext
+import kotlin.coroutines.EmptyCoroutineContext
/**
* A convenience builder for context
@@ -59,8 +61,15 @@ public class ContextBuilder internal constructor(
plugin(DeFactoPluginFactory(plugin))
}
+ private var coroutineContext: CoroutineContext = EmptyCoroutineContext
+
+ public fun coroutineContext(coroutineContext: CoroutineContext) {
+ this.coroutineContext = coroutineContext
+ }
+
+
public fun build(): Context {
- val contextName = name ?: NameToken("@auto",hashCode().toUInt().toString(16)).asName()
+ val contextName = name ?: NameToken("@auto", hashCode().toUInt().toString(16)).asName()
val plugins = HashMap<PluginTag, Plugin>()
fun addPlugin(factory: PluginFactory<*>, meta: Meta) {
@@ -86,7 +95,7 @@ public class ContextBuilder internal constructor(
addPlugin(factory, meta)
}
- return Context(contextName, parent, plugins.values.toSet(), meta.seal())
+ return Context(contextName, parent, plugins.values.toSet(), meta.seal(), coroutineContext)
}
}
diff --git a/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/context/Plugin.kt b/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/context/Plugin.kt
index 152f5a76..6c5648a6 100644
--- a/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/context/Plugin.kt
+++ b/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/context/Plugin.kt
@@ -3,7 +3,7 @@ package space.kscience.dataforge.context
import space.kscience.dataforge.context.Plugin.Companion.TARGET
import space.kscience.dataforge.meta.Meta
import space.kscience.dataforge.meta.MetaRepr
-import space.kscience.dataforge.misc.DfId
+import space.kscience.dataforge.misc.DfType
import space.kscience.dataforge.misc.Named
import space.kscience.dataforge.names.Name
import space.kscience.dataforge.names.parseAsName
@@ -18,7 +18,7 @@ import space.kscience.dataforge.provider.Provider
*
* create - configure - attach - detach - destroy
*/
-@DfId(TARGET)
+@DfType(TARGET)
public interface Plugin : Named, ContextAware, Provider, MetaRepr {
/**
diff --git a/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/context/PluginFactory.kt b/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/context/PluginFactory.kt
index 0273d327..9cc67168 100644
--- a/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/context/PluginFactory.kt
+++ b/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/context/PluginFactory.kt
@@ -1,9 +1,9 @@
package space.kscience.dataforge.context
import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.misc.DfId
+import space.kscience.dataforge.misc.DfType
-@DfId(PluginFactory.TYPE)
+@DfType(PluginFactory.TYPE)
public interface PluginFactory<T : Plugin> : Factory<T> {
public val tag: PluginTag
diff --git a/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/properties/MetaProperty.kt b/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/properties/MetaProperty.kt
deleted file mode 100644
index e79ce931..00000000
--- a/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/properties/MetaProperty.kt
+++ /dev/null
@@ -1,35 +0,0 @@
-package space.kscience.dataforge.properties
-
-
-import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.meta.ObservableMutableMeta
-import space.kscience.dataforge.meta.transformations.MetaConverter
-import space.kscience.dataforge.meta.transformations.nullableMetaToObject
-import space.kscience.dataforge.meta.transformations.nullableObjectToMeta
-import space.kscience.dataforge.misc.DFExperimental
-import space.kscience.dataforge.names.Name
-import space.kscience.dataforge.names.startsWith
-
-@DFExperimental
-public class MetaProperty<T : Any>(
- public val meta: ObservableMutableMeta,
- public val name: Name,
- public val converter: MetaConverter<T>,
-) : Property<T?> {
-
- override var value: T?
- get() = converter.nullableMetaToObject(meta[name])
- set(value) {
- meta[name] = converter.nullableObjectToMeta(value) ?: Meta.EMPTY
- }
-
- override fun onChange(owner: Any?, callback: (T?) -> Unit) {
- meta.onChange(owner) { name ->
- if (name.startsWith(this@MetaProperty.name)) callback(converter.nullableMetaToObject(this[name]))
- }
- }
-
- override fun removeChangeListener(owner: Any?) {
- meta.removeListener(owner)
- }
-}
\ No newline at end of file
diff --git a/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/properties/Property.kt b/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/properties/Property.kt
deleted file mode 100644
index 7b0280eb..00000000
--- a/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/properties/Property.kt
+++ /dev/null
@@ -1,47 +0,0 @@
-package space.kscience.dataforge.properties
-
-import kotlinx.coroutines.ExperimentalCoroutinesApi
-import kotlinx.coroutines.flow.MutableStateFlow
-import kotlinx.coroutines.flow.StateFlow
-import space.kscience.dataforge.misc.DFExperimental
-
-@DFExperimental
-public interface Property<T> {
- public var value: T
-
- public fun onChange(owner: Any? = null, callback: (T) -> Unit)
- public fun removeChangeListener(owner: Any? = null)
-}
-
-@DFExperimental
-@OptIn(ExperimentalCoroutinesApi::class)
-public fun <T> Property<T>.toFlow(): StateFlow<T> = MutableStateFlow(value).also { stateFlow ->
- onChange {
- stateFlow.value = it
- }
-}
-
-/**
- * Reflect all changes in the [source] property onto this property. Does not reflect changes back.
- *
- * @return a mirroring job
- */
-@DFExperimental
-public fun <T> Property<T>.mirror(source: Property<T>) {
- source.onChange(this) {
- this.value = it
- }
-}
-
-/**
- * Bi-directional connection between properties
- */
-@DFExperimental
-public fun <T> Property<T>.bind(other: Property<T>) {
- onChange(other) {
- other.value = it
- }
- other.onChange {
- this.value = it
- }
-}
\ No newline at end of file
diff --git a/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/properties/metaAsFlow.kt b/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/properties/metaAsFlow.kt
new file mode 100644
index 00000000..da539fcb
--- /dev/null
+++ b/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/properties/metaAsFlow.kt
@@ -0,0 +1,51 @@
+package space.kscience.dataforge.properties
+
+
+import kotlinx.coroutines.CoroutineScope
+import kotlinx.coroutines.Job
+import kotlinx.coroutines.channels.awaitClose
+import kotlinx.coroutines.flow.*
+import kotlinx.coroutines.launch
+import space.kscience.dataforge.meta.*
+import space.kscience.dataforge.misc.DFExperimental
+
+@DFExperimental
+public fun <T> ObservableMeta.asFlow(converter: MetaReader<T>): Flow<T> = callbackFlow {
+ onChange(this){
+ trySend(converter.read(this))
+ }
+
+ awaitClose{
+ removeListener(this)
+ }
+}
+
+@DFExperimental
+public fun <T> MutableMeta.listenTo(
+ scope: CoroutineScope,
+ converter: MetaConverter<T>,
+ flow: Flow<T>,
+): Job = flow.onEach {
+ update(converter.convert(it))
+}.launchIn(scope)
+
+@DFExperimental
+public fun <T> ObservableMutableMeta.bind(
+ scope: CoroutineScope,
+ converter: MetaConverter<T>,
+ flow: MutableSharedFlow<T>,
+): Job = scope.launch{
+ listenTo(this, converter,flow)
+ onChange(flow){
+ launch {
+ flow.emit(converter.read(this@onChange))
+ }
+ }
+ flow.onCompletion {
+ removeListener(flow)
+ }
+}.also {
+ it.invokeOnCompletion {
+ removeListener(flow)
+ }
+}
diff --git a/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/properties/schemeProperty.kt b/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/properties/schemeProperty.kt
deleted file mode 100644
index 3b4d948d..00000000
--- a/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/properties/schemeProperty.kt
+++ /dev/null
@@ -1,31 +0,0 @@
-package space.kscience.dataforge.properties
-
-
-import space.kscience.dataforge.meta.Scheme
-import space.kscience.dataforge.misc.DFExperimental
-import space.kscience.dataforge.names.parseAsName
-import space.kscience.dataforge.names.startsWith
-import kotlin.reflect.KMutableProperty1
-
-@DFExperimental
-public fun <S : Scheme, T : Any> S.property(property: KMutableProperty1<S, T?>): Property<T?> =
- object : Property<T?> {
- override var value: T?
- get() = property.get(this@property)
- set(value) {
- property.set(this@property, value)
- }
-
- override fun onChange(owner: Any?, callback: (T?) -> Unit) {
- this@property.meta.onChange(this) { name ->
- if (name.startsWith(property.name.parseAsName(true))) {
- callback(property.get(this@property))
- }
- }
- }
-
- override fun removeChangeListener(owner: Any?) {
- this@property.meta.removeListener(this@property)
- }
-
- }
\ No newline at end of file
diff --git a/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/provider/Path.kt b/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/provider/Path.kt
index 9ecb68c2..3bfa83e1 100644
--- a/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/provider/Path.kt
+++ b/dataforge-context/src/commonMain/kotlin/space/kscience/dataforge/provider/Path.kt
@@ -15,15 +15,37 @@
*/
package space.kscience.dataforge.provider
+import kotlinx.serialization.KSerializer
+import kotlinx.serialization.Serializable
+import kotlinx.serialization.builtins.serializer
+import kotlinx.serialization.descriptors.SerialDescriptor
+import kotlinx.serialization.encoding.Decoder
+import kotlinx.serialization.encoding.Encoder
import space.kscience.dataforge.names.Name
import space.kscience.dataforge.names.parseAsName
import kotlin.jvm.JvmInline
+private object PathSerializer : KSerializer<Path> {
+
+ override val descriptor: SerialDescriptor
+ get() = String.serializer().descriptor
+
+ override fun serialize(encoder: Encoder, value: Path) {
+ encoder.encodeString(value.toString())
+ }
+
+ override fun deserialize(decoder: Decoder): Path {
+ return Path.parse(decoder.decodeString())
+ }
+}
+
+
/**
* Path interface.
*
*/
@JvmInline
+@Serializable(PathSerializer::class)
public value class Path(public val tokens: List<PathToken>) : Iterable<PathToken> {
override fun iterator(): Iterator<PathToken> = tokens.iterator()
@@ -33,6 +55,7 @@ public value class Path(public val tokens: List<PathToken>) : Iterable<PathToken
public companion object {
public const val PATH_SEGMENT_SEPARATOR: String = "/"
+
public fun parse(path: String): Path = Path(path.split(PATH_SEGMENT_SEPARATOR).map { PathToken.parse(it) })
}
}
diff --git a/dataforge-context/src/commonTest/kotlin/space/kscience/dataforge/properties/MetaPropertiesTest.kt b/dataforge-context/src/commonTest/kotlin/space/kscience/dataforge/properties/MetaPropertiesTest.kt
deleted file mode 100644
index 00b71673..00000000
--- a/dataforge-context/src/commonTest/kotlin/space/kscience/dataforge/properties/MetaPropertiesTest.kt
+++ /dev/null
@@ -1,28 +0,0 @@
-package space.kscience.dataforge.properties
-
-import space.kscience.dataforge.meta.Scheme
-import space.kscience.dataforge.meta.SchemeSpec
-import space.kscience.dataforge.meta.int
-import space.kscience.dataforge.misc.DFExperimental
-import kotlin.test.Test
-import kotlin.test.assertEquals
-
-internal class TestScheme : Scheme() {
- var a by int()
- var b by int()
- companion object : SchemeSpec<TestScheme>(::TestScheme)
-}
-
-@DFExperimental
-class MetaPropertiesTest {
- @Test
- fun testBinding() {
- val scheme = TestScheme.empty()
- val a = scheme.property(TestScheme::a)
- val b = scheme.property(TestScheme::b)
- a.bind(b)
- scheme.a = 2
- assertEquals(2, scheme.b)
- assertEquals(2, b.value)
- }
-}
\ No newline at end of file
diff --git a/dataforge-context/src/jsMain/kotlin/space/kscience/dataforge/properties/bindings.kt b/dataforge-context/src/jsMain/kotlin/space/kscience/dataforge/properties/bindings.kt
deleted file mode 100644
index b4451c97..00000000
--- a/dataforge-context/src/jsMain/kotlin/space/kscience/dataforge/properties/bindings.kt
+++ /dev/null
@@ -1,32 +0,0 @@
-package space.kscience.dataforge.properties
-
-import org.w3c.dom.HTMLInputElement
-import space.kscience.dataforge.misc.DFExperimental
-
-@DFExperimental
-public fun HTMLInputElement.bindValue(property: Property<String>) {
- if (this.onchange != null) error("Input element already bound")
- this.onchange = {
- property.value = this.value
- Unit
- }
- property.onChange(this) {
- if (value != it) {
- value = it
- }
- }
-}
-
-@DFExperimental
-public fun HTMLInputElement.bindChecked(property: Property<Boolean>) {
- if (this.onchange != null) error("Input element already bound")
- this.onchange = {
- property.value = this.checked
- Unit
- }
- property.onChange(this) {
- if (checked != it) {
- checked = it
- }
- }
-}
\ No newline at end of file
diff --git a/dataforge-context/src/jvmMain/kotlin/space/kscience/dataforge/context/ClassLoaderPlugin.kt b/dataforge-context/src/jvmMain/kotlin/space/kscience/dataforge/context/ClassLoaderPlugin.kt
index b2c703e1..44ba290f 100644
--- a/dataforge-context/src/jvmMain/kotlin/space/kscience/dataforge/context/ClassLoaderPlugin.kt
+++ b/dataforge-context/src/jvmMain/kotlin/space/kscience/dataforge/context/ClassLoaderPlugin.kt
@@ -17,7 +17,7 @@ package space.kscience.dataforge.context
import java.util.*
import kotlin.reflect.KClass
-import kotlin.reflect.full.cast
+import kotlin.reflect.cast
public class ClassLoaderPlugin(private val classLoader: ClassLoader) : AbstractPlugin() {
override val tag: PluginTag = PluginTag("classLoader", PluginTag.DATAFORGE_GROUP)
diff --git a/dataforge-context/src/jvmMain/kotlin/space/kscience/dataforge/descriptors/annotations.kt b/dataforge-context/src/jvmMain/kotlin/space/kscience/dataforge/descriptors/annotations.kt
deleted file mode 100644
index f97cb28d..00000000
--- a/dataforge-context/src/jvmMain/kotlin/space/kscience/dataforge/descriptors/annotations.kt
+++ /dev/null
@@ -1,126 +0,0 @@
-/*
- * Copyright 2018 Alexander Nozik.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package space.kscience.dataforge.descriptors
-
-//@MustBeDocumented
-//annotation class Attribute(
-// val key: String,
-// val value: String
-//)
-//
-//@MustBeDocumented
-//annotation class Attributes(
-// val attrs: Array<Attribute>
-//)
-//
-//@MustBeDocumented
-//annotation class ItemDef(
-// val info: String = "",
-// val multiple: Boolean = false,
-// val required: Boolean = false
-//)
-//
-//@Target(AnnotationTarget.PROPERTY)
-//@MustBeDocumented
-//annotation class ValueDef(
-// val type: Array<ValueType> = [ValueType.STRING],
-// val def: String = "",
-// val allowed: Array<String> = [],
-// val enumeration: KClass<*> = Any::class
-//)
-
-///**
-// * Description text for meta property, node or whole object
-// */
-//@Target(AnnotationTarget.CLASS, AnnotationTarget.PROPERTY)
-//@Retention(AnnotationRetention.RUNTIME)
-//@MustBeDocumented
-//annotation class Description(val value: String)
-//
-///**
-// * Annotation for value property which states that lists are expected
-// */
-//@Target(AnnotationTarget.PROPERTY)
-//@Retention(AnnotationRetention.RUNTIME)
-//@MustBeDocumented
-//annotation class Multiple
-//
-///**
-// * Descriptor target
-// * The DataForge path to the resource containing the description. Following targets are supported:
-// * 1. resource
-// * 1. file
-// * 1. class
-// * 1. method
-// * 1. property
-// *
-// *
-// * Does not work if [type] is provided
-// */
-//@Target(AnnotationTarget.CLASS, AnnotationTarget.PROPERTY, AnnotationTarget.VALUE_PARAMETER)
-//@Retention(AnnotationRetention.RUNTIME)
-//@MustBeDocumented
-//annotation class Descriptor(val value: String)
-//
-//
-///**
-// * Aggregator class for descriptor nodes
-// */
-//@Target(AnnotationTarget.CLASS, AnnotationTarget.PROPERTY, AnnotationTarget.FUNCTION, AnnotationTarget.VALUE_PARAMETER)
-//@Retention(AnnotationRetention.RUNTIME)
-//@MustBeDocumented
-//annotation class DescriptorNodes(vararg val nodes: NodeDef)
-//
-///**
-// * Aggregator class for descriptor values
-// */
-//@Target(AnnotationTarget.CLASS, AnnotationTarget.PROPERTY, AnnotationTarget.FUNCTION, AnnotationTarget.VALUE_PARAMETER)
-//@Retention(AnnotationRetention.RUNTIME)
-//@MustBeDocumented
-//annotation class DescriptorValues(vararg val nodes: ValueDef)
-//
-///**
-// * Alternative name for property descriptor declaration
-// */
-//@Target(AnnotationTarget.PROPERTY, AnnotationTarget.VALUE_PARAMETER)
-//@Retention(AnnotationRetention.RUNTIME)
-//@MustBeDocumented
-//annotation class DescriptorName(val name: String)
-//
-//@Target(AnnotationTarget.PROPERTY)
-//@Retention(AnnotationRetention.RUNTIME)
-//@MustBeDocumented
-//annotation class DescriptorValue(val def: ValueDef)
-////TODO enter fields directly?
-//
-//@Target(AnnotationTarget.PROPERTY)
-//@Retention(AnnotationRetention.RUNTIME)
-//@MustBeDocumented
-//annotation class ValueProperty(
-// val name: String = "",
-// val type: Array<ValueType> = arrayOf(ValueType.STRING),
-// val multiple: Boolean = false,
-// val def: String = "",
-// val enumeration: KClass<*> = Any::class,
-// val tags: Array<String> = emptyArray()
-//)
-//
-//
-//@Target(AnnotationTarget.PROPERTY)
-//@Retention(AnnotationRetention.RUNTIME)
-//@MustBeDocumented
-//annotation class NodeProperty(val name: String = "")
diff --git a/dataforge-context/src/jvmMain/kotlin/space/kscience/dataforge/descriptors/reflectiveDescriptors.kt b/dataforge-context/src/jvmMain/kotlin/space/kscience/dataforge/descriptors/reflectiveDescriptors.kt
index 9de5e280..590324d7 100644
--- a/dataforge-context/src/jvmMain/kotlin/space/kscience/dataforge/descriptors/reflectiveDescriptors.kt
+++ b/dataforge-context/src/jvmMain/kotlin/space/kscience/dataforge/descriptors/reflectiveDescriptors.kt
@@ -1,53 +1,132 @@
package space.kscience.dataforge.descriptors
+import kotlinx.serialization.ExperimentalSerializationApi
+import kotlinx.serialization.json.Json
+import kotlinx.serialization.json.decodeFromStream
+import org.slf4j.LoggerFactory
+import space.kscience.dataforge.meta.descriptors.MetaDescriptor
+import space.kscience.dataforge.meta.descriptors.MetaDescriptorBuilder
+import space.kscience.dataforge.misc.DFExperimental
+import java.net.URL
+import kotlin.reflect.KAnnotatedElement
+import kotlin.reflect.KProperty
-//inline fun <reified T : Scheme> T.buildDescriptor(): NodeDescriptor = NodeDescriptor {
-// T::class.apply {
-// findAnnotation<ItemDef>()?.let { def ->
-// info = def.info
-// required = def.required
-// multiple = def.multiple
-// }
-// findAnnotation<Attribute>()?.let { attr ->
-// attributes {
-// this[attr.key] = attr.value.parseValue()
-// }
-// }
-// findAnnotation<Attributes>()?.attrs?.forEach { attr ->
-// attributes {
-// this[attr.key] = attr.value.parseValue()
-// }
-// }
-// }
-// T::class.memberProperties.forEach { property ->
-// val delegate = property.getDelegate(this@buildDescriptor)
+/**
+ * Description text for meta property, node or whole object
+ */
+@Target(AnnotationTarget.CLASS, AnnotationTarget.PROPERTY)
+@Retention(AnnotationRetention.RUNTIME)
+@MustBeDocumented
+public annotation class Description(val value: String)
+
+@Target(AnnotationTarget.PROPERTY)
+@Retention(AnnotationRetention.RUNTIME)
+@MustBeDocumented
+public annotation class Multiple()
+
+@Target(AnnotationTarget.CLASS, AnnotationTarget.PROPERTY)
+@Retention(AnnotationRetention.RUNTIME)
+@MustBeDocumented
+public annotation class DescriptorResource(val resourceName: String)
+
+@Target(AnnotationTarget.CLASS, AnnotationTarget.PROPERTY)
+@Retention(AnnotationRetention.RUNTIME)
+@MustBeDocumented
+public annotation class DescriptorUrl(val url: String)
+
+
+@OptIn(ExperimentalSerializationApi::class)
+private fun MetaDescriptorBuilder.loadDescriptorFromUrl(url: URL) {
+ url.openStream().use {
+ from(Json.decodeFromStream(MetaDescriptor.serializer(), it))
+ }
+}
+
+private fun MetaDescriptorBuilder.loadDescriptorFromResource(resource: DescriptorResource) {
+ val url = {}.javaClass.getResource(resource.resourceName)
+ if (url != null) {
+ loadDescriptorFromUrl(url)
+ } else {
+ LoggerFactory.getLogger("System")
+ .error("Can't find descriptor resource with name ${resource.resourceName}")
+ }
+}
+
+@DFExperimental
+public fun MetaDescriptorBuilder.forAnnotatedElement(element: KAnnotatedElement) {
+ element.annotations.forEach {
+ when (it) {
+ is Description -> description = it.value
+
+ is DescriptorResource -> loadDescriptorFromResource(it)
+
+ is DescriptorUrl -> loadDescriptorFromUrl(URL(it.url))
+ }
+ }
+}
+
+@DFExperimental
+public fun MetaDescriptorBuilder.forProperty(property: KProperty<*>) {
+ property.annotations.forEach {
+ when (it) {
+ is Description -> description = it.value
+
+ is DescriptorResource -> loadDescriptorFromResource(it)
+
+ is DescriptorUrl -> loadDescriptorFromUrl(URL(it.url))
+ }
+ }
+}
//
-// val descriptor: ItemDescriptor = when (delegate) {
-// is ConfigurableDelegate -> buildPropertyDescriptor(property, delegate)
-// is ReadWriteDelegateWrapper<*, *> -> {
-// if (delegate.delegate is ConfigurableDelegate) {
-// buildPropertyDescriptor(property, delegate.delegate as ConfigurableDelegate)
-// } else {
-// return@forEach
+//@DFExperimental
+//public fun <T : Scheme> MetaDescriptor.Companion.forScheme(
+// spec: SchemeSpec<T>,
+// mod: MetaDescriptorBuilder.() -> Unit = {},
+//): MetaDescriptor = MetaDescriptor {
+// val scheme = spec.empty()
+// val kClass: KClass<T> = scheme::class as KClass<T>
+// when {
+// kClass.isSubclassOf(Number::class) -> valueType(ValueType.NUMBER)
+// kClass == String::class -> ValueType.STRING
+// kClass == Boolean::class -> ValueType.BOOLEAN
+// kClass == DoubleArray::class -> ValueType.LIST
+// kClass == ByteArray::class -> ValueType.LIST
+// }
+//
+// forAnnotatedElement(kClass)
+// kClass.memberProperties.forEach { property ->
+// node(property.name) {
+//
+// (property.getDelegate(scheme) as? MetaDelegate<*>)?.descriptor?.let {
+// from(it)
+// }
+//
+// property.annotations.forEach {
+// when (it) {
+// is Description -> {
+// description = it.value
+// }
+//
+// is Multiple -> {
+// multiple = true
+// }
+//
+// is DescriptorResource -> {
+// loadDescriptorFromResource(it)
+// }
+//
+// is DescriptorUrl -> {
+// loadDescriptorFromUrl(URL(it.url))
+// }
// }
// }
-// else -> return@forEach
-// }
-// defineItem(property.name, descriptor)
-// }
-//}
-
-//inline fun <T : Scheme, reified V : Any?> buildPropertyDescriptor(
-// property: KProperty1<T, V>,
-// delegate: ConfigurableDelegate
-//): ItemDescriptor {
-// when {
-// V::class.isSubclassOf(Scheme::class) -> NodeDescriptor {
-// default = delegate.default.node
-// }
-// V::class.isSubclassOf(Meta::class) -> NodeDescriptor {
-// default = delegate.default.node
// }
//
// }
+// mod()
//}
+//
+//@DFExperimental
+//public inline fun <reified T : Scheme> SchemeSpec<T>.autoDescriptor(
+// noinline mod: MetaDescriptorBuilder.() -> Unit = {},
+//): MetaDescriptor = MetaDescriptor.forScheme(this, mod)
\ No newline at end of file
diff --git a/dataforge-context/src/jvmMain/kotlin/space/kscience/dataforge/provider/dfType.kt b/dataforge-context/src/jvmMain/kotlin/space/kscience/dataforge/provider/dfType.kt
index ab34ea50..d291f378 100644
--- a/dataforge-context/src/jvmMain/kotlin/space/kscience/dataforge/provider/dfType.kt
+++ b/dataforge-context/src/jvmMain/kotlin/space/kscience/dataforge/provider/dfType.kt
@@ -4,29 +4,34 @@ import space.kscience.dataforge.context.Context
import space.kscience.dataforge.context.PluginBuilder
import space.kscience.dataforge.context.gather
import space.kscience.dataforge.misc.DFExperimental
-import space.kscience.dataforge.misc.DfId
+import space.kscience.dataforge.misc.DfType
import space.kscience.dataforge.misc.Named
import space.kscience.dataforge.names.Name
import kotlin.reflect.KClass
+import kotlin.reflect.KType
import kotlin.reflect.full.findAnnotation
-
+import kotlin.reflect.typeOf
@DFExperimental
-public val KClass<*>.dfId: String
- get() = findAnnotation<DfId>()?.id ?: simpleName ?: ""
+public val KClass<*>.dfType: String
+ get() = findAnnotation<DfType>()?.id ?: simpleName ?: ""
+
+@DFExperimental
+public val KType.dfType: String
+ get() = findAnnotation<DfType>()?.id ?: (classifier as? KClass<*>)?.simpleName ?: ""
/**
- * Provide an object with given name inferring target from its type using [DfId] annotation
+ * Provide an object with given name inferring target from its type using [DfType] annotation
*/
@DFExperimental
public inline fun <reified T : Any> Provider.provideByType(name: String): T? {
- val target = T::class.dfId
+ val target = typeOf<T>().dfType
return provide(target, name)
}
@DFExperimental
public inline fun <reified T : Any> Provider.top(): Map<Name, T> {
- val target = T::class.dfId
+ val target = typeOf<T>().dfType
return top(target)
}
@@ -35,15 +40,15 @@ public inline fun <reified T : Any> Provider.top(): Map<Name, T> {
*/
@DFExperimental
public inline fun <reified T : Any> Context.gather(inherit: Boolean = true): Map<Name, T> =
- gather<T>(T::class.dfId, inherit)
+ gather<T>(typeOf<T>().dfType, inherit)
@DFExperimental
public inline fun <reified T : Any> PluginBuilder.provides(items: Map<Name, T>) {
- provides(T::class.dfId, items)
+ provides(typeOf<T>().dfType, items)
}
@DFExperimental
public inline fun <reified T : Any> PluginBuilder.provides(vararg items: Named) {
- provides(T::class.dfId, *items)
+ provides(typeOf<T>().dfType, *items)
}
diff --git a/dataforge-context/src/jvmTest/kotlin/space/kscience/dataforge/descriptors/TestAutoDescriptors.kt b/dataforge-context/src/jvmTest/kotlin/space/kscience/dataforge/descriptors/TestAutoDescriptors.kt
new file mode 100644
index 00000000..3b1fce3d
--- /dev/null
+++ b/dataforge-context/src/jvmTest/kotlin/space/kscience/dataforge/descriptors/TestAutoDescriptors.kt
@@ -0,0 +1,29 @@
+@file:OptIn(DFExperimental::class)
+
+package space.kscience.dataforge.descriptors
+
+import space.kscience.dataforge.misc.DFExperimental
+
+//
+//class TestScheme : Scheme() {
+//
+// @Description("A")
+// val a by string()
+//
+// @Description("B")
+// val b by int()
+//
+// val c by int()
+//
+// companion object : SchemeSpec<TestScheme>(::TestScheme) {
+// override val descriptor: MetaDescriptor = autoDescriptor()
+// }
+//}
+//
+//class TestAutoDescriptors {
+// @Test
+// fun autoDescriptor() {
+// val autoDescriptor = MetaDescriptor.forScheme(TestScheme)
+// println(Json { prettyPrint = true }.encodeToString(autoDescriptor))
+// }
+//}
\ No newline at end of file
diff --git a/dataforge-context/src/wasmJsMain/kotlin/space/kscience/dataforge/context/loggingWasm.kt b/dataforge-context/src/wasmJsMain/kotlin/space/kscience/dataforge/context/loggingWasm.kt
new file mode 100644
index 00000000..740957b4
--- /dev/null
+++ b/dataforge-context/src/wasmJsMain/kotlin/space/kscience/dataforge/context/loggingWasm.kt
@@ -0,0 +1,3 @@
+package space.kscience.dataforge.context
+
+internal actual fun getGlobalLoggerFactory(): PluginFactory<out LogManager> = DefaultLogManager
\ No newline at end of file
diff --git a/dataforge-data/README.md b/dataforge-data/README.md
index 5935af6e..3970fc31 100644
--- a/dataforge-data/README.md
+++ b/dataforge-data/README.md
@@ -6,18 +6,16 @@
## Artifact:
-The Maven coordinates of this project are `space.kscience:dataforge-data:0.7.0`.
+The Maven coordinates of this project are `space.kscience:dataforge-data:0.10.0`.
**Gradle Kotlin DSL:**
```kotlin
repositories {
maven("https://repo.kotlin.link")
- //uncomment to access development builds
- //maven("https://maven.pkg.jetbrains.space/spc/p/sci/dev")
mavenCentral()
}
dependencies {
- implementation("space.kscience:dataforge-data:0.7.0")
+ implementation("space.kscience:dataforge-data:0.10.0")
}
```
diff --git a/dataforge-data/build.gradle.kts b/dataforge-data/build.gradle.kts
index 9f96604a..9451f59a 100644
--- a/dataforge-data/build.gradle.kts
+++ b/dataforge-data/build.gradle.kts
@@ -6,10 +6,10 @@ kscience{
jvm()
js()
native()
+ wasm()
useCoroutines()
dependencies {
- api(project(":dataforge-meta"))
- api(kotlin("reflect"))
+ api(projects.dataforgeMeta)
}
}
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/actions/AbstractAction.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/actions/AbstractAction.kt
index e7bbe6f6..1f1bbf2d 100644
--- a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/actions/AbstractAction.kt
+++ b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/actions/AbstractAction.kt
@@ -1,9 +1,9 @@
package space.kscience.dataforge.actions
-import kotlinx.coroutines.launch
+import kotlinx.coroutines.CoroutineScope
import space.kscience.dataforge.data.*
import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.misc.DFInternal
+import space.kscience.dataforge.misc.UnsafeKType
import space.kscience.dataforge.names.Name
import space.kscience.dataforge.names.startsWith
import kotlin.reflect.KType
@@ -19,47 +19,55 @@ internal fun MutableMap<Name, *>.removeWhatStartsWith(name: Name) {
/**
* An action that caches results on-demand and recalculates them on source push
*/
-public abstract class AbstractAction<in T : Any, R : Any>(
+public abstract class AbstractAction<T, R>(
public val outputType: KType,
) : Action<T, R> {
/**
* Generate initial content of the output
*/
- protected abstract fun DataSetBuilder<R>.generate(
- data: DataSet<T>,
+ protected abstract fun DataBuilderScope<R>.generate(
+ source: DataTree<T>,
meta: Meta,
- )
+ ): Map<Name, Data<R>>
/**
- * Update part of the data set when given [updateKey] is triggered by the source
+ * Update part of the data set using provided data
+ *
+ * @param source the source data tree in case we need several data items to update
+ * @param actionMeta the metadata used for the whole data tree
+ * @param updatedData an updated item
*/
- protected open fun DataSourceBuilder<R>.update(
- dataSet: DataSet<T>,
- meta: Meta,
- updateKey: Name,
+ protected open suspend fun DataSink<R>.update(
+ source: DataTree<T>,
+ actionMeta: Meta,
+ updateName: Name,
) {
- // By default, recalculate the whole dataset
- generate(dataSet, meta)
+ //by default regenerate the whole data set
+ writeAll(generate(source, actionMeta))
}
- @OptIn(DFInternal::class)
+ @OptIn(UnsafeKType::class)
override fun execute(
- dataSet: DataSet<T>,
+ source: DataTree<T>,
meta: Meta,
- ): DataSet<R> = if (dataSet is DataSource) {
- DataSource(outputType, dataSet){
- generate(dataSet, meta)
+ updatesScope: CoroutineScope
+ ): DataTree<R> = DataTree(
+ dataType = outputType,
+ scope = updatesScope,
+ initialData = DataBuilderScope<R>().generate(source, meta)
+ ) {
- launch {
- dataSet.updates.collect { name ->
- update(dataSet, meta, name)
- }
- }
+ //propagate updates
+ val updateSink = DataSink<R> { name, data ->
+ write(name, data)
}
- } else {
- DataTree<R>(outputType) {
- generate(dataSet, meta)
+
+ with(updateSink) {
+ source.updates.collect {
+ update(source, meta, it)
+ }
}
}
}
+
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/actions/Action.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/actions/Action.kt
index 4fed8e51..80898aa8 100644
--- a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/actions/Action.kt
+++ b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/actions/Action.kt
@@ -1,40 +1,50 @@
package space.kscience.dataforge.actions
-import space.kscience.dataforge.data.DataSet
+import kotlinx.coroutines.CoroutineScope
+import kotlinx.coroutines.DelicateCoroutinesApi
+import kotlinx.coroutines.GlobalScope
+import space.kscience.dataforge.data.DataTree
import space.kscience.dataforge.meta.Meta
import space.kscience.dataforge.misc.DFExperimental
/**
* A simple data transformation on a data node. Actions should avoid doing actual dependency evaluation in [execute].
*/
-public interface Action<in T : Any, out R : Any> {
+public fun interface Action<T, R> {
/**
* Transform the data in the node, producing a new node. By default, it is assumed that all calculations are lazy
* so not actual computation is started at this moment.
*/
- public fun execute(dataSet: DataSet<T>, meta: Meta = Meta.EMPTY): DataSet<R>
+ public fun execute(source: DataTree<T>, meta: Meta, updatesScope: CoroutineScope): DataTree<R>
public companion object
}
+/**
+ * A convenience method to transform data using given [action]
+ */
+@OptIn(DelicateCoroutinesApi::class)
+public fun <T, R> DataTree<T>.transform(
+ action: Action<T, R>,
+ meta: Meta = Meta.EMPTY,
+ updateScope: CoroutineScope = GlobalScope,
+): DataTree<R> = action.execute(this, meta, updateScope)
+
/**
* Action composition. The result is terminal if one of its parts is terminal
*/
-public infix fun <T : Any, I : Any, R : Any> Action<T, I>.then(action: Action<I, R>): Action<T, R> {
- // TODO introduce composite action and add optimize by adding action to the list
- return object : Action<T, R> {
-
- override fun execute(
- dataSet: DataSet<T>,
- meta: Meta,
- ): DataSet<R> = action.execute(this@then.execute(dataSet, meta), meta)
- }
+public infix fun <T, I, R> Action<T, I>.then(action: Action<I, R>): Action<T, R> = Action { dataSet, meta, scope ->
+ action.execute(this@then.execute(dataSet, meta, scope), meta, scope)
}
@DFExperimental
-public operator fun <T : Any, R : Any> Action<T, R>.invoke(
- dataSet: DataSet<T>,
+@OptIn(DelicateCoroutinesApi::class)
+public operator fun <T, R> Action<T, R>.invoke(
+ dataSet: DataTree<T>,
meta: Meta = Meta.EMPTY,
-): DataSet<R> = execute(dataSet, meta)
+ updateScope: CoroutineScope = GlobalScope,
+): DataTree<R> = execute(dataSet, meta, updateScope)
+
+
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/actions/MapAction.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/actions/MapAction.kt
index 883b3928..ffa313f2 100644
--- a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/actions/MapAction.kt
+++ b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/actions/MapAction.kt
@@ -6,8 +6,7 @@ import space.kscience.dataforge.meta.MutableMeta
import space.kscience.dataforge.meta.seal
import space.kscience.dataforge.meta.toMutableMeta
import space.kscience.dataforge.misc.DFBuilder
-import space.kscience.dataforge.misc.DFExperimental
-import space.kscience.dataforge.misc.DFInternal
+import space.kscience.dataforge.misc.UnsafeKType
import space.kscience.dataforge.names.Name
import kotlin.reflect.KType
import kotlin.reflect.typeOf
@@ -29,6 +28,7 @@ public class MapActionBuilder<T, R>(
public var name: Name,
public var meta: MutableMeta,
public val actionMeta: Meta,
+ public val dataType: KType,
@PublishedApi internal var outputType: KType,
) {
@@ -37,6 +37,7 @@ public class MapActionBuilder<T, R>(
/**
* Set unsafe [outputType] for the resulting data. Be sure that it is correct.
*/
+ @UnsafeKType
public fun <R1 : R> result(outputType: KType, f: suspend ActionEnv.(T) -> R1) {
this.outputType = outputType
result = f;
@@ -45,28 +46,31 @@ public class MapActionBuilder<T, R>(
/**
* Calculate the result of goal
*/
- public inline fun <reified R1 : R> result(noinline f: suspend ActionEnv.(T) -> R1) {
- outputType = typeOf<R1>()
- result = f;
- }
+ @OptIn(UnsafeKType::class)
+ public inline fun <reified R1 : R> result(noinline f: suspend ActionEnv.(T) -> R1): Unit = result(typeOf<R1>(), f)
}
-@PublishedApi
-internal class MapAction<in T : Any, R : Any>(
+@UnsafeKType
+public class MapAction<T, R>(
outputType: KType,
private val block: MapActionBuilder<T, R>.() -> Unit,
) : AbstractAction<T, R>(outputType) {
- private fun DataSetBuilder<R>.mapOne(name: Name, data: Data<T>, meta: Meta) {
+ private fun mapOne(name: Name, data: Data<T>?, meta: Meta): Pair<Name, Data<R>?> {
+ //fast return for null data
+ if (data == null) {
+ return name to null
+ }
// Creating a new environment for action using **old** name, old meta and task meta
val env = ActionEnv(name, data.meta, meta)
//applying transformation from builder
val builder = MapActionBuilder<T, R>(
- name,
- data.meta.toMutableMeta(), // using data meta
- meta,
- outputType
+ name = name,
+ meta = data.meta.toMutableMeta(), // using data meta
+ actionMeta = meta,
+ dataType = data.type,
+ outputType = outputType
).apply(block)
//getting new name
@@ -75,21 +79,30 @@ internal class MapAction<in T : Any, R : Any>(
//getting new meta
val newMeta = builder.meta.seal()
- @OptIn(DFInternal::class)
val newData = Data(builder.outputType, newMeta, dependencies = listOf(data)) {
builder.result(env, data.await())
}
//setting the data node
- data(newName, newData)
+ return newName to newData
}
- override fun DataSetBuilder<R>.generate(data: DataSet<T>, meta: Meta) {
- data.forEach { mapOne(it.name, it.data, meta) }
+ override fun DataBuilderScope<R>.generate(source: DataTree<T>, meta: Meta): Map<Name, Data<R>> = buildMap {
+ source.forEach { data ->
+ val (name, data) = mapOne(data.name, data, meta)
+ if (data != null) {
+ check(name !in keys) { "Data with key $name already exist in the result" }
+ put(name, data)
+ }
+ }
}
- override fun DataSourceBuilder<R>.update(dataSet: DataSet<T>, meta: Meta, updateKey: Name) {
- remove(updateKey)
- dataSet[updateKey]?.let { mapOne(updateKey, it, meta) }
+ override suspend fun DataSink<R>.update(
+ source: DataTree<T>,
+ actionMeta: Meta,
+ updateName: Name,
+ ) {
+ val (name, data) = mapOne(updateName, source.read(updateName), actionMeta)
+ write(name, data)
}
}
@@ -97,9 +110,9 @@ internal class MapAction<in T : Any, R : Any>(
/**
* A one-to-one mapping action
*/
-@DFExperimental
-@Suppress("FunctionName")
-public inline fun <T : Any, reified R : Any> Action.Companion.map(
+
+@OptIn(UnsafeKType::class)
+public inline fun <T, reified R> Action.Companion.mapping(
noinline builder: MapActionBuilder<T, R>.() -> Unit,
): Action<T, R> = MapAction(typeOf<R>(), builder)
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/actions/ReduceAction.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/actions/ReduceAction.kt
index fe823bd7..d7bacda5 100644
--- a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/actions/ReduceAction.kt
+++ b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/actions/ReduceAction.kt
@@ -3,18 +3,19 @@ package space.kscience.dataforge.actions
import space.kscience.dataforge.data.*
import space.kscience.dataforge.meta.Meta
import space.kscience.dataforge.meta.MutableMeta
+import space.kscience.dataforge.meta.get
+import space.kscience.dataforge.meta.string
import space.kscience.dataforge.misc.DFBuilder
-import space.kscience.dataforge.misc.DFExperimental
-import space.kscience.dataforge.misc.DFInternal
+import space.kscience.dataforge.misc.UnsafeKType
import space.kscience.dataforge.names.Name
import space.kscience.dataforge.names.parseAsName
import kotlin.reflect.KType
import kotlin.reflect.typeOf
-public class JoinGroup<T : Any, R : Any>(
+public class JoinGroup<T, R>(
public var name: String,
- internal val set: DataSet<T>,
+ internal val data: DataTree<T>,
@PublishedApi internal var outputType: KType,
) {
@@ -22,12 +23,12 @@ public class JoinGroup<T : Any, R : Any>(
public lateinit var result: suspend ActionEnv.(Map<Name, ValueWithMeta<T>>) -> R
- internal fun <R1 : R> result(outputType: KType, f: suspend ActionEnv.(Map<Name, ValueWithMeta<T>>) -> R1) {
+ internal fun <R1 : R> result(outputType: KType, f: suspend ActionEnv.(Map<Name, ValueWithMeta<T>>) -> R1) {
this.outputType = outputType
this.result = f;
}
- public inline fun <reified R1 : R> result(noinline f: suspend ActionEnv.(Map<Name, ValueWithMeta<T>>) -> R1) {
+ public inline fun <reified R1 : R> result(noinline f: suspend ActionEnv.(Map<Name, ValueWithMeta<T>>) -> R1) {
outputType = typeOf<R1>()
this.result = f;
}
@@ -35,31 +36,36 @@ public class JoinGroup<T : Any, R : Any>(
}
@DFBuilder
-public class ReduceGroupBuilder<T : Any, R : Any>(
+public class ReduceGroupBuilder<T, R>(
public val actionMeta: Meta,
private val outputType: KType,
) {
- private val groupRules: MutableList<(DataSet<T>) -> List<JoinGroup<T, R>>> = ArrayList();
+ private val groupRules: MutableList<(DataTree<T>) -> List<JoinGroup<T, R>>> = ArrayList();
/**
- * introduce grouping by meta value
+ * Group by a meta value
*/
- public fun byValue(tag: String, defaultTag: String = "@default", action: JoinGroup<T, R>.() -> Unit) {
+ @OptIn(UnsafeKType::class)
+ public fun byMetaValue(tag: String, defaultTag: String = "@default", action: JoinGroup<T, R>.() -> Unit) {
groupRules += { node ->
- GroupRule.byMetaValue(tag, defaultTag).gather(node).map {
- JoinGroup<T, R>(it.key, it.value, outputType).apply(action)
+ val groups = mutableMapOf<String, MutableMap<Name, Data<T>>>()
+ node.forEach { data ->
+ groups.getOrPut(data.meta[tag]?.string ?: defaultTag) { mutableMapOf() }.put(data.name, data)
+ }
+ groups.map { (key, dataMap) ->
+ JoinGroup<T, R>(key, dataMap.asTree(node.dataType), outputType).apply(action)
}
}
}
public fun group(
groupName: String,
- predicate: (Name, Meta) -> Boolean,
+ predicate: DataFilter,
action: JoinGroup<T, R>.() -> Unit,
) {
groupRules += { source ->
listOf(
- JoinGroup<T, R>(groupName, source.filter(predicate), outputType).apply(action)
+ JoinGroup<T, R>(groupName, source.filterData(predicate), outputType).apply(action)
)
}
}
@@ -67,29 +73,29 @@ public class ReduceGroupBuilder<T : Any, R : Any>(
/**
* Apply transformation to the whole node
*/
- public fun result(resultName: String, f: suspend ActionEnv.(Map<Name, ValueWithMeta<T>>) -> R) {
+ public fun result(resultName: String, f: suspend ActionEnv.(Map<Name, ValueWithMeta<T>>) -> R) {
groupRules += { node ->
listOf(JoinGroup<T, R>(resultName, node, outputType).apply { result(outputType, f) })
}
}
- internal fun buildGroups(input: DataSet<T>): List<JoinGroup<T, R>> =
+ internal fun buildGroups(input: DataTree<T>): List<JoinGroup<T, R>> =
groupRules.flatMap { it.invoke(input) }
}
@PublishedApi
-internal class ReduceAction<T : Any, R : Any>(
+internal class ReduceAction<T, R>(
outputType: KType,
private val action: ReduceGroupBuilder<T, R>.() -> Unit,
) : AbstractAction<T, R>(outputType) {
//TODO optimize reduction. Currently, the whole action recalculates on push
- override fun DataSetBuilder<R>.generate(data: DataSet<T>, meta: Meta) {
- ReduceGroupBuilder<T, R>(meta, outputType).apply(action).buildGroups(data).forEach { group ->
- val dataFlow: Map<Name, Data<T>> = group.set.asSequence().fold(HashMap()) { acc, value ->
+ override fun DataBuilderScope<R>.generate(source: DataTree<T>, meta: Meta): Map<Name, Data<R>> = buildMap {
+ ReduceGroupBuilder<T, R>(meta, outputType).apply(action).buildGroups(source).forEach { group ->
+ val dataFlow: Map<Name, Data<T>> = group.data.asSequence().fold(HashMap()) { acc, value ->
acc.apply {
- acc[value.name] = value.data
+ acc[value.name] = value
}
}
@@ -98,12 +104,12 @@ internal class ReduceAction<T : Any, R : Any>(
val groupMeta = group.meta
val env = ActionEnv(groupName.parseAsName(), groupMeta, meta)
- @OptIn(DFInternal::class) val res: Data<R> = dataFlow.reduceToData(
+ @OptIn(UnsafeKType::class) val res: Data<R> = dataFlow.reduceToData(
group.outputType,
meta = groupMeta
) { group.result.invoke(env, it) }
- data(env.name, res)
+ put(env.name, res)
}
}
}
@@ -111,7 +117,6 @@ internal class ReduceAction<T : Any, R : Any>(
/**
* A one-to-one mapping action
*/
-@DFExperimental
-public inline fun <reified T : Any, reified R : Any> Action.Companion.reduce(
+public inline fun <reified T, reified R> Action.Companion.reducing(
noinline builder: ReduceGroupBuilder<T, R>.() -> Unit,
): Action<T, R> = ReduceAction(typeOf<R>(), builder)
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/actions/SplitAction.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/actions/SplitAction.kt
index 24745929..acc1ba36 100644
--- a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/actions/SplitAction.kt
+++ b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/actions/SplitAction.kt
@@ -5,17 +5,15 @@ import space.kscience.dataforge.meta.Laminate
import space.kscience.dataforge.meta.Meta
import space.kscience.dataforge.meta.MutableMeta
import space.kscience.dataforge.meta.toMutableMeta
-import space.kscience.dataforge.misc.DFExperimental
import space.kscience.dataforge.names.Name
import space.kscience.dataforge.names.parseAsName
-import kotlin.collections.set
import kotlin.reflect.KType
import kotlin.reflect.typeOf
-public class SplitBuilder<T : Any, R : Any>(public val name: Name, public val meta: Meta) {
+public class SplitBuilder<T, R>(public val name: Name, public val meta: Meta) {
- public class FragmentRule<T : Any, R : Any>(
+ public class FragmentRule<T, R>(
public val name: Name,
public var meta: MutableMeta,
@PublishedApi internal var outputType: KType,
@@ -44,15 +42,15 @@ public class SplitBuilder<T : Any, R : Any>(public val name: Name, public val me
* Action that splits each incoming element into a number of fragments defined in builder
*/
@PublishedApi
-internal class SplitAction<T : Any, R : Any>(
+internal class SplitAction<T, R>(
outputType: KType,
private val action: SplitBuilder<T, R>.() -> Unit,
) : AbstractAction<T, R>(outputType) {
- private fun DataSetBuilder<R>.splitOne(name: Name, data: Data<T>, meta: Meta) {
- val laminate = Laminate(data.meta, meta)
+ private fun splitOne(name: Name, data: Data<T>?, meta: Meta): Map<Name, Data<R>?> = buildMap {
+ val laminate = Laminate(data?.meta, meta)
- val split = SplitBuilder<T, R>(name, data.meta).apply(action)
+ val split = SplitBuilder<T, R>(name, data?.meta ?: Meta.EMPTY).apply(action)
// apply individual fragment rules to result
@@ -64,29 +62,46 @@ internal class SplitAction<T : Any, R : Any>(
).apply(rule)
//data.map<R>(outputType, meta = env.meta) { env.result(it) }.named(fragmentName)
- data(
- fragmentName,
- @Suppress("OPT_IN_USAGE") Data(outputType, meta = env.meta, dependencies = listOf(data)) {
- env.result(data.await())
- }
- )
+ if (data == null) {
+ put(fragmentName, null)
+ } else {
+ put(
+ fragmentName,
+ @Suppress("OPT_IN_USAGE") Data(outputType, meta = env.meta, dependencies = listOf(data)) {
+ env.result(data.await())
+ }
+ )
+ }
}
}
- override fun DataSetBuilder<R>.generate(data: DataSet<T>, meta: Meta) {
- data.forEach { splitOne(it.name, it.data, meta) }
+ override fun DataBuilderScope<R>.generate(
+ source: DataTree<T>,
+ meta: Meta
+ ): Map<Name, Data<R>> = buildMap {
+ source.forEach {
+ splitOne(it.name, it, meta).forEach { (name, data) ->
+ check(name !in keys) { "Data with key $name already exist in the result" }
+ if (data != null) {
+ put(name, data)
+ }
+ }
+ }
}
- override fun DataSourceBuilder<R>.update(dataSet: DataSet<T>, meta: Meta, updateKey: Name) {
- remove(updateKey)
- dataSet[updateKey]?.let { splitOne(updateKey, it, meta) }
+ override suspend fun DataSink<R>.update(
+ source: DataTree<T>,
+ actionMeta: Meta,
+ updateName: Name,
+ ) {
+ writeAll(splitOne(updateName, source.read(updateName), actionMeta))
}
}
/**
* Action that splits each incoming element into a number of fragments defined in builder
*/
-@DFExperimental
-public inline fun <T : Any, reified R : Any> Action.Companion.split(
+
+public inline fun <T, reified R> Action.Companion.splitting(
noinline builder: SplitBuilder<T, R>.() -> Unit,
): Action<T, R> = SplitAction(typeOf<R>(), builder)
\ No newline at end of file
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/Data.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/Data.kt
index 984582e5..b9946a48 100644
--- a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/Data.kt
+++ b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/Data.kt
@@ -4,8 +4,8 @@ import kotlinx.coroutines.*
import space.kscience.dataforge.meta.Meta
import space.kscience.dataforge.meta.MetaRepr
import space.kscience.dataforge.meta.isEmpty
-import space.kscience.dataforge.misc.DFInternal
-import space.kscience.dataforge.misc.DfId
+import space.kscience.dataforge.misc.DfType
+import space.kscience.dataforge.misc.UnsafeKType
import kotlin.coroutines.CoroutineContext
import kotlin.coroutines.EmptyCoroutineContext
import kotlin.reflect.KType
@@ -14,7 +14,7 @@ import kotlin.reflect.typeOf
/**
* A data element characterized by its meta
*/
-@DfId(Data.TYPE)
+@DfType(Data.TYPE)
public interface Data<out T> : Goal<T>, MetaRepr {
/**
* Type marker for the data. The type is known before the calculation takes place so it could be checked.
@@ -41,7 +41,7 @@ public interface Data<out T> : Goal<T>, MetaRepr {
*/
internal val TYPE_OF_NOTHING: KType = typeOf<Unit>()
- public inline fun <reified T : Any> static(
+ public inline fun <reified T> wrapValue(
value: T,
meta: Meta = Meta.EMPTY,
): Data<T> = StaticData(typeOf<T>(), value, meta)
@@ -50,10 +50,10 @@ public interface Data<out T> : Goal<T>, MetaRepr {
* An empty data containing only meta
*/
@OptIn(DelicateCoroutinesApi::class)
- public fun empty(meta: Meta): Data<Nothing> = object : Data<Nothing> {
- override val type: KType = TYPE_OF_NOTHING
+ public fun buildEmpty(meta: Meta): Data<Nothing> = object : Data<Nothing> {
+ override val type: KType get() = TYPE_OF_NOTHING
override val meta: Meta = meta
- override val dependencies: Collection<Goal<*>> = emptyList()
+ override val dependencies: Collection<Goal<*>> get() = emptyList()
override val deferred: Deferred<Nothing>
get() = GlobalScope.async(start = CoroutineStart.LAZY) {
error("The Data is empty and could not be computed")
@@ -62,6 +62,8 @@ public interface Data<out T> : Goal<T>, MetaRepr {
override fun async(coroutineScope: CoroutineScope): Deferred<Nothing> = deferred
override fun reset() {}
}
+
+ public val EMPTY: Data<Nothing> = buildEmpty(Meta.EMPTY)
}
}
@@ -69,39 +71,37 @@ public interface Data<out T> : Goal<T>, MetaRepr {
* A lazily computed variant of [Data] based on [LazyGoal]
* One must ensure that proper [type] is used so this method should not be used
*/
-private class LazyData<T : Any>(
+private class LazyData<T>(
override val type: KType,
override val meta: Meta = Meta.EMPTY,
additionalContext: CoroutineContext = EmptyCoroutineContext,
- dependencies: Collection<Goal<*>> = emptyList(),
+ dependencies: Iterable<Goal<*>> = emptyList(),
block: suspend () -> T,
) : Data<T>, LazyGoal<T>(additionalContext, dependencies, block)
-public class StaticData<T : Any>(
+public class StaticData<T>(
override val type: KType,
value: T,
override val meta: Meta = Meta.EMPTY,
) : Data<T>, StaticGoal<T>(value)
@Suppress("FunctionName")
-public inline fun <reified T : Any> Data(value: T, meta: Meta = Meta.EMPTY): StaticData<T> =
+public inline fun <reified T> Data(value: T, meta: Meta = Meta.EMPTY): StaticData<T> =
StaticData(typeOf<T>(), value, meta)
-@Suppress("FunctionName")
-@DFInternal
-public fun <T : Any> Data(
+@UnsafeKType
+public fun <T> Data(
type: KType,
meta: Meta = Meta.EMPTY,
context: CoroutineContext = EmptyCoroutineContext,
- dependencies: Collection<Goal<*>> = emptyList(),
+ dependencies: Iterable<Goal<*>> = emptyList(),
block: suspend () -> T,
): Data<T> = LazyData(type, meta, context, dependencies, block)
-@OptIn(DFInternal::class)
-@Suppress("FunctionName")
-public inline fun <reified T : Any> Data(
+@OptIn(UnsafeKType::class)
+public inline fun <reified T> Data(
meta: Meta = Meta.EMPTY,
context: CoroutineContext = EmptyCoroutineContext,
- dependencies: Collection<Goal<*>> = emptyList(),
+ dependencies: Iterable<Goal<*>> = emptyList(),
noinline block: suspend () -> T,
): Data<T> = Data(typeOf<T>(), meta, context, dependencies, block)
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataFilter.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataFilter.kt
new file mode 100644
index 00000000..e9392da8
--- /dev/null
+++ b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataFilter.kt
@@ -0,0 +1,82 @@
+package space.kscience.dataforge.data
+
+import kotlinx.coroutines.flow.Flow
+import kotlinx.coroutines.flow.filter
+import space.kscience.dataforge.meta.Meta
+import space.kscience.dataforge.misc.DFInternal
+import space.kscience.dataforge.names.Name
+import space.kscience.dataforge.names.NameToken
+import space.kscience.dataforge.names.plus
+import kotlin.reflect.KType
+
+public fun interface DataFilter {
+
+ public fun accepts(name: Name, meta: Meta?, type: KType): Boolean
+
+ public companion object {
+ public val EMPTY: DataFilter = DataFilter { _, _, _ -> true }
+ }
+}
+
+public fun <T> DataSource<T>.filterData(
+ dataFilter: DataFilter,
+): DataSource<T> = object : DataSource<T> {
+ override val dataType: KType get() = this@filterData.dataType
+
+ override fun read(name: Name): Data<T>? =
+ this@filterData.read(name)?.takeIf {
+ dataFilter.accepts(name, it.meta, it.type)
+ }
+}
+
+/**
+ * Stateless filtered [ObservableDataSource]
+ */
+public fun <T> ObservableDataSource<T>.filterData(
+ predicate: DataFilter,
+): ObservableDataSource<T> = object : ObservableDataSource<T> {
+
+ override val updates: Flow<Name>
+ get() = this@filterData.updates.filter {
+ val data = read(it)
+ predicate.accepts(it, data?.meta, data?.type ?: dataType)
+ }
+
+ override val dataType: KType get() = this@filterData.dataType
+
+ override fun read(name: Name): Data<T>? =
+ this@filterData.read(name)?.takeIf { predicate.accepts(name, it.meta, it.type) }
+}
+
+/**
+ * A [DataTree] filtered by branch and some criterion, possibly changing resulting type
+ */
+@DFInternal
+public class FilteredDataTree<T>(
+ public val source: DataTree<T>,
+ public val filter: DataFilter,
+ public val branch: Name,
+ override val dataType: KType = source.dataType,
+) : DataTree<T> {
+
+ override val data: Data<T>?
+ get() = source[branch].takeIf {
+ filter.accepts(Name.EMPTY, it?.meta, it?.type ?: dataType)
+ }
+
+ override val items: Map<NameToken, DataTree<T>>
+ get() = source.branch(branch)?.items
+ ?.mapValues { FilteredDataTree(source, filter, branch + it.key) }
+ ?.filter { !it.value.isEmpty() }
+ ?: emptyMap()
+
+ override val updates: Flow<Name>
+ get() = source.updates.filter {
+ val data = read(it)
+ filter.accepts(it, data?.meta, data?.type ?: dataType)
+ }
+}
+
+public fun <T> DataTree<T>.filterData(
+ predicate: DataFilter,
+): FilteredDataTree<T> = FilteredDataTree(this, predicate, Name.EMPTY)
\ No newline at end of file
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataRenamer.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataRenamer.kt
new file mode 100644
index 00000000..af2596ac
--- /dev/null
+++ b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataRenamer.kt
@@ -0,0 +1,56 @@
+/*
+ * Copyright 2015 Alexander Nozik.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package space.kscience.dataforge.data
+
+import space.kscience.dataforge.meta.Meta
+import space.kscience.dataforge.meta.get
+import space.kscience.dataforge.meta.string
+import space.kscience.dataforge.misc.DFExperimental
+import space.kscience.dataforge.misc.UnsafeKType
+import space.kscience.dataforge.names.Name
+import space.kscience.dataforge.names.NameToken
+import space.kscience.dataforge.names.plus
+import kotlin.reflect.KType
+
+/**
+ * Interface that define rename rule for [Data]
+ */
+@DFExperimental
+public fun interface DataRenamer {
+ public fun rename(name: Name, meta: Meta, type: KType): Name
+
+ public companion object {
+
+ /**
+ * Prepend name token `key\[tagValue\]` to data name
+ */
+ @OptIn(UnsafeKType::class)
+ public fun groupByMetaValue(
+ key: String,
+ defaultTagValue: String,
+ ): DataRenamer = object : DataRenamer {
+
+ override fun rename(
+ name: Name,
+ meta: Meta,
+ type: KType
+ ): Name {
+ val tagValue: String = meta[key]?.string ?: defaultTagValue
+ return NameToken(key,tagValue).plus(name)
+ }
+ }
+ }
+}
\ No newline at end of file
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataSet.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataSet.kt
deleted file mode 100644
index 44639653..00000000
--- a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataSet.kt
+++ /dev/null
@@ -1,124 +0,0 @@
-package space.kscience.dataforge.data
-
-import kotlinx.coroutines.*
-import kotlinx.coroutines.flow.Flow
-import kotlinx.coroutines.flow.emptyFlow
-import kotlinx.coroutines.flow.mapNotNull
-import space.kscience.dataforge.data.Data.Companion.TYPE_OF_NOTHING
-import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.names.Name
-import space.kscience.dataforge.names.asName
-import space.kscience.dataforge.names.endsWith
-import space.kscience.dataforge.names.parseAsName
-import kotlin.reflect.KType
-
-public interface DataSet<out T : Any> {
-
- /**
- * The minimal common ancestor to all data in the node
- */
- public val dataType: KType
-
- /**
- * Meta-data associated with this node. If no meta is provided, returns [Meta.EMPTY].
- */
- public val meta: Meta
-
- /**
- * Traverse this [DataSet] returning named data instances. The order is not guaranteed.
- */
- public operator fun iterator(): Iterator<NamedData<T>>
-
- /**
- * Get data with given name.
- */
- public operator fun get(name: Name): Data<T>?
-
- public companion object {
- public val META_KEY: Name = "@meta".asName()
-
- /**
- * An empty [DataSet] that suits all types
- */
- public val EMPTY: DataSet<Nothing> = object : DataSet<Nothing> {
- override val dataType: KType = TYPE_OF_NOTHING
- override val meta: Meta get() = Meta.EMPTY
-
- override fun iterator(): Iterator<NamedData<Nothing>> = emptySequence<NamedData<Nothing>>().iterator()
-
- override fun get(name: Name): Data<Nothing>? = null
- }
- }
-}
-
-public fun <T : Any> DataSet<T>.asSequence(): Sequence<NamedData<T>> = object : Sequence<NamedData<T>> {
- override fun iterator(): Iterator<NamedData<T>> = this@asSequence.iterator()
-}
-
-/**
- * Return a single [Data] in this [DataSet]. Throw error if it is not single.
- */
-public fun <T : Any> DataSet<T>.single(): NamedData<T> = asSequence().single()
-
-public fun <T : Any> DataSet<T>.asIterable(): Iterable<NamedData<T>> = object : Iterable<NamedData<T>> {
- override fun iterator(): Iterator<NamedData<T>> = this@asIterable.iterator()
-}
-
-public operator fun <T : Any> DataSet<T>.get(name: String): Data<T>? = get(name.parseAsName())
-
-/**
- * A [DataSet] with propagated updates.
- */
-public interface DataSource<out T : Any> : DataSet<T>, CoroutineScope {
-
- /**
- * A flow of updated item names. Updates are propagated in a form of [Flow] of names of updated nodes.
- * Those can include new data items and replacement of existing ones. The replaced items could update existing data content
- * and replace it completely, so they should be pulled again.
- *
- */
- public val updates: Flow<Name>
-
- /**
- * Stop generating updates from this [DataSource]
- */
- public fun close() {
- coroutineContext[Job]?.cancel()
- }
-}
-
-public val <T : Any> DataSet<T>.updates: Flow<Name> get() = if (this is DataSource) updates else emptyFlow()
-//
-///**
-// * Flow all data nodes with names starting with [branchName]
-// */
-//public fun <T : Any> DataSet<T>.children(branchName: Name): Sequence<NamedData<T>> =
-// this@children.asSequence().filter {
-// it.name.startsWith(branchName)
-// }
-
-/**
- * Start computation for all goals in data node and return a job for the whole node
- */
-public fun <T : Any> DataSet<T>.startAll(coroutineScope: CoroutineScope): Job = coroutineScope.launch {
- asIterable().map {
- it.launch(this@launch)
- }.joinAll()
-}
-
-public suspend fun <T : Any> DataSet<T>.computeAndJoinAll(): Unit = coroutineScope { startAll(this).join() }
-
-public fun DataSet<*>.toMeta(): Meta = Meta {
- forEach {
- if (it.name.endsWith(DataSet.META_KEY)) {
- set(it.name, it.meta)
- } else {
- it.name put {
- "type" put it.type.toString()
- "meta" put it.meta
- }
- }
- }
-}
-
-public val <T : Any> DataSet<T>.updatesWithData: Flow<NamedData<T>> get() = updates.mapNotNull { get(it)?.named(it) }
\ No newline at end of file
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataSetBuilder.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataSetBuilder.kt
deleted file mode 100644
index f9f14f37..00000000
--- a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataSetBuilder.kt
+++ /dev/null
@@ -1,165 +0,0 @@
-package space.kscience.dataforge.data
-
-import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.meta.MutableMeta
-import space.kscience.dataforge.misc.DFExperimental
-import space.kscience.dataforge.names.Name
-import space.kscience.dataforge.names.isEmpty
-import space.kscience.dataforge.names.plus
-import kotlin.reflect.KType
-
-public interface DataSetBuilder<in T : Any> {
- public val dataType: KType
-
- /**
- * Remove all data items starting with [name]
- */
- public fun remove(name: Name)
-
- public fun data(name: Name, data: Data<T>?)
-
- /**
- * Set a current state of given [dataSet] into a branch [name]. Does not propagate updates
- */
- public fun node(name: Name, dataSet: DataSet<T>) {
- //remove previous items
- if (name != Name.EMPTY) {
- remove(name)
- }
-
- //Set new items
- dataSet.forEach {
- data(name + it.name, it.data)
- }
- }
-
- /**
- * Set meta for the given node
- */
- public fun meta(name: Name, meta: Meta)
-
-}
-
-/**
- * Define meta in this [DataSet]
- */
-public fun <T : Any> DataSetBuilder<T>.meta(value: Meta): Unit = meta(Name.EMPTY, value)
-
-/**
- * Define meta in this [DataSet]
- */
-public fun <T : Any> DataSetBuilder<T>.meta(mutableMeta: MutableMeta.() -> Unit): Unit = meta(Meta(mutableMeta))
-
-@PublishedApi
-internal class SubSetBuilder<in T : Any>(
- private val parent: DataSetBuilder<T>,
- private val branch: Name,
-) : DataSetBuilder<T> {
- override val dataType: KType get() = parent.dataType
-
- override fun remove(name: Name) {
- parent.remove(branch + name)
- }
-
- override fun data(name: Name, data: Data<T>?) {
- parent.data(branch + name, data)
- }
-
- override fun node(name: Name, dataSet: DataSet<T>) {
- parent.node(branch + name, dataSet)
- }
-
- override fun meta(name: Name, meta: Meta) {
- parent.meta(branch + name, meta)
- }
-}
-
-public inline fun <T : Any> DataSetBuilder<T>.node(
- name: Name,
- crossinline block: DataSetBuilder<T>.() -> Unit,
-) {
- if (name.isEmpty()) block() else SubSetBuilder(this, name).block()
-}
-
-
-public fun <T : Any> DataSetBuilder<T>.data(name: String, value: Data<T>) {
- data(Name.parse(name), value)
-}
-
-public fun <T : Any> DataSetBuilder<T>.node(name: String, set: DataSet<T>) {
- node(Name.parse(name), set)
-}
-
-public inline fun <T : Any> DataSetBuilder<T>.node(
- name: String,
- crossinline block: DataSetBuilder<T>.() -> Unit,
-): Unit = node(Name.parse(name), block)
-
-public fun <T : Any> DataSetBuilder<T>.set(value: NamedData<T>) {
- data(value.name, value.data)
-}
-
-/**
- * Produce lazy [Data] and emit it into the [DataSetBuilder]
- */
-public inline fun <reified T : Any> DataSetBuilder<T>.produce(
- name: String,
- meta: Meta = Meta.EMPTY,
- noinline producer: suspend () -> T,
-) {
- val data = Data(meta, block = producer)
- data(name, data)
-}
-
-public inline fun <reified T : Any> DataSetBuilder<T>.produce(
- name: Name,
- meta: Meta = Meta.EMPTY,
- noinline producer: suspend () -> T,
-) {
- val data = Data(meta, block = producer)
- data(name, data)
-}
-
-/**
- * Emit a static data with the fixed value
- */
-public inline fun <reified T : Any> DataSetBuilder<T>.static(
- name: String,
- data: T,
- meta: Meta = Meta.EMPTY,
-): Unit = data(name, Data.static(data, meta))
-
-public inline fun <reified T : Any> DataSetBuilder<T>.static(
- name: Name,
- data: T,
- meta: Meta = Meta.EMPTY,
-): Unit = data(name, Data.static(data, meta))
-
-public inline fun <reified T : Any> DataSetBuilder<T>.static(
- name: String,
- data: T,
- mutableMeta: MutableMeta.() -> Unit,
-): Unit = data(Name.parse(name), Data.static(data, Meta(mutableMeta)))
-
-/**
- * Update data with given node data and meta with node meta.
- */
-@DFExperimental
-public fun <T : Any> DataSetBuilder<T>.populateFrom(tree: DataSet<T>): Unit {
- tree.forEach {
- //TODO check if the place is occupied
- data(it.name, it.data)
- }
-}
-
-//public fun <T : Any> DataSetBuilder<T>.populateFrom(flow: Flow<NamedData<T>>) {
-// flow.collect {
-// data(it.name, it.data)
-// }
-//}
-
-public fun <T : Any> DataSetBuilder<T>.populateFrom(sequence: Sequence<NamedData<T>>) {
- sequence.forEach {
- data(it.name, it.data)
- }
-}
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataSink.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataSink.kt
new file mode 100644
index 00000000..4345866e
--- /dev/null
+++ b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataSink.kt
@@ -0,0 +1,133 @@
+package space.kscience.dataforge.data
+
+import kotlinx.coroutines.flow.Flow
+import kotlinx.coroutines.flow.MutableSharedFlow
+import kotlinx.coroutines.flow.mapNotNull
+import space.kscience.dataforge.misc.UnsafeKType
+import space.kscience.dataforge.names.*
+import kotlin.reflect.KType
+import kotlin.reflect.typeOf
+
+/**
+ * A marker scope for data builders
+ */
+public interface DataBuilderScope<in T> {
+ public companion object : DataBuilderScope<Nothing>
+}
+
+@Suppress("UNCHECKED_CAST")
+public fun <T> DataBuilderScope(): DataBuilderScope<T> = DataBuilderScope as DataBuilderScope<T>
+
+/**
+ * Asynchronous data sink
+ */
+public fun interface DataSink<in T> : DataBuilderScope<T> {
+ /**
+ * Put data and notify listeners if needed
+ */
+ public suspend fun write(name: Name, data: Data<T>?)
+}
+
+
+/**
+ * A mutable version of [DataTree]
+ */
+public interface MutableDataTree<T> : DataTree<T>, DataSink<T> {
+ override val items: Map<NameToken, MutableDataTree<T>>
+//
+// public fun getOrCreateItem(token: NameToken): MutableDataTree<T>
+//
+// public suspend fun put(token: NameToken, data: Data<T>?)
+//
+// override suspend fun put(name: Name, data: Data<T>?): Unit {
+// when (name.length) {
+// 0 -> this.data = data
+// 1 -> put(name.first(), data)
+// else -> getOrCreateItem(name.first()).put(name.cutFirst(), data)
+// }
+// }
+}
+
+/**
+ * Provide a mutable subtree if it exists
+ */
+public tailrec fun <T> MutableDataTree<T>.branch(name: Name): MutableDataTree<T>? =
+ when (name.length) {
+ 0 -> this
+ 1 -> items[name.first()]
+ else -> items[name.first()]?.branch(name.cutFirst())
+ }
+
+private class MutableDataTreeRoot<T>(
+ override val dataType: KType,
+) : MutableDataTree<T> {
+
+ override val items = HashMap<NameToken, MutableDataTree<T>>()
+ override val updates = MutableSharedFlow<Name>()
+
+ inner class MutableDataTreeBranch(val branchName: Name) : MutableDataTree<T> {
+
+ override var data: Data<T>? = null
+ private set
+
+ override val items = HashMap<NameToken, MutableDataTree<T>>()
+
+ override val updates: Flow<Name> = this@MutableDataTreeRoot.updates.mapNotNull { update ->
+ update.removeFirstOrNull(branchName)
+ }
+ override val dataType: KType get() = this@MutableDataTreeRoot.dataType
+
+ override suspend fun write(
+ name: Name,
+ data: Data<T>?
+ ) {
+ when (name.length) {
+ 0 -> {
+ this.data = data
+ this@MutableDataTreeRoot.updates.emit(branchName)
+ }
+
+ else -> {
+ val token = name.first()
+ items.getOrPut(token) { MutableDataTreeBranch(branchName + token) }.write(name.cutFirst(), data)
+ }
+ }
+ }
+ }
+ override var data: Data<T>? = null
+ private set
+
+ override suspend fun write(
+ name: Name,
+ data: Data<T>?
+ ) {
+ when (name.length) {
+ 0 -> {
+ this.data = data
+ this@MutableDataTreeRoot.updates.emit(Name.EMPTY)
+ }
+
+ else -> {
+ val token = name.first()
+ items.getOrPut(token) { MutableDataTreeBranch(token.asName()) }.write(name.cutFirst(), data)
+ }
+ }
+ }
+
+}
+
+/**
+ * Create a new [MutableDataTree]
+ */
+@UnsafeKType
+public fun <T> MutableDataTree(
+ type: KType,
+): MutableDataTree<T> = MutableDataTreeRoot<T>(type)
+
+/**
+ * Create and initialize an observable mutable data tree.
+ */
+@OptIn(UnsafeKType::class)
+public inline fun <reified T> MutableDataTree(
+ generator: MutableDataTree<T>.() -> Unit = {},
+): MutableDataTree<T> = MutableDataTree<T>(typeOf<T>()).apply { generator() }
\ No newline at end of file
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataSource.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataSource.kt
new file mode 100644
index 00000000..531d37fd
--- /dev/null
+++ b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataSource.kt
@@ -0,0 +1,139 @@
+package space.kscience.dataforge.data
+
+import kotlinx.coroutines.flow.*
+import space.kscience.dataforge.meta.Meta
+import space.kscience.dataforge.names.*
+import kotlin.contracts.contract
+import kotlin.reflect.KType
+import kotlin.reflect.typeOf
+
+/**
+ * A generic data provider
+ */
+public interface DataSource<out T> {
+
+ /**
+ * The minimal common ancestor to all data in the node
+ */
+ public val dataType: KType
+
+ /**
+ * Get data with given name. Or null if it is not present
+ */
+ public fun read(name: Name): Data<T>?
+}
+
+/**
+ * A data provider with possible dynamic updates
+ */
+public interface ObservableDataSource<out T> : DataSource<T> {
+
+ /**
+ * Names of updated elements.
+ * Data updates with the same names could be glued together.
+ *
+ * Updates are considered critical.
+ * The producer will suspend unless all updates are consumed.
+ */
+ public val updates: Flow<Name>
+}
+
+public suspend fun <T> ObservableDataSource<T>.awaitData(name: Name): Data<T> =
+ read(name) ?: updates.filter { it == name }.mapNotNull { read(name) }.first()
+
+public suspend fun <T> ObservableDataSource<T>.awaitData(name: String): Data<T> =
+ awaitData(name.parseAsName())
+
+/**
+ * A tree like structure for data holding
+ */
+public interface DataTree<out T> : ObservableDataSource<T> {
+
+ public val data: Data<T>?
+ public val items: Map<NameToken, DataTree<T>>
+
+ override fun read(name: Name): Data<T>? = when (name.length) {
+ 0 -> data
+ else -> items[name.first()]?.read(name.cutFirst())
+ }
+
+ /**
+ * Flow updates made to the data
+ */
+ override val updates: Flow<Name>
+
+ public companion object {
+ private object EmptyDataTree : DataTree<Nothing> {
+ override val data: Data<Nothing>? = null
+ override val items: Map<NameToken, EmptyDataTree> = emptyMap()
+ override val dataType: KType = typeOf<Unit>()
+
+ override fun read(name: Name): Data<Nothing>? = null
+ override val updates: Flow<Name> get() = emptyFlow()
+ }
+
+ public val EMPTY: DataTree<Nothing> = EmptyDataTree
+ }
+}
+
+/**
+ * An alias for easier access to tree values
+ */
+public operator fun <T> DataTree<T>.get(name: Name): Data<T>? = read(name)
+
+public operator fun <T> DataTree<T>.get(name: String): Data<T>? = read(name.parseAsName())
+
+/**
+ * Return a sequence of all data items in this tree.
+ * This method does not take updates into account.
+ */
+public fun <T> DataTree<T>.asSequence(
+ namePrefix: Name = Name.EMPTY,
+): Sequence<NamedData<T>> = sequence {
+ data?.let { yield(it.named(namePrefix)) }
+ items.forEach { (token, tree) ->
+ yieldAll(tree.asSequence(namePrefix + token))
+ }
+}
+
+/**
+ * Walk the data tree depth-first.
+ *
+ * @return a [Sequence] of pairs [Name]-[DataTree] for all nodes including the root one.
+ */
+public fun <T> DataTree<T>.walk(
+ namePrefix: Name = Name.EMPTY,
+): Sequence<Pair<Name, DataTree<T>>> = sequence {
+ yield(namePrefix to this@walk)
+ items.forEach { (token, tree) ->
+ yieldAll(tree.walk(namePrefix + token))
+ }
+}
+
+public val DataTree<*>.meta: Meta? get() = data?.meta
+
+/**
+ * Provide subtree if it exists
+ */
+public tailrec fun <T> DataTree<T>.branch(name: Name): DataTree<T>? =
+ when (name.length) {
+ 0 -> this
+ 1 -> items[name.first()]
+ else -> items[name.first()]?.branch(name.cutFirst())
+ }
+
+public fun <T> DataTree<T>.branch(name: String): DataTree<T>? =
+ branch(name.parseAsName())
+
+public fun DataTree<*>.isEmpty(): Boolean = data == null && items.isEmpty()
+
+/**
+ * Check if the [DataTree] is observable
+ */
+public fun <T> DataSource<T>.isObservable(): Boolean {
+ contract {
+ returns(true) implies (this@isObservable is ObservableDataSource<T>)
+ }
+ return this is ObservableDataSource<T>
+}
+
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataTree.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataTree.kt
deleted file mode 100644
index bafcbea2..00000000
--- a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataTree.kt
+++ /dev/null
@@ -1,119 +0,0 @@
-package space.kscience.dataforge.data
-
-import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.misc.DFInternal
-import space.kscience.dataforge.misc.DfId
-import space.kscience.dataforge.names.*
-import kotlin.collections.component1
-import kotlin.collections.component2
-import kotlin.reflect.KType
-import kotlin.reflect.typeOf
-
-public sealed class DataTreeItem<out T : Any> {
-
- public abstract val meta: Meta
-
- public class Node<out T : Any>(public val tree: DataTree<T>) : DataTreeItem<T>() {
- override val meta: Meta get() = tree.meta
- }
-
- public class Leaf<out T : Any>(public val data: Data<T>) : DataTreeItem<T>() {
- override val meta: Meta get() = data.meta
- }
-}
-
-public val <T : Any> DataTreeItem<T>.type: KType
- get() = when (this) {
- is DataTreeItem.Node -> tree.dataType
- is DataTreeItem.Leaf -> data.type
- }
-
-/**
- * A tree-like [DataSet] grouped into the node. All data inside the node must inherit its type
- */
-@DfId(DataTree.TYPE)
-public interface DataTree<out T : Any> : DataSet<T> {
-
- /**
- * Top-level children items of this [DataTree]
- */
- public val items: Map<NameToken, DataTreeItem<T>>
-
- override val meta: Meta get() = items[META_ITEM_NAME_TOKEN]?.meta ?: Meta.EMPTY
-
- override fun iterator(): Iterator<NamedData<T>> = iterator {
- items.forEach { (token, childItem: DataTreeItem<T>) ->
- if (!token.body.startsWith("@")) {
- when (childItem) {
- is DataTreeItem.Leaf -> yield(childItem.data.named(token.asName()))
- is DataTreeItem.Node -> yieldAll(childItem.tree.asSequence().map { it.named(token + it.name) })
- }
- }
- }
- }
-
- override fun get(name: Name): Data<T>? = when (name.length) {
- 0 -> null
- 1 -> items[name.firstOrNull()!!].data
- else -> items[name.firstOrNull()!!].tree?.get(name.cutFirst())
- }
-
- public companion object {
- public const val TYPE: String = "dataTree"
-
- /**
- * A name token used to designate tree node meta
- */
- public val META_ITEM_NAME_TOKEN: NameToken = NameToken("@meta")
-
- @DFInternal
- public fun <T : Any> emptyWithType(type: KType, meta: Meta = Meta.EMPTY): DataTree<T> = object : DataTree<T> {
- override val items: Map<NameToken, DataTreeItem<T>> get() = emptyMap()
- override val dataType: KType get() = type
- override val meta: Meta get() = meta
- }
-
- @OptIn(DFInternal::class)
- public inline fun <reified T : Any> empty(meta: Meta = Meta.EMPTY): DataTree<T> =
- emptyWithType<T>(typeOf<T>(), meta)
- }
-}
-
-public fun <T : Any> DataTree<T>.listChildren(prefix: Name): List<Name> =
- getItem(prefix).tree?.items?.keys?.map { prefix + it } ?: emptyList()
-
-/**
- * Get a [DataTreeItem] with given [name] or null if the item does not exist
- */
-public tailrec fun <T : Any> DataTree<T>.getItem(name: Name): DataTreeItem<T>? = when (name.length) {
- 0 -> DataTreeItem.Node(this)
- 1 -> items[name.firstOrNull()]
- else -> items[name.firstOrNull()!!].tree?.getItem(name.cutFirst())
-}
-
-public val <T : Any> DataTreeItem<T>?.tree: DataTree<T>? get() = (this as? DataTreeItem.Node<T>)?.tree
-public val <T : Any> DataTreeItem<T>?.data: Data<T>? get() = (this as? DataTreeItem.Leaf<T>)?.data
-
-/**
- * A [Sequence] of all children including nodes
- */
-public fun <T : Any> DataTree<T>.traverseItems(): Sequence<Pair<Name, DataTreeItem<T>>> = sequence {
- items.forEach { (head, item) ->
- yield(head.asName() to item)
- if (item is DataTreeItem.Node) {
- val subSequence = item.tree.traverseItems()
- .map { (name, data) -> (head.asName() + name) to data }
- yieldAll(subSequence)
- }
- }
-}
-
-/**
- * Get a branch of this [DataTree] with a given [branchName].
- * The difference from similar method for [DataSet] is that internal logic is more simple and the return value is a [DataTree]
- */
-@OptIn(DFInternal::class)
-public fun <T : Any> DataTree<T>.branch(branchName: Name): DataTree<T> =
- getItem(branchName)?.tree ?: DataTree.emptyWithType(dataType)
-
-public fun <T : Any> DataTree<T>.branch(branchName: String): DataTree<T> = branch(branchName.parseAsName())
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataTreeBuilder.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataTreeBuilder.kt
deleted file mode 100644
index 303ba44e..00000000
--- a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/DataTreeBuilder.kt
+++ /dev/null
@@ -1,127 +0,0 @@
-package space.kscience.dataforge.data
-
-import kotlinx.coroutines.CoroutineScope
-import kotlinx.coroutines.Job
-import kotlinx.coroutines.flow.MutableSharedFlow
-import kotlinx.coroutines.launch
-import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.misc.DFInternal
-import space.kscience.dataforge.misc.ThreadSafe
-import space.kscience.dataforge.names.*
-import kotlin.collections.set
-import kotlin.coroutines.CoroutineContext
-import kotlin.coroutines.coroutineContext
-import kotlin.reflect.KType
-import kotlin.reflect.typeOf
-
-public interface DataSourceBuilder<T : Any> : DataSetBuilder<T>, DataSource<T> {
- override val updates: MutableSharedFlow<Name>
-}
-
-/**
- * A mutable [DataTree] that propagates updates
- */
-public class DataTreeBuilder<T : Any>(
- override val dataType: KType,
- coroutineContext: CoroutineContext,
-) : DataTree<T>, DataSourceBuilder<T> {
-
- override val coroutineContext: CoroutineContext =
- coroutineContext + Job(coroutineContext[Job]) + GoalExecutionRestriction()
-
- private val treeItems = HashMap<NameToken, DataTreeItem<T>>()
-
- override val items: Map<NameToken, DataTreeItem<T>>
- get() = treeItems.filter { !it.key.body.startsWith("@") }
-
- override val updates: MutableSharedFlow<Name> = MutableSharedFlow<Name>()
-
- @ThreadSafe
- private fun remove(token: NameToken) {
- if (treeItems.remove(token) != null) {
- launch {
- updates.emit(token.asName())
- }
- }
- }
-
- override fun remove(name: Name) {
- if (name.isEmpty()) error("Can't remove the root node")
- (getItem(name.cutLast()).tree as? DataTreeBuilder)?.remove(name.lastOrNull()!!)
- }
-
- @ThreadSafe
- private fun set(token: NameToken, data: Data<T>) {
- treeItems[token] = DataTreeItem.Leaf(data)
- }
-
- @ThreadSafe
- private fun set(token: NameToken, node: DataTree<T>) {
- treeItems[token] = DataTreeItem.Node(node)
- }
-
- private fun getOrCreateNode(token: NameToken): DataTreeBuilder<T> =
- (treeItems[token] as? DataTreeItem.Node<T>)?.tree as? DataTreeBuilder<T>
- ?: DataTreeBuilder<T>(dataType, coroutineContext).also { set(token, it) }
-
- private fun getOrCreateNode(name: Name): DataTreeBuilder<T> = when (name.length) {
- 0 -> this
- 1 -> getOrCreateNode(name.firstOrNull()!!)
- else -> getOrCreateNode(name.firstOrNull()!!).getOrCreateNode(name.cutFirst())
- }
-
- override fun data(name: Name, data: Data<T>?) {
- if (data == null) {
- remove(name)
- } else {
- when (name.length) {
- 0 -> error("Can't add data with empty name")
- 1 -> set(name.firstOrNull()!!, data)
- 2 -> getOrCreateNode(name.cutLast()).set(name.lastOrNull()!!, data)
- }
- }
- launch {
- updates.emit(name)
- }
- }
-
- override fun meta(name: Name, meta: Meta) {
- val item = getItem(name)
- if (item is DataTreeItem.Leaf) error("TODO: Can't change meta of existing leaf item.")
- data(name + DataTree.META_ITEM_NAME_TOKEN, Data.empty(meta))
- }
-}
-
-/**
- * Create a dynamic [DataSource]. Initial data is placed synchronously.
- */
-@DFInternal
-@Suppress("FunctionName")
-public fun <T : Any> DataSource(
- type: KType,
- parent: CoroutineScope,
- block: DataSourceBuilder<T>.() -> Unit,
-): DataTreeBuilder<T> = DataTreeBuilder<T>(type, parent.coroutineContext).apply(block)
-
-@Suppress("OPT_IN_USAGE", "FunctionName")
-public inline fun <reified T : Any> DataSource(
- parent: CoroutineScope,
- crossinline block: DataSourceBuilder<T>.() -> Unit,
-): DataTreeBuilder<T> = DataSource(typeOf<T>(), parent) { block() }
-
-@Suppress("FunctionName")
-public suspend inline fun <reified T : Any> DataSource(
- crossinline block: DataSourceBuilder<T>.() -> Unit = {},
-): DataTreeBuilder<T> = DataTreeBuilder<T>(typeOf<T>(), coroutineContext).apply { block() }
-
-public inline fun <reified T : Any> DataSourceBuilder<T>.emit(
- name: Name,
- parent: CoroutineScope,
- noinline block: DataSourceBuilder<T>.() -> Unit,
-): Unit = node(name, DataSource(parent, block))
-
-public inline fun <reified T : Any> DataSourceBuilder<T>.emit(
- name: String,
- parent: CoroutineScope,
- noinline block: DataSourceBuilder<T>.() -> Unit,
-): Unit = node(Name.parse(name), DataSource(parent, block))
\ No newline at end of file
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/Goal.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/Goal.kt
index 678711c1..e9b022b2 100644
--- a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/Goal.kt
+++ b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/Goal.kt
@@ -9,7 +9,7 @@ import kotlin.coroutines.EmptyCoroutineContext
* Lazy computation result with its dependencies to allowing to stat computing dependencies ahead of time
*/
public interface Goal<out T> {
- public val dependencies: Collection<Goal<*>>
+ public val dependencies: Iterable<Goal<*>>
/**
* Returns current running coroutine if the goal is started. Null if the computation is not started.
@@ -32,7 +32,7 @@ public interface Goal<out T> {
public companion object
}
-public fun Goal<*>.launch(coroutineScope: CoroutineScope): Job = async(coroutineScope)
+public fun Goal<*>.launchIn(coroutineScope: CoroutineScope): Job = async(coroutineScope)
public suspend fun <T> Goal<T>.await(): T = coroutineScope { async(this).await() }
@@ -54,7 +54,7 @@ public open class StaticGoal<T>(public val value: T) : Goal<T> {
*/
public open class LazyGoal<T>(
private val coroutineContext: CoroutineContext = EmptyCoroutineContext,
- override val dependencies: Collection<Goal<*>> = emptyList(),
+ override val dependencies: Iterable<Goal<*>> = emptyList(),
public val block: suspend () -> T,
) : Goal<T> {
@@ -64,11 +64,14 @@ public open class LazyGoal<T>(
/**
* Get ongoing computation or start a new one.
* Does not guarantee thread safety. In case of multi-thread access, could create orphan computations.
- * If [GoalExecutionRestriction] is present in the [coroutineScope] context, the call could produce a error a warning
+ * If [GoalExecutionRestriction] is present in the [coroutineScope] context, the call could produce an error or a warning
* depending on the settings.
+ *
+ * If [Goal] is already started on a different scope, it is not restarted.
*/
@OptIn(DFExperimental::class)
- override fun async(coroutineScope: CoroutineScope): Deferred<T> {
+ override fun async(coroutineScope: CoroutineScope): Deferred<T> = deferred ?: run {
+
val log = coroutineScope.coroutineContext[GoalLogger]
// Check if context restricts goal computation
coroutineScope.coroutineContext[GoalExecutionRestriction]?.let { restriction ->
@@ -82,16 +85,17 @@ public open class LazyGoal<T>(
}
log?.emit { "Starting dependencies computation for ${this@LazyGoal}" }
- val startedDependencies = this.dependencies.map { goal ->
- goal.run { async(coroutineScope) }
+ val startedDependencies = dependencies.map { goal ->
+ goal.async(coroutineScope)
}
- return deferred ?: coroutineScope.async(
+
+ coroutineScope.async(
coroutineContext
+ CoroutineMonitor()
+ Dependencies(startedDependencies)
+ GoalExecutionRestriction(GoalExecutionRestrictionPolicy.NONE) // Remove restrictions on goal execution
) {
- //cancel execution if error encountered in one of dependencies
+ //cancel execution if error encountered in one of the dependencies
startedDependencies.forEach { deferred ->
deferred.invokeOnCompletion { error ->
if (error != null) this.cancel(CancellationException("Dependency $deferred failed with error: ${error.message}"))
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/GroupRule.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/GroupRule.kt
deleted file mode 100644
index 189087a3..00000000
--- a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/GroupRule.kt
+++ /dev/null
@@ -1,78 +0,0 @@
-/*
- * Copyright 2015 Alexander Nozik.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package space.kscience.dataforge.data
-
-import kotlinx.coroutines.launch
-import space.kscience.dataforge.meta.get
-import space.kscience.dataforge.meta.string
-import space.kscience.dataforge.misc.DFInternal
-
-public interface GroupRule {
- public fun <T : Any> gather(set: DataSet<T>): Map<String, DataSet<T>>
-
- public companion object {
- /**
- * Create grouping rule that creates groups for different values of value
- * field with name [key]
- *
- * @param key
- * @param defaultTagValue
- * @return
- */
- @OptIn(DFInternal::class)
- public fun byMetaValue(
- key: String,
- defaultTagValue: String,
- ): GroupRule = object : GroupRule {
-
- override fun <T : Any> gather(
- set: DataSet<T>,
- ): Map<String, DataSet<T>> {
- val map = HashMap<String, DataSet<T>>()
-
- if (set is DataSource) {
- set.forEach { data ->
- val tagValue: String = data.meta[key]?.string ?: defaultTagValue
- (map.getOrPut(tagValue) { DataTreeBuilder(set.dataType, set.coroutineContext) } as DataTreeBuilder<T>)
- .data(data.name, data.data)
-
- set.launch {
- set.updates.collect { name ->
- val dataUpdate = set[name]
-
- val updateTagValue = dataUpdate?.meta?.get(key)?.string ?: defaultTagValue
- map.getOrPut(updateTagValue) {
- DataSource(set.dataType, this) {
- data(name, dataUpdate)
- }
- }
- }
- }
- }
- } else {
- set.forEach { data ->
- val tagValue: String = data.meta[key]?.string ?: defaultTagValue
- (map.getOrPut(tagValue) { StaticDataTree(set.dataType) } as StaticDataTree<T>)
- .data(data.name, data.data)
- }
- }
-
-
- return map
- }
- }
- }
-}
\ No newline at end of file
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/MetaMaskData.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/MetaMaskData.kt
new file mode 100644
index 00000000..2fbece21
--- /dev/null
+++ b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/MetaMaskData.kt
@@ -0,0 +1,23 @@
+package space.kscience.dataforge.data
+
+import space.kscience.dataforge.meta.Meta
+import space.kscience.dataforge.meta.MutableMeta
+import space.kscience.dataforge.meta.copy
+
+
+private class MetaMaskData<T>(val origin: Data<T>, override val meta: Meta) : Data<T> by origin
+
+/**
+ * A data with overridden meta. It reflects original data computed state.
+ */
+public fun <T> Data<T>.withMeta(newMeta: Meta): Data<T> = if (this is MetaMaskData) {
+ MetaMaskData(origin, newMeta)
+} else {
+ MetaMaskData(this, newMeta)
+}
+
+/**
+ * Create a new [Data] with the same computation, but different meta. The meta is created by applying [block] to
+ * the existing data meta.
+ */
+public inline fun <T> Data<T>.withMeta(block: MutableMeta.() -> Unit): Data<T> = withMeta(meta.copy(block))
\ No newline at end of file
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/NamedData.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/NamedData.kt
index 4c9d4bb3..b20736ae 100644
--- a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/NamedData.kt
+++ b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/NamedData.kt
@@ -4,17 +4,15 @@ import space.kscience.dataforge.meta.isEmpty
import space.kscience.dataforge.misc.Named
import space.kscience.dataforge.names.Name
-public interface NamedData<out T : Any> : Named, Data<T> {
- override val name: Name
- public val data: Data<T>
-}
+/**
+ * A data coupled to a name.
+ */
+public interface NamedData<out T> : Data<T>, Named
-public operator fun NamedData<*>.component1(): Name = name
-public operator fun <T: Any> NamedData<T>.component2(): Data<T> = data
-private class NamedDataImpl<out T : Any>(
+private class NamedDataImpl<T>(
override val name: Name,
- override val data: Data<T>,
+ val data: Data<T>,
) : Data<T> by data, NamedData<T> {
override fun toString(): String = buildString {
append("NamedData(name=\"$name\"")
@@ -28,8 +26,10 @@ private class NamedDataImpl<out T : Any>(
}
}
-public fun <T : Any> Data<T>.named(name: Name): NamedData<T> = if (this is NamedData) {
- NamedDataImpl(name, this.data)
+public fun <T> Data<T>.named(name: Name): NamedData<T> = if (this is NamedData) {
+ NamedDataImpl(name, this)
} else {
NamedDataImpl(name, this)
-}
\ No newline at end of file
+}
+
+public fun <T> NamedData(name: Name, data: Data<T>): NamedData<T> = data.named(name)
\ No newline at end of file
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/StaticDataBuilder.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/StaticDataBuilder.kt
new file mode 100644
index 00000000..54bcf19c
--- /dev/null
+++ b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/StaticDataBuilder.kt
@@ -0,0 +1,63 @@
+package space.kscience.dataforge.data
+
+import space.kscience.dataforge.meta.Meta
+import space.kscience.dataforge.meta.MutableMeta
+import space.kscience.dataforge.misc.UnsafeKType
+import space.kscience.dataforge.names.Name
+import space.kscience.dataforge.names.parseAsName
+import space.kscience.dataforge.names.plus
+import kotlin.reflect.KType
+import kotlin.reflect.typeOf
+
+
+public fun interface StaticDataBuilder<T> : DataBuilderScope<T> {
+ public fun data(name: Name, data: Data<T>)
+}
+
+private class DataMapBuilder<T> : StaticDataBuilder<T> {
+ val map = mutableMapOf<Name, Data<T>>()
+
+ override fun data(name: Name, data: Data<T>) {
+ if (map.containsKey(name)) {
+ error("Duplicate key '$name'")
+ } else {
+ map.put(name, data)
+ }
+ }
+}
+
+public fun <T> StaticDataBuilder<T>.data(name: String, data: Data<T>) {
+ data(name.parseAsName(), data)
+}
+
+public inline fun <T, reified T1 : T> StaticDataBuilder<T>.value(
+ name: String,
+ value: T1,
+ metaBuilder: MutableMeta.() -> Unit = {}
+) {
+ data(name, Data(value, Meta(metaBuilder)))
+}
+
+public fun <T> StaticDataBuilder<T>.node(prefix: Name, block: StaticDataBuilder<T>.() -> Unit) {
+ val map = DataMapBuilder<T>().apply(block).map
+ map.forEach { (name, data) ->
+ data(prefix + name, data)
+ }
+}
+
+public fun <T> StaticDataBuilder<T>.node(prefix: String, block: StaticDataBuilder<T>.() -> Unit) =
+ node(prefix.parseAsName(), block)
+
+public fun <T> StaticDataBuilder<T>.node(prefix: String, tree: DataTree<T>) {
+ tree.forEach { data ->
+ data(prefix.parseAsName() + data.name, data)
+ }
+}
+
+@UnsafeKType
+public fun <T> DataTree.Companion.static(type: KType, block: StaticDataBuilder<T>.() -> Unit): DataTree<T> =
+ DataMapBuilder<T>().apply(block).map.asTree(type)
+
+@OptIn(UnsafeKType::class)
+public inline fun <reified T> DataTree.Companion.static(noinline block: StaticDataBuilder<T>.() -> Unit): DataTree<T> =
+ static(typeOf<T>(), block)
\ No newline at end of file
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/StaticDataTree.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/StaticDataTree.kt
deleted file mode 100644
index 4f0f455e..00000000
--- a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/StaticDataTree.kt
+++ /dev/null
@@ -1,82 +0,0 @@
-package space.kscience.dataforge.data
-
-import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.misc.DFExperimental
-import space.kscience.dataforge.names.*
-import kotlin.reflect.KType
-import kotlin.reflect.typeOf
-
-@PublishedApi
-internal class StaticDataTree<T : Any>(
- override val dataType: KType,
-) : DataSetBuilder<T>, DataTree<T> {
-
- private val _items: MutableMap<NameToken, DataTreeItem<T>> = HashMap()
-
- override val items: Map<NameToken, DataTreeItem<T>>
- get() = _items.filter { !it.key.body.startsWith("@") }
-
- override fun remove(name: Name) {
- when (name.length) {
- 0 -> error("Can't remove root tree node")
- 1 -> _items.remove(name.firstOrNull()!!)
- else -> (_items[name.firstOrNull()!!].tree as? StaticDataTree<T>)?.remove(name.cutFirst())
- }
- }
-
- private fun getOrCreateNode(name: Name): StaticDataTree<T> = when (name.length) {
- 0 -> this
- 1 -> {
- val itemName = name.firstOrNull()!!
- (_items[itemName].tree as? StaticDataTree<T>) ?: StaticDataTree<T>(dataType).also {
- _items[itemName] = DataTreeItem.Node(it)
- }
- }
- else -> getOrCreateNode(name.cutLast()).getOrCreateNode(name.lastOrNull()!!.asName())
- }
-
- private fun set(name: Name, item: DataTreeItem<T>?) {
- if (name.isEmpty()) error("Can't set top level tree node")
- if (item == null) {
- remove(name)
- } else {
- getOrCreateNode(name.cutLast())._items[name.lastOrNull()!!] = item
- }
- }
-
- override fun data(name: Name, data: Data<T>?) {
- set(name, data?.let { DataTreeItem.Leaf(it) })
- }
-
- override fun node(name: Name, dataSet: DataSet<T>) {
- if (dataSet is StaticDataTree) {
- set(name, DataTreeItem.Node(dataSet))
- } else {
- dataSet.forEach {
- data(name + it.name, it.data)
- }
- }
- }
-
- override fun meta(name: Name, meta: Meta) {
- val item = getItem(name)
- if (item is DataTreeItem.Leaf) TODO("Can't change meta of existing leaf item.")
- data(name + DataTree.META_ITEM_NAME_TOKEN, Data.empty(meta))
- }
-}
-
-@Suppress("FunctionName")
-public inline fun <T : Any> DataTree(
- dataType: KType,
- block: DataSetBuilder<T>.() -> Unit,
-): DataTree<T> = StaticDataTree<T>(dataType).apply { block() }
-
-@Suppress("FunctionName")
-public inline fun <reified T : Any> DataTree(
- noinline block: DataSetBuilder<T>.() -> Unit,
-): DataTree<T> = DataTree(typeOf<T>(), block)
-
-@OptIn(DFExperimental::class)
-public fun <T : Any> DataSet<T>.seal(): DataTree<T> = DataTree(dataType) {
- populateFrom(this@seal)
-}
\ No newline at end of file
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/dataBuilders.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/dataBuilders.kt
new file mode 100644
index 00000000..80c5fca9
--- /dev/null
+++ b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/dataBuilders.kt
@@ -0,0 +1,113 @@
+package space.kscience.dataforge.data
+
+import space.kscience.dataforge.meta.Meta
+import space.kscience.dataforge.meta.MutableMeta
+import space.kscience.dataforge.names.Name
+import space.kscience.dataforge.names.asName
+import space.kscience.dataforge.names.isEmpty
+import space.kscience.dataforge.names.plus
+
+
+public suspend fun <T> DataSink<T>.write(value: NamedData<T>) {
+ write(value.name, value)
+}
+
+public inline fun <T> DataSink<T>.writeAll(
+ prefix: Name,
+ block: DataSink<T>.() -> Unit,
+) {
+ if (prefix.isEmpty()) {
+ apply(block)
+ } else {
+ val proxyDataSink = DataSink<T> { name, data -> this@writeAll.write(prefix + name, data) }
+
+ proxyDataSink.apply(block)
+ }
+}
+
+
+public inline fun <T> DataSink<T>.writeAll(
+ prefix: String,
+ block: DataSink<T>.() -> Unit,
+): Unit = writeAll(prefix.asName(), block)
+
+
+public suspend fun <T> DataSink<T>.write(name: String, value: Data<T>) {
+ write(Name.parse(name), value)
+}
+
+public suspend fun <T> DataSink<T>.writeAll(name: Name, tree: DataTree<T>) {
+ writeAll(name) { writeAll(tree.asSequence()) }
+}
+
+
+public suspend fun <T> DataSink<T>.writeAll(name: String, tree: DataTree<T>) {
+ writeAll(Name.parse(name)) { writeAll(tree.asSequence()) }
+}
+
+/**
+ * Produce lazy [Data] and emit it into the [MutableDataTree]
+ */
+public suspend inline fun <reified T> DataSink<T>.writeValue(
+ name: String,
+ meta: Meta = Meta.EMPTY,
+ noinline producer: suspend () -> T,
+) {
+ val data = Data(meta, block = producer)
+ write(name, data)
+}
+
+public suspend inline fun <reified T> DataSink<T>.writeValue(
+ name: Name,
+ meta: Meta = Meta.EMPTY,
+ noinline producer: suspend () -> T,
+) {
+ val data = Data(meta, block = producer)
+ write(name, data)
+}
+
+/**
+ * Emit static data with the fixed value
+ */
+public suspend inline fun <reified T> DataSink<T>.writeValue(
+ name: Name,
+ value: T,
+ meta: Meta = Meta.EMPTY,
+): Unit = write(name, Data.wrapValue(value, meta))
+
+public suspend inline fun <reified T> DataSink<T>.writeValue(
+ name: String,
+ value: T,
+ meta: Meta = Meta.EMPTY,
+): Unit = write(name, Data.wrapValue(value, meta))
+
+public suspend inline fun <reified T> DataSink<T>.writeValue(
+ name: String,
+ value: T,
+ metaBuilder: MutableMeta.() -> Unit,
+): Unit = write(Name.parse(name), Data.wrapValue(value, Meta(metaBuilder)))
+
+public suspend fun <T> DataSink<T>.writeAll(sequence: Sequence<NamedData<T>>) {
+ sequence.forEach {
+ write(it)
+ }
+}
+
+public suspend fun <T> DataSink<T>.writeAll(map: Map<Name, Data<T>?>) {
+ map.forEach { (name, data) ->
+ write(name, data)
+ }
+}
+
+/**
+ * Copy all data from [this] and mirror changes if they appear. Suspends indefinitely.
+ */
+public suspend fun <T : Any> MutableDataTree<T>.writeAllAndWatch(
+ source: DataTree<T>,
+ prefix: Name = Name.EMPTY,
+) {
+ writeAll(prefix, source)
+ source.updates.collect {
+ write(prefix + it, source.read(it))
+ }
+}
\ No newline at end of file
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/dataFilter.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/dataFilter.kt
deleted file mode 100644
index 3d2b6537..00000000
--- a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/dataFilter.kt
+++ /dev/null
@@ -1,105 +0,0 @@
-package space.kscience.dataforge.data
-
-import kotlinx.coroutines.flow.Flow
-import kotlinx.coroutines.flow.filter
-import kotlinx.coroutines.flow.map
-import kotlinx.coroutines.flow.mapNotNull
-import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.misc.DFExperimental
-import space.kscience.dataforge.names.*
-import kotlin.coroutines.CoroutineContext
-import kotlin.coroutines.EmptyCoroutineContext
-import kotlin.reflect.KType
-
-
-/**
- * A stateless filtered [DataSet]
- */
-public fun <T : Any> DataSet<T>.filter(
- predicate: (Name, Meta) -> Boolean,
-): DataSource<T> = object : DataSource<T> {
-
- override val dataType: KType get() = this@filter.dataType
-
- override val coroutineContext: CoroutineContext
- get() = (this@filter as? DataSource)?.coroutineContext ?: EmptyCoroutineContext
-
-
- override val meta: Meta get() = this@filter.meta
-
- override fun iterator(): Iterator<NamedData<T>> = iterator {
- for (d in this@filter) {
- if (predicate(d.name, d.meta)) {
- yield(d)
- }
- }
- }
-
- override fun get(name: Name): Data<T>? = this@filter.get(name)?.takeIf {
- predicate(name, it.meta)
- }
-
- override val updates: Flow<Name> = this@filter.updates.filter flowFilter@{ name ->
- val theData = this@filter[name] ?: return@flowFilter false
- predicate(name, theData.meta)
- }
-}
-
-/**
- * Generate a wrapper data set with a given name prefix appended to all names
- */
-public fun <T : Any> DataSet<T>.withNamePrefix(prefix: Name): DataSet<T> = if (prefix.isEmpty()) {
- this
-} else object : DataSource<T> {
-
- override val dataType: KType get() = this@withNamePrefix.dataType
-
- override val coroutineContext: CoroutineContext
- get() = (this@withNamePrefix as? DataSource)?.coroutineContext ?: EmptyCoroutineContext
-
- override val meta: Meta get() = this@withNamePrefix.meta
-
-
- override fun iterator(): Iterator<NamedData<T>> = iterator {
- for (d in this@withNamePrefix) {
- yield(d.data.named(prefix + d.name))
- }
- }
-
- override fun get(name: Name): Data<T>? =
- name.removeFirstOrNull(name)?.let { this@withNamePrefix.get(it) }
-
- override val updates: Flow<Name> get() = this@withNamePrefix.updates.map { prefix + it }
-}
-
-/**
- * Get a subset of data starting with a given [branchName]
- */
-public fun <T : Any> DataSet<T>.branch(branchName: Name): DataSet<T> = if (branchName.isEmpty()) {
- this
-} else object : DataSource<T> {
- override val dataType: KType get() = this@branch.dataType
-
- override val coroutineContext: CoroutineContext
- get() = (this@branch as? DataSource)?.coroutineContext ?: EmptyCoroutineContext
-
- override val meta: Meta get() = this@branch.meta
-
- override fun iterator(): Iterator<NamedData<T>> = iterator {
- for (d in this@branch) {
- d.name.removeFirstOrNull(branchName)?.let { name ->
- yield(d.data.named(name))
- }
- }
- }
-
- override fun get(name: Name): Data<T>? = this@branch.get(branchName + name)
-
- override val updates: Flow<Name> get() = this@branch.updates.mapNotNull { it.removeFirstOrNull(branchName) }
-}
-
-public fun <T : Any> DataSet<T>.branch(branchName: String): DataSet<T> = this@branch.branch(branchName.parseAsName())
-
-@DFExperimental
-public suspend fun <T : Any> DataSet<T>.rootData(): Data<T>? = get(Name.EMPTY)
-
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/dataTransform.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/dataTransform.kt
index 76577346..868cb82f 100644
--- a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/dataTransform.kt
+++ b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/dataTransform.kt
@@ -1,24 +1,40 @@
package space.kscience.dataforge.data
-import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.meta.MutableMeta
-import space.kscience.dataforge.meta.seal
-import space.kscience.dataforge.meta.toMutableMeta
-import space.kscience.dataforge.misc.DFInternal
+import kotlinx.coroutines.CoroutineScope
+import space.kscience.dataforge.meta.*
+import space.kscience.dataforge.misc.UnsafeKType
import space.kscience.dataforge.names.Name
+import space.kscience.dataforge.names.NameToken
import kotlin.coroutines.CoroutineContext
import kotlin.coroutines.EmptyCoroutineContext
import kotlin.reflect.KType
import kotlin.reflect.typeOf
-public data class ValueWithMeta<T>(val meta: Meta, val value: T)
+public data class ValueWithMeta<T>(val value: T, val meta: Meta)
-public suspend fun <T : Any> Data<T>.awaitWithMeta(): ValueWithMeta<T> = ValueWithMeta(meta, await())
+public suspend fun <T> Data<T>.awaitWithMeta(): ValueWithMeta<T> = ValueWithMeta(await(), meta)
-public data class NamedValueWithMeta<T>(val name: Name, val meta: Meta, val value: T)
+public data class NamedValueWithMeta<T>(val name: Name, val value: T, val meta: Meta)
-public suspend fun <T : Any> NamedData<T>.awaitWithMeta(): NamedValueWithMeta<T> =
- NamedValueWithMeta(name, meta, await())
+public suspend fun <T> NamedData<T>.awaitWithMeta(): NamedValueWithMeta<T> =
+ NamedValueWithMeta(name, await(), meta)
+
+/**
+ * Lazily transform this data to another data. By convention [block] should not use external data (be pure).
+ * @param type explicit type of the resulting [Data]
+ * @param coroutineContext additional [CoroutineContext] elements used for data computation.
+ * @param meta for the resulting data. By default equals input data.
+ * @param block the transformation itself
+ */
+@UnsafeKType
+public fun <T, R> Data<T>.transform(
+ type: KType,
+ meta: Meta = this.meta,
+ coroutineContext: CoroutineContext = EmptyCoroutineContext,
+ block: suspend (T) -> R,
+): Data<R> = Data(type, meta, coroutineContext, listOf(this)) {
+ block(await())
+}
/**
@@ -27,9 +43,9 @@ public suspend fun <T : Any> NamedData<T>.awaitWithMeta(): NamedValueWithMeta<T>
* @param meta for the resulting data. By default equals input data.
* @param block the transformation itself
*/
-public inline fun <T : Any, reified R : Any> Data<T>.map(
- coroutineContext: CoroutineContext = EmptyCoroutineContext,
+public inline fun <T, reified R> Data<T>.transform(
meta: Meta = this.meta,
+ coroutineContext: CoroutineContext = EmptyCoroutineContext,
crossinline block: suspend (T) -> R,
): Data<R> = Data(meta, coroutineContext, listOf(this)) {
block(await())
@@ -38,10 +54,10 @@ public inline fun <T : Any, reified R : Any> Data<T>.map(
/**
* Combine this data with the other data using [block]. See [Data::map] for other details
*/
-public inline fun <T1 : Any, T2 : Any, reified R : Any> Data<T1>.combine(
+public inline fun <T1, T2, reified R> Data<T1>.combine(
other: Data<T2>,
- coroutineContext: CoroutineContext = EmptyCoroutineContext,
meta: Meta = this.meta,
+ coroutineContext: CoroutineContext = EmptyCoroutineContext,
crossinline block: suspend (left: T1, right: T2) -> R,
): Data<R> = Data(meta, coroutineContext, listOf(this, other)) {
block(await(), other.await())
@@ -50,26 +66,30 @@ public inline fun <T1 : Any, T2 : Any, reified R : Any> Data<T1>.combine(
//data collection operations
-/**
- * Lazily reduce a collection of [Data] to a single data.
- */
-public inline fun <T : Any, reified R : Any> Collection<Data<T>>.reduceToData(
- coroutineContext: CoroutineContext = EmptyCoroutineContext,
- meta: Meta = Meta.EMPTY,
- crossinline block: suspend (List<ValueWithMeta<T>>) -> R,
-): Data<R> = Data(
- meta,
- coroutineContext,
- this
-) {
- block(map { it.awaitWithMeta() })
+@PublishedApi
+internal fun Iterable<Data<*>>.joinMeta(): Meta = Meta {
+ var counter = 0
+ forEach { data ->
+ val inputIndex = (data as? NamedData)?.name?.toString() ?: (counter++).toString()
+ val token = NameToken("data", inputIndex)
+ set(token, data.meta)
+ }
}
-@DFInternal
-public fun <K, T : Any, R : Any> Map<K, Data<T>>.reduceToData(
+
+@PublishedApi
+internal fun Map<*, Data<*>>.joinMeta(): Meta = Meta {
+ forEach { (key, data) ->
+ val token = NameToken("data", key.toString())
+ set(token, data.meta)
+ }
+}
+
+@UnsafeKType
+public fun <K, T, R> Map<K, Data<T>>.reduceToData(
outputType: KType,
+ meta: Meta = joinMeta(),
coroutineContext: CoroutineContext = EmptyCoroutineContext,
- meta: Meta = Meta.EMPTY,
block: suspend (Map<K, ValueWithMeta<T>>) -> R,
): Data<R> = Data(
outputType,
@@ -86,9 +106,9 @@ public fun <K, T : Any, R : Any> Map<K, Data<T>>.reduceToData(
* @param T type of the input goal
* @param R type of the result goal
*/
-public inline fun <K, T : Any, reified R : Any> Map<K, Data<T>>.reduceToData(
+public inline fun <K, T, reified R> Map<K, Data<T>>.reduceToData(
+ meta: Meta = joinMeta(),
coroutineContext: CoroutineContext = EmptyCoroutineContext,
- meta: Meta = Meta.EMPTY,
crossinline block: suspend (Map<K, ValueWithMeta<T>>) -> R,
): Data<R> = Data(
meta,
@@ -100,11 +120,11 @@ public inline fun <K, T : Any, reified R : Any> Map<K, Data<T>>.reduceToData(
//Iterable operations
-@DFInternal
-public inline fun <T : Any, R : Any> Iterable<Data<T>>.reduceToData(
+@UnsafeKType
+public inline fun <T, R> Iterable<Data<T>>.reduceToData(
outputType: KType,
+ meta: Meta = joinMeta(),
coroutineContext: CoroutineContext = EmptyCoroutineContext,
- meta: Meta = Meta.EMPTY,
crossinline transformation: suspend (Collection<ValueWithMeta<T>>) -> R,
): Data<R> = Data(
outputType,
@@ -115,22 +135,22 @@ public inline fun <T : Any, R : Any> Iterable<Data<T>>.reduceToData(
transformation(map { it.awaitWithMeta() })
}
-@OptIn(DFInternal::class)
-public inline fun <T : Any, reified R : Any> Iterable<Data<T>>.reduceToData(
+@OptIn(UnsafeKType::class)
+public inline fun <T, reified R> Iterable<Data<T>>.reduceToData(
+ meta: Meta = joinMeta(),
coroutineContext: CoroutineContext = EmptyCoroutineContext,
- meta: Meta = Meta.EMPTY,
crossinline transformation: suspend (Collection<ValueWithMeta<T>>) -> R,
-): Data<R> = reduceToData(typeOf<R>(), coroutineContext, meta) {
+): Data<R> = reduceToData(typeOf<R>(), meta, coroutineContext) {
transformation(it)
}
-public inline fun <T : Any, reified R : Any> Iterable<Data<T>>.foldToData(
+public inline fun <T, reified R> Iterable<Data<T>>.foldToData(
initial: R,
+ meta: Meta = joinMeta(),
coroutineContext: CoroutineContext = EmptyCoroutineContext,
- meta: Meta = Meta.EMPTY,
crossinline block: suspend (result: R, data: ValueWithMeta<T>) -> R,
): Data<R> = reduceToData(
- coroutineContext, meta
+ meta, coroutineContext
) {
it.fold(initial) { acc, t -> block(acc, t) }
}
@@ -138,11 +158,11 @@ public inline fun <T : Any, reified R : Any> Iterable<Data<T>>.foldToData(
/**
* Transform an [Iterable] of [NamedData] to a single [Data].
*/
-@DFInternal
-public inline fun <T : Any, R : Any> Iterable<NamedData<T>>.reduceNamedToData(
+@UnsafeKType
+public inline fun <T, R> Iterable<NamedData<T>>.reduceNamedToData(
outputType: KType,
+ meta: Meta = joinMeta(),
coroutineContext: CoroutineContext = EmptyCoroutineContext,
- meta: Meta = Meta.EMPTY,
crossinline transformation: suspend (Collection<NamedValueWithMeta<T>>) -> R,
): Data<R> = Data(
outputType,
@@ -153,69 +173,99 @@ public inline fun <T : Any, R : Any> Iterable<NamedData<T>>.reduceNamedToData(
transformation(map { it.awaitWithMeta() })
}
-@OptIn(DFInternal::class)
-public inline fun <T : Any, reified R : Any> Iterable<NamedData<T>>.reduceNamedToData(
+@OptIn(UnsafeKType::class)
+public inline fun <T, reified R> Iterable<NamedData<T>>.reduceNamedToData(
+ meta: Meta = joinMeta(),
coroutineContext: CoroutineContext = EmptyCoroutineContext,
- meta: Meta = Meta.EMPTY,
crossinline transformation: suspend (Collection<NamedValueWithMeta<T>>) -> R,
-): Data<R> = reduceNamedToData(typeOf<R>(), coroutineContext, meta) {
+): Data<R> = reduceNamedToData(typeOf<R>(), meta, coroutineContext) {
transformation(it)
}
/**
* Fold a [Iterable] of named data into a single [Data]
*/
-public inline fun <T : Any, reified R : Any> Iterable<NamedData<T>>.foldNamedToData(
+public inline fun <T, reified R> Iterable<NamedData<T>>.foldNamedToData(
initial: R,
+ meta: Meta = joinMeta(),
coroutineContext: CoroutineContext = EmptyCoroutineContext,
- meta: Meta = Meta.EMPTY,
crossinline block: suspend (result: R, data: NamedValueWithMeta<T>) -> R,
): Data<R> = reduceNamedToData(
- coroutineContext, meta
+ meta, coroutineContext
) {
it.fold(initial) { acc, t -> block(acc, t) }
}
//DataSet operations
-@DFInternal
-public suspend fun <T : Any, R : Any> DataSet<T>.map(
+
+@UnsafeKType
+public fun <T, R> DataTree<T>.transformEach(
outputType: KType,
- coroutineContext: CoroutineContext = EmptyCoroutineContext,
- metaTransform: MutableMeta.() -> Unit = {},
- block: suspend (NamedValueWithMeta<T>) -> R,
-): DataTree<R> = DataTree<R>(outputType) {
- forEach {
- val newMeta = it.meta.toMutableMeta().apply(metaTransform).seal()
- val d = Data(outputType, newMeta, coroutineContext, listOf(it)) {
- block(it.awaitWithMeta())
+ scope: CoroutineScope,
+ metaTransform: MutableMeta.(name: Name) -> Unit = {},
+ compute: suspend (NamedValueWithMeta<T>) -> R,
+): DataTree<R> = DataTree<R>(
+ outputType,
+ scope,
+ initialData = asSequence().associate { namedData: NamedData<T> ->
+ val newMeta = namedData.meta.toMutableMeta().apply {
+ metaTransform(namedData.name)
+ }.seal()
+ val newData = Data(outputType, newMeta, scope.coroutineContext, listOf(namedData)) {
+ compute(namedData.awaitWithMeta())
+ }
+ namedData.name to newData
+ }
+) {
+ updates.collect { name ->
+ val data: Data<T>? = read(name)
+ if (data == null) write(name, null) else {
+ val newMeta = data.meta.toMutableMeta().apply {
+ metaTransform(name)
+ }.seal()
+ val d = Data(outputType, newMeta, scope.coroutineContext, listOf(data)) {
+ compute(NamedValueWithMeta(name, data.await(), data.meta))
+ }
+ write(name, d)
}
- data(it.name, d)
}
}
-@OptIn(DFInternal::class)
-public suspend inline fun <T : Any, reified R : Any> DataSet<T>.map(
- coroutineContext: CoroutineContext = EmptyCoroutineContext,
- noinline metaTransform: MutableMeta.() -> Unit = {},
+@OptIn(UnsafeKType::class)
+public inline fun <T, reified R> DataTree<T>.transformEach(
+ scope: CoroutineScope,
+ noinline metaTransform: MutableMeta.(name: Name) -> Unit = {},
noinline block: suspend (NamedValueWithMeta<T>) -> R,
-): DataTree<R> = map(typeOf<R>(), coroutineContext, metaTransform, block)
+): DataTree<R> = transformEach(typeOf<R>(), scope, metaTransform, block)
-public inline fun <T : Any> DataSet<T>.forEach(block: (NamedData<T>) -> Unit) {
- for (d in this) {
- block(d)
+public inline fun <T> DataTree<T>.forEach(block: (NamedData<T>) -> Unit) {
+ asSequence().forEach(block)
+}
+
+// DataSet snapshot reduction
+
+@PublishedApi
+internal fun DataTree<*>.joinMeta(): Meta = Meta {
+ asSequence().forEach {
+ val token = NameToken("data", it.name.toString())
+ set(token, it.meta)
}
}
-public inline fun <T : Any, reified R : Any> DataSet<T>.reduceToData(
+/**
+ * Reduce current snapshot of the [DataTree] to a single [Data].
+ * Even if a tree is changed in the future, only current data set is taken.
+ */
+public inline fun <T, reified R> DataTree<T>.reduceToData(
+ meta: Meta = joinMeta(),
coroutineContext: CoroutineContext = EmptyCoroutineContext,
- meta: Meta = Meta.EMPTY,
crossinline transformation: suspend (Iterable<NamedValueWithMeta<T>>) -> R,
-): Data<R> = asIterable().reduceNamedToData(coroutineContext, meta, transformation)
+): Data<R> = asSequence().asIterable().reduceNamedToData(meta, coroutineContext, transformation)
-public inline fun <T : Any, reified R : Any> DataSet<T>.foldToData(
+public inline fun <T, reified R> DataTree<T>.foldToData(
initial: R,
+ meta: Meta = joinMeta(),
coroutineContext: CoroutineContext = EmptyCoroutineContext,
- meta: Meta = Meta.EMPTY,
crossinline block: suspend (result: R, data: NamedValueWithMeta<T>) -> R,
-): Data<R> = asIterable().foldNamedToData(initial, coroutineContext, meta, block)
\ No newline at end of file
+): Data<R> = asSequence().asIterable().foldNamedToData(initial, meta, coroutineContext, block)
\ No newline at end of file
diff --git a/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/dataTreeBuilder.kt b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/dataTreeBuilder.kt
new file mode 100644
index 00000000..069e60bc
--- /dev/null
+++ b/dataforge-data/src/commonMain/kotlin/space/kscience/dataforge/data/dataTreeBuilder.kt
@@ -0,0 +1,122 @@
+package space.kscience.dataforge.data
+
+import kotlinx.coroutines.CoroutineScope
+import kotlinx.coroutines.flow.Flow
+import kotlinx.coroutines.flow.MutableSharedFlow
+import kotlinx.coroutines.flow.SharedFlow
+import kotlinx.coroutines.flow.mapNotNull
+import kotlinx.coroutines.launch
+import kotlinx.coroutines.sync.Mutex
+import kotlinx.coroutines.sync.withLock
+import space.kscience.dataforge.misc.UnsafeKType
+import space.kscience.dataforge.names.*
+import kotlin.reflect.KType
+import kotlin.reflect.typeOf
+
+
+private class FlatDataTree<T>(
+ override val dataType: KType,
+ private val dataSet: Map<Name, Data<T>>,
+ private val sourceUpdates: SharedFlow<Name>,
+ private val prefix: Name,
+) : DataTree<T> {
+ override val data: Data<T>? get() = dataSet[prefix]
+ override val items: Map<NameToken, FlatDataTree<T>>
+ get() = dataSet.keys
+ .filter { it.startsWith(prefix) && it.length > prefix.length }
+ .map { it.tokens[prefix.length] }
+ .associateWith { FlatDataTree(dataType, dataSet, sourceUpdates, prefix + it) }
+
+ override fun read(name: Name): Data<T>? = dataSet[prefix + name]
+
+ override val updates: Flow<Name> = sourceUpdates.mapNotNull { update ->
+ update.removeFirstOrNull(prefix)
+ }
+}
+
+/**
+ * A builder for [DataTree].
+ */
+private class DataTreeBuilder<T>(
+ private val type: KType,
+ initialData: Map<Name, Data<T>> = emptyMap(),
+) : DataSink<T> {
+
+ private val map = HashMap<Name, Data<T>>(initialData)
+
+ private val mutex = Mutex()
+
+ private val updatesFlow = MutableSharedFlow<Name>()
+
+
+ override suspend fun write(name: Name, data: Data<T>?) {
+ mutex.withLock {
+ if (data == null) {
+ map.remove(name)
+ } else {
+ map[name] = data
+ }
+ }
+ updatesFlow.emit(name)
+ }
+
+ fun build(): DataTree<T> = FlatDataTree(type, map, updatesFlow, Name.EMPTY)
+}
+
+/**
+ * Create a static [DataTree]
+ */
+@UnsafeKType
+public fun <T> DataTree(
+ dataType: KType,
+ scope: CoroutineScope,
+ initialData: Map<Name, Data<T>> = emptyMap(),
+ updater: suspend DataSink<T>.() -> Unit,
+): DataTree<T> = DataTreeBuilder<T>(dataType, initialData).apply {
+ scope.launch(GoalExecutionRestriction(GoalExecutionRestrictionPolicy.ERROR)) {
+ updater()
+ }
+}.build()
+
+/**
+ * Create and a data tree.
+ */
+@OptIn(UnsafeKType::class)
+public inline fun <reified T> DataTree(
+ scope: CoroutineScope,
+ initialData: Map<Name, Data<T>> = emptyMap(),
+ noinline updater: suspend DataSink<T>.() -> Unit,
+): DataTree<T> = DataTree(typeOf<T>(), scope, initialData, updater)
+
+@UnsafeKType
+public fun <T> DataTree(type: KType, data: Map<Name, Data<T>>): DataTree<T> =
+ DataTreeBuilder(type, data).build()
+
+@OptIn(UnsafeKType::class)
+public inline fun <reified T> DataTree(data: Map<Name, Data<T>>): DataTree<T> =
+ DataTree(typeOf<T>(), data)
+
+/**
+ * Represent this flat data map as a [DataTree] without copying it
+ */
+@UnsafeKType
+public fun <T> Map<Name, Data<T>>.asTree(type: KType): DataTree<T> =
+ DataTreeBuilder(type, this).build()
+
+/**
+ * Represent this flat data map as a [DataTree] without copying it
+ */
+@OptIn(UnsafeKType::class)
+public inline fun <reified T> Map<Name, Data<T>>.asTree(): DataTree<T> = asTree(typeOf<T>())
+
+
+@UnsafeKType
+public fun <T> Sequence<NamedData<T>>.toTree(type: KType): DataTree<T> =
+ DataTreeBuilder(type, associate { it.name to it }).build()
+
+
+/**
+ * Collect a sequence of [NamedData] to a [DataTree]
+ */
+@OptIn(UnsafeKType::class)
+public inline fun <reified T> Sequence<NamedData<T>>.toTree(): DataTree<T> = toTree(typeOf<T>())
diff --git a/dataforge-data/src/commonTest/kotlin/space/kscience/dataforge/data/DataTreeBuilderTest.kt b/dataforge-data/src/commonTest/kotlin/space/kscience/dataforge/data/DataTreeBuilderTest.kt
new file mode 100644
index 00000000..6a90664a
--- /dev/null
+++ b/dataforge-data/src/commonTest/kotlin/space/kscience/dataforge/data/DataTreeBuilderTest.kt
@@ -0,0 +1,73 @@
+package space.kscience.dataforge.data
+
+import kotlinx.coroutines.Job
+import kotlinx.coroutines.launch
+import kotlinx.coroutines.test.runTest
+import space.kscience.dataforge.names.asName
+import kotlin.test.Test
+import kotlin.test.assertEquals
+import kotlin.time.Duration.Companion.milliseconds
+
+
+internal class DataTreeBuilderTest {
+ @Test
+ fun testTreeBuild() = runTest(timeout = 500.milliseconds) {
+ val node = DataTree.static<Any> {
+ node("primary") {
+ value("a", "a")
+ value("b", "b")
+ }
+ value("c.d", "c.d")
+ value("c.f", "c.f")
+ }
+ assertEquals("a", node["primary.a"]?.await())
+ assertEquals("b", node["primary.b"]?.await())
+ assertEquals("c.d", node["c.d"]?.await())
+ assertEquals("c.f", node["c.f"]?.await())
+
+ }
+
+ @Test
+ fun testDataUpdate() = runTest(timeout = 500.milliseconds) {
+ val updateData = DataTree.static<Any> {
+ data("a", Data.wrapValue("a"))
+ data("b", Data.wrapValue("b"))
+ }
+
+ val node = DataTree.static<Any> {
+ node("primary") {
+ value("a", "a")
+ value("b", "b")
+ }
+ value("root", "root")
+ node("update", updateData)
+ }
+
+ assertEquals("a", node["update.a"]?.await())
+ assertEquals("a", node["primary.a"]?.await())
+ }
+
+ @Test
+ fun testDynamicUpdates() = runTest(timeout = 500.milliseconds) {
+ var job: Job? = null
+
+ val subNode = MutableDataTree<Int>()
+
+ val rootNode = MutableDataTree<Int>() {
+ job = launch {
+ writeAllAndWatch(subNode, "sub".asName())
+ }
+ }
+
+ repeat(10) {
+ subNode.writeValue("value[$it]", it)
+ }
+
+ assertEquals(9, subNode.awaitData("value[9]").await())
+ assertEquals(8, subNode.awaitData("value[8]").await())
+ assertEquals(9, rootNode.awaitData("sub.value[9]").await())
+ assertEquals(8, rootNode.awaitData("sub.value[8]").await())
+ println("finished")
+ job?.cancel()
+ }
+}
\ No newline at end of file
diff --git a/dataforge-data/src/jvmMain/kotlin/space/kscience/dataforge/data/actionInContext.kt b/dataforge-data/src/jvmMain/kotlin/space/kscience/dataforge/data/actionInContext.kt
deleted file mode 100644
index 33731a95..00000000
--- a/dataforge-data/src/jvmMain/kotlin/space/kscience/dataforge/data/actionInContext.kt
+++ /dev/null
@@ -1,2 +0,0 @@
-package space.kscience.dataforge.data
-
diff --git a/dataforge-data/src/jvmMain/kotlin/space/kscience/dataforge/data/dataFilterJvm.kt b/dataforge-data/src/jvmMain/kotlin/space/kscience/dataforge/data/dataFilterJvm.kt
deleted file mode 100644
index 74d67d9d..00000000
--- a/dataforge-data/src/jvmMain/kotlin/space/kscience/dataforge/data/dataFilterJvm.kt
+++ /dev/null
@@ -1,85 +0,0 @@
-package space.kscience.dataforge.data
-
-import kotlinx.coroutines.flow.Flow
-import kotlinx.coroutines.flow.filter
-import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.misc.DFExperimental
-import space.kscience.dataforge.names.Name
-import kotlin.coroutines.CoroutineContext
-import kotlin.coroutines.EmptyCoroutineContext
-import kotlin.reflect.KType
-import kotlin.reflect.full.isSubtypeOf
-import kotlin.reflect.typeOf
-
-
-/**
- * Cast the node to given type if the cast is possible or return null
- */
-@Suppress("UNCHECKED_CAST")
-private fun <R : Any> Data<*>.castOrNull(type: KType): Data<R>? =
- if (!this.type.isSubtypeOf(type)) {
- null
- } else {
- object : Data<R> by (this as Data<R>) {
- override val type: KType = type
- }
- }
-
-/**
- * Select all data matching given type and filters. Does not modify paths
- *
- * @param predicate addition filtering condition based on item name and meta. By default, accepts all
- */
-@OptIn(DFExperimental::class)
-public fun <R : Any> DataSet<*>.filterByType(
- type: KType,
- predicate: (name: Name, meta: Meta) -> Boolean = { _, _ -> true },
-): DataSource<R> = object : DataSource<R> {
- override val dataType = type
-
- override val coroutineContext: CoroutineContext
- get() = (this@filterByType as? DataSource)?.coroutineContext ?: EmptyCoroutineContext
-
- override val meta: Meta get() = this@filterByType.meta
-
- private fun checkDatum(name: Name, datum: Data<*>): Boolean = datum.type.isSubtypeOf(type)
- && predicate(name, datum.meta)
-
- override fun iterator(): Iterator<NamedData<R>> = iterator {
- for(d in this@filterByType){
- if(checkDatum(d.name,d.data)){
- @Suppress("UNCHECKED_CAST")
- yield(d as NamedData<R>)
- }
- }
- }
-
- override fun get(name: Name): Data<R>? = this@filterByType[name]?.let { datum ->
- if (checkDatum(name, datum)) datum.castOrNull(type) else null
- }
-
- override val updates: Flow<Name> = this@filterByType.updates.filter { name ->
- get(name)?.let { datum ->
- checkDatum(name, datum)
- } ?: false
- }
-}
-
-/**
- * Select a single datum of the appropriate type
- */
-public inline fun <reified R : Any> DataSet<*>.filterByType(
- noinline predicate: (name: Name, meta: Meta) -> Boolean = { _, _ -> true },
-): DataSet<R> = filterByType(typeOf<R>(), predicate)
-
-/**
- * Select a single datum if it is present and of given [type]
- */
-public fun <R : Any> DataSet<*>.getByType(type: KType, name: Name): NamedData<R>? =
- get(name)?.castOrNull<R>(type)?.named(name)
-
-public inline fun <reified R : Any> DataSet<*>.getByType(name: Name): NamedData<R>? =
- this@getByType.getByType(typeOf<R>(), name)
-
-public inline fun <reified R : Any> DataSet<*>.getByType(name: String): NamedData<R>? =
- this@getByType.getByType(typeOf<R>(), Name.parse(name))
\ No newline at end of file
diff --git a/dataforge-data/src/jvmMain/kotlin/space/kscience/dataforge/data/dataSetBuilderInContext.kt b/dataforge-data/src/jvmMain/kotlin/space/kscience/dataforge/data/dataSetBuilderInContext.kt
index cb222ea0..2bcca5e4 100644
--- a/dataforge-data/src/jvmMain/kotlin/space/kscience/dataforge/data/dataSetBuilderInContext.kt
+++ b/dataforge-data/src/jvmMain/kotlin/space/kscience/dataforge/data/dataSetBuilderInContext.kt
@@ -1,40 +1,29 @@
+@file:Suppress("CONTEXT_RECEIVERS_DEPRECATED")
+
package space.kscience.dataforge.data
-import kotlinx.coroutines.CoroutineScope
-import kotlinx.coroutines.Job
-import kotlinx.coroutines.launch
import space.kscience.dataforge.names.Name
-import space.kscience.dataforge.names.plus
/**
* Append data to node
*/
-context(DataSetBuilder<T>) public infix fun <T : Any> String.put(data: Data<T>): Unit =
- data(Name.parse(this), data)
+context(DataSink<T>)
+public suspend infix fun <T : Any> String.put(data: Data<T>): Unit =
+ write(Name.parse(this), data)
/**
* Append node
*/
-context(DataSetBuilder<T>) public infix fun <T : Any> String.put(dataSet: DataSet<T>): Unit =
- node(Name.parse(this), dataSet)
+context(DataSink<T>)
+public suspend infix fun <T : Any> String.putAll(dataSet: DataTree<T>): Unit =
+ writeAll(this, dataSet)
/**
* Build and append node
*/
-context(DataSetBuilder<T>) public infix fun <T : Any> String.put(
- block: DataSetBuilder<T>.() -> Unit,
-): Unit = node(Name.parse(this), block)
+context(DataSink<T>)
+public infix fun <T : Any> String.putAll(
+ block: DataSink<T>.() -> Unit,
+): Unit = writeAll(Name.parse(this), block)
-/**
- * Copy given data set and mirror its changes to this [DataTreeBuilder] in [this@setAndObserve]. Returns an update [Job]
- */
-context(DataSetBuilder<T>) public fun <T : Any> CoroutineScope.setAndWatch(
- name: Name,
- dataSet: DataSet<T>,
-): Job = launch {
- node(name, dataSet)
- dataSet.updates.collect { nameInBranch ->
- data(name + nameInBranch, dataSet.get(nameInBranch))
- }
-}
\ No newline at end of file
diff --git a/dataforge-data/src/jvmTest/kotlin/space/kscience/dataforge/data/ActionsTest.kt b/dataforge-data/src/jvmTest/kotlin/space/kscience/dataforge/data/ActionsTest.kt
index 3987cd19..fcc3e299 100644
--- a/dataforge-data/src/jvmTest/kotlin/space/kscience/dataforge/data/ActionsTest.kt
+++ b/dataforge-data/src/jvmTest/kotlin/space/kscience/dataforge/data/ActionsTest.kt
@@ -1,50 +1,49 @@
package space.kscience.dataforge.data
import kotlinx.coroutines.ExperimentalCoroutinesApi
-import kotlinx.coroutines.delay
import kotlinx.coroutines.test.runTest
-import org.junit.jupiter.api.Test
import space.kscience.dataforge.actions.Action
import space.kscience.dataforge.actions.invoke
-import space.kscience.dataforge.actions.map
+import space.kscience.dataforge.actions.mapping
import space.kscience.dataforge.misc.DFExperimental
+import kotlin.test.Test
import kotlin.test.assertEquals
+import kotlin.time.Duration.Companion.milliseconds
@OptIn(DFExperimental::class, ExperimentalCoroutinesApi::class)
internal class ActionsTest {
@Test
- fun testStaticMapAction() = runTest {
- val data: DataTree<Int> = DataTree {
+ fun testStaticMapAction() = runTest(timeout = 200.milliseconds) {
+ val plusOne = Action.mapping<Int, Int> {
+ result { it + 1 }
+ }
+
+ val data: DataTree<Int> = DataTree.static {
repeat(10) {
- static(it.toString(), it)
+ value(it.toString(), it)
}
}
- val plusOne = Action.map<Int, Int> {
- result { it + 1 }
- }
val result = plusOne(data)
- assertEquals(2, result["1"]?.await())
+
+ assertEquals(5, result.awaitData("4").await())
}
@Test
- fun testDynamicMapAction() = runTest {
- val data: DataSourceBuilder<Int> = DataSource()
-
- val plusOne = Action.map<Int, Int> {
+ fun testDynamicMapAction() = runTest(timeout = 200.milliseconds) {
+ val plusOne = Action.mapping<Int, Int> {
result { it + 1 }
}
- val result = plusOne(data)
+ val source: MutableDataTree<Int> = MutableDataTree()
+
+ val result: DataTree<Int> = plusOne(source)
repeat(10) {
- data.static(it.toString(), it)
+ source.writeValue(it.toString(), it)
}
- delay(20)
-
- assertEquals(2, result["1"]?.await())
- data.close()
+ assertEquals(5, result.awaitData("4").await())
}
}
\ No newline at end of file
diff --git a/dataforge-data/src/jvmTest/kotlin/space/kscience/dataforge/data/DataTreeBuilderTest.kt b/dataforge-data/src/jvmTest/kotlin/space/kscience/dataforge/data/DataTreeBuilderTest.kt
deleted file mode 100644
index b77f7ea2..00000000
--- a/dataforge-data/src/jvmTest/kotlin/space/kscience/dataforge/data/DataTreeBuilderTest.kt
+++ /dev/null
@@ -1,91 +0,0 @@
-package space.kscience.dataforge.data
-
-import kotlinx.coroutines.*
-import space.kscience.dataforge.misc.DFExperimental
-import space.kscience.dataforge.names.asName
-import kotlin.test.Test
-import kotlin.test.assertEquals
-
-
-internal class DataTreeBuilderTest {
- @Test
- fun testTreeBuild() = runBlocking {
- val node = DataTree<Any> {
- "primary" put {
- static("a", "a")
- static("b", "b")
- }
- static("c.d", "c.d")
- static("c.f", "c.f")
- }
- runBlocking {
- assertEquals("a", node["primary.a"]?.await())
- assertEquals("b", node["primary.b"]?.await())
- assertEquals("c.d", node["c.d"]?.await())
- assertEquals("c.f", node["c.f"]?.await())
- }
- }
-
- @OptIn(DFExperimental::class)
- @Test
- fun testDataUpdate() = runBlocking {
- val updateData: DataTree<Any> = DataTree {
- "update" put {
- "a" put Data.static("a")
- "b" put Data.static("b")
- }
- }
-
- val node = DataTree<Any> {
- "primary" put {
- static("a", "a")
- static("b", "b")
- }
- static("root", "root")
- populateFrom(updateData)
- }
-
- runBlocking {
- assertEquals("a", node["update.a"]?.await())
- assertEquals("a", node["primary.a"]?.await())
- }
- }
-
- @Test
- fun testDynamicUpdates() = runBlocking {
- try {
- lateinit var updateJob: Job
- supervisorScope {
- val subNode = DataSource<Int> {
- updateJob = launch {
- repeat(10) {
- delay(10)
- static("value", it)
- }
- delay(10)
- }
- }
- launch {
- subNode.updatesWithData.collect {
- println(it)
- }
- }
- val rootNode = DataSource<Int> {
- setAndWatch("sub".asName(), subNode)
- }
-
- launch {
- rootNode.updatesWithData.collect {
- println(it)
- }
- }
- updateJob.join()
- assertEquals(9, rootNode["sub.value"]?.await())
- cancel()
- }
- } catch (t: Throwable) {
- if (t !is CancellationException) throw t
- }
-
- }
-}
\ No newline at end of file
diff --git a/dataforge-io/README.md b/dataforge-io/README.md
index ec431a04..9b56352e 100644
--- a/dataforge-io/README.md
+++ b/dataforge-io/README.md
@@ -2,22 +2,29 @@
IO module
+## Features
+
+ - [IO format](src/commonMain/kotlin/space/kscience/dataforge/io/IOFormat.kt) : A generic API for reading something from binary representation and writing it to Binary.
+ - [Binary](src/commonMain/kotlin/space/kscience/dataforge/io/Binary.kt) : Multi-read random access binary.
+ - [Envelope](src/commonMain/kotlin/space/kscience/dataforge/io/Envelope.kt) : API and implementations for combined data and metadata format.
+ - [Tagged envelope](src/commonMain/kotlin/space/kscience/dataforge/io/TaggedEnvelope.kt) : Implementation for binary-friendly envelope format with machine readable tag and forward size declaration.
+ - [Tagged envelope](src/commonMain/kotlin/space/kscience/dataforge/io/TaglessEnvelope.kt) : Implementation for text-friendly envelope format with text separators for sections.
+
+
## Usage
## Artifact:
-The Maven coordinates of this project are `space.kscience:dataforge-io:0.7.0`.
+The Maven coordinates of this project are `space.kscience:dataforge-io:0.10.0`.
**Gradle Kotlin DSL:**
```kotlin
repositories {
maven("https://repo.kotlin.link")
- //uncomment to access development builds
- //maven("https://maven.pkg.jetbrains.space/spc/p/sci/dev")
mavenCentral()
}
dependencies {
- implementation("space.kscience:dataforge-io:0.7.0")
+ implementation("space.kscience:dataforge-io:0.10.0")
}
```
diff --git a/dataforge-io/build.gradle.kts b/dataforge-io/build.gradle.kts
index 6d3c888c..52e47bb3 100644
--- a/dataforge-io/build.gradle.kts
+++ b/dataforge-io/build.gradle.kts
@@ -4,12 +4,13 @@ plugins {
description = "IO module"
-val ioVersion = "0.2.1"
+val ioVersion = "0.6.0"
kscience {
jvm()
js()
native()
+ wasm()
useSerialization()
useSerialization(sourceSet = space.kscience.gradle.DependencySourceSet.TEST) {
cbor()
@@ -21,6 +22,60 @@ kscience {
}
}
-readme{
+readme {
maturity = space.kscience.gradle.Maturity.EXPERIMENTAL
+
+ description = """
+ Serialization foundation for Meta objects and Envelope processing.
+ """.trimIndent()
+
+ feature(
+ "io-format",
+ ref = "src/commonMain/kotlin/space/kscience/dataforge/io/IOFormat.kt",
+ name = "IO format"
+ ) {
+ """
+ A generic API for reading something from binary representation and writing it to Binary.
+
+ Similar to KSerializer, but without schema.
+ """.trimIndent()
+ }
+
+ feature(
+ "binary",
+ ref = "src/commonMain/kotlin/space/kscience/dataforge/io/Binary.kt",
+ name = "Binary"
+ ) {
+ "Multi-read random access binary."
+ }
+
+ feature(
+ "envelope",
+ ref = "src/commonMain/kotlin/space/kscience/dataforge/io/Envelope.kt",
+ name = "Envelope"
+ ) {
+ """
+ API and implementations for combined data and metadata format.
+ """.trimIndent()
+ }
+
+ feature(
+ "envelope.tagged",
+ ref = "src/commonMain/kotlin/space/kscience/dataforge/io/TaggedEnvelope.kt",
+ name = "Tagged envelope"
+ ) {
+ """
+ Implementation for binary-friendly envelope format with machine readable tag and forward size declaration.
+ """.trimIndent()
+ }
+
+ feature(
+ "envelope.tagless",
+ ref = "src/commonMain/kotlin/space/kscience/dataforge/io/TaglessEnvelope.kt",
+ name = "Tagged envelope"
+ ) {
+ """
+ Implementation for text-friendly envelope format with text separators for sections.
+ """.trimIndent()
+ }
}
\ No newline at end of file
diff --git a/dataforge-io/dataforge-io-proto/README.md b/dataforge-io/dataforge-io-proto/README.md
new file mode 100644
index 00000000..31e48c15
--- /dev/null
+++ b/dataforge-io/dataforge-io-proto/README.md
@@ -0,0 +1,21 @@
+# Module dataforge-io-proto
+
+ProtoBuf meta IO
+
+## Usage
+
+## Artifact:
+
+The Maven coordinates of this project are `space.kscience:dataforge-io-proto:0.10.0`.
+
+**Gradle Kotlin DSL:**
+```kotlin
+repositories {
+ maven("https://repo.kotlin.link")
+ mavenCentral()
+}
+
+dependencies {
+ implementation("space.kscience:dataforge-io-proto:0.10.0")
+}
+```
diff --git a/dataforge-io/dataforge-io-proto/build.gradle.kts b/dataforge-io/dataforge-io-proto/build.gradle.kts
new file mode 100644
index 00000000..a9099cb5
--- /dev/null
+++ b/dataforge-io/dataforge-io-proto/build.gradle.kts
@@ -0,0 +1,33 @@
+plugins {
+ id("space.kscience.gradle.mpp")
+ id("com.squareup.wire") version "4.9.9"
+}
+
+description = "ProtoBuf meta IO"
+
+kscience {
+ jvm()
+// js()
+ dependencies {
+ api(projects.dataforgeIo)
+ api("com.squareup.wire:wire-runtime:4.9.9")
+ }
+ useSerialization {
+ protobuf()
+ }
+}
+
+wire {
+ kotlin {
+ sourcePath {
+ srcDir("src/commonMain/proto")
+ }
+ }
+}
+
+readme {
+ maturity = space.kscience.gradle.Maturity.PROTOTYPE
+ description = """
+ ProtoBuf Meta representation
+ """.trimIndent()
+}
diff --git a/dataforge-io/dataforge-io-proto/src/commonMain/kotlin/ProtoEnvelopeFormat.kt b/dataforge-io/dataforge-io-proto/src/commonMain/kotlin/ProtoEnvelopeFormat.kt
new file mode 100644
index 00000000..d60539f5
--- /dev/null
+++ b/dataforge-io/dataforge-io-proto/src/commonMain/kotlin/ProtoEnvelopeFormat.kt
@@ -0,0 +1,32 @@
+package pace.kscience.dataforge.io.proto
+
+import kotlinx.io.Sink
+import kotlinx.io.Source
+import kotlinx.io.readByteArray
+import okio.ByteString
+import okio.ByteString.Companion.toByteString
+import space.kscience.dataforge.io.Envelope
+import space.kscience.dataforge.io.EnvelopeFormat
+import space.kscience.dataforge.io.asBinary
+import space.kscience.dataforge.io.proto.ProtoEnvelope
+import space.kscience.dataforge.io.toByteArray
+import space.kscience.dataforge.meta.Meta
+
+
+public object ProtoEnvelopeFormat : EnvelopeFormat {
+ override fun readFrom(source: Source): Envelope {
+ val protoEnvelope = ProtoEnvelope.ADAPTER.decode(source.readByteArray())
+ return Envelope(
+ meta = protoEnvelope.meta?.let { ProtoMetaWrapper(it) } ?: Meta.EMPTY,
+ data = protoEnvelope.dataBytes.toByteArray().asBinary()
+ )
+ }
+
+ override fun writeTo(sink: Sink, obj: Envelope) {
+ val protoEnvelope = ProtoEnvelope(
+ obj.meta.toProto(),
+ obj.data?.toByteArray()?.toByteString() ?: ByteString.EMPTY
+ )
+ sink.write(ProtoEnvelope.ADAPTER.encode(protoEnvelope))
+ }
+}
\ No newline at end of file
diff --git a/dataforge-io/dataforge-io-proto/src/commonMain/kotlin/ProtoMetaFormat.kt b/dataforge-io/dataforge-io-proto/src/commonMain/kotlin/ProtoMetaFormat.kt
new file mode 100644
index 00000000..8857832a
--- /dev/null
+++ b/dataforge-io/dataforge-io-proto/src/commonMain/kotlin/ProtoMetaFormat.kt
@@ -0,0 +1,76 @@
+package pace.kscience.dataforge.io.proto
+
+import kotlinx.io.Sink
+import kotlinx.io.Source
+import kotlinx.io.readByteArray
+import space.kscience.dataforge.io.MetaFormat
+import space.kscience.dataforge.io.proto.ProtoMeta
+import space.kscience.dataforge.meta.*
+import space.kscience.dataforge.meta.descriptors.MetaDescriptor
+import space.kscience.dataforge.names.NameToken
+
+internal class ProtoMetaWrapper(private val proto: ProtoMeta) : Meta {
+
+ private fun ProtoMeta.ProtoValue.toValue(): Value? = when {
+ stringValue != null -> stringValue.asValue()
+ booleanValue != null -> booleanValue.asValue()
+ doubleValue != null -> doubleValue.asValue()
+ floatValue != null -> floatValue.asValue()
+ int32Value != null -> int32Value.asValue()
+ int64Value != null -> int64Value.asValue()
+ bytesValue != null -> bytesValue.toByteArray().asValue()
+ listValue != null -> listValue.values.mapNotNull { it.toValue() }.asValue()
+ float64ListValue != null -> float64ListValue.values.map { it.asValue() }.asValue()
+ else -> null
+ }
+
+ override val value: Value?
+ get() = proto.protoValue?.toValue()
+
+
+ override val items: Map<NameToken, Meta>
+ get() = proto.items.entries.associate { NameToken.parse(it.key) to ProtoMetaWrapper(it.value) }
+
+ override fun toString(): String = Meta.toString(this)
+
+ override fun equals(other: Any?): Boolean = Meta.equals(this, other as? Meta)
+
+ override fun hashCode(): Int = Meta.hashCode(this)
+}
+
+internal fun Meta.toProto(): ProtoMeta {
+
+
+ fun Value.toProto(): ProtoMeta.ProtoValue = when (type) {
+ ValueType.NULL -> ProtoMeta.ProtoValue()
+
+ ValueType.NUMBER -> when (value) {
+ is Int, is Short, is Byte -> ProtoMeta.ProtoValue(int32Value = int)
+ is Long -> ProtoMeta.ProtoValue(int64Value = long)
+ is Float -> ProtoMeta.ProtoValue(floatValue = float)
+ else -> {
+// LoggerFactory.getLogger(ProtoMeta::class.java)
+// .warn("Unknown number type ${value} encoded as Double")
+ ProtoMeta.ProtoValue(doubleValue = double)
+ }
+ }
+
+ ValueType.STRING -> ProtoMeta.ProtoValue(stringValue = string)
+ ValueType.BOOLEAN -> ProtoMeta.ProtoValue(booleanValue = boolean)
+ ValueType.LIST -> ProtoMeta.ProtoValue(listValue = ProtoMeta.ProtoValueList(list.map { it.toProto() }))
+ }
+
+ return ProtoMeta(
+ protoValue = value?.toProto(),
+ items.entries.associate { it.key.toString() to it.value.toProto() }
+ )
+}
+
+public object ProtoMetaFormat : MetaFormat {
+ override fun writeMeta(sink: Sink, meta: Meta, descriptor: MetaDescriptor?) {
+ sink.write(ProtoMeta.ADAPTER.encode(meta.toProto()))
+ }
+
+ override fun readMeta(source: Source, descriptor: MetaDescriptor?): Meta =
+ ProtoMetaWrapper(ProtoMeta.ADAPTER.decode(source.readByteArray()))
+}
\ No newline at end of file
diff --git a/dataforge-io/dataforge-io-proto/src/commonMain/proto/meta.proto b/dataforge-io/dataforge-io-proto/src/commonMain/proto/meta.proto
new file mode 100644
index 00000000..c6233872
--- /dev/null
+++ b/dataforge-io/dataforge-io-proto/src/commonMain/proto/meta.proto
@@ -0,0 +1,35 @@
+syntax = "proto3";
+package space.kscience.dataforge.io.proto;
+
+message ProtoMeta {
+ message ProtoValue {
+ oneof value {
+ string stringValue = 2;
+ bool booleanValue = 3;
+ double doubleValue = 4;
+ float floatValue = 5;
+ int32 int32Value = 6;
+ int64 int64Value = 7;
+ bytes bytesValue = 8;
+ ProtoValueList listValue = 9;
+ Float64List float64ListValue = 10;
+ }
+ }
+
+ message ProtoValueList{
+ repeated ProtoValue values = 1;
+ }
+
+ message Float64List{
+ repeated double values = 1 [packed=true];
+ }
+
+ ProtoValue protoValue = 1;
+
+ map<string, ProtoMeta> items = 2;
+}
+
+message ProtoEnvelope{
+ ProtoMeta meta = 1;
+ bytes dataBytes = 2;
+}
\ No newline at end of file
diff --git a/dataforge-io/dataforge-io-proto/src/commonTest/kotlin/ProtoBufTest.kt b/dataforge-io/dataforge-io-proto/src/commonTest/kotlin/ProtoBufTest.kt
new file mode 100644
index 00000000..f8abef24
--- /dev/null
+++ b/dataforge-io/dataforge-io-proto/src/commonTest/kotlin/ProtoBufTest.kt
@@ -0,0 +1,83 @@
+package pace.kscience.dataforge.io.proto
+
+import kotlinx.io.writeString
+import space.kscience.dataforge.io.Envelope
+import space.kscience.dataforge.io.toByteArray
+import space.kscience.dataforge.meta.Meta
+import space.kscience.dataforge.meta.asValue
+import space.kscience.dataforge.meta.get
+import kotlin.test.Test
+import kotlin.test.assertContentEquals
+import kotlin.test.assertEquals
+
+class ProtoBufTest {
+
+ @Test
+ fun testProtoBufMetaFormat(){
+ val meta = Meta {
+ "a" put 22
+ "node" put {
+ "b" put "DDD"
+ "c" put 11.1
+ "d" put {
+ "d1" put {
+ "d11" put "aaa"
+ "d12" put "bbb"
+ }
+ "d2" put 2
+ }
+ "array" put doubleArrayOf(1.0, 2.0, 3.0)
+ "array2d" put listOf(
+ doubleArrayOf(1.0, 2.0, 3.0).asValue(),
+ doubleArrayOf(1.0, 2.0, 3.0).asValue()
+ ).asValue()
+ }
+ }
+ val buffer = kotlinx.io.Buffer()
+ ProtoMetaFormat.writeTo(buffer,meta)
+ val result = ProtoMetaFormat.readFrom(buffer)
+
+// println(result["a"]?.value)
+
+ meta.items.keys.forEach {
+ assertEquals(meta[it],result[it],"${meta[it]} != ${result[it]}")
+ }
+
+ assertEquals(meta, result)
+ }
+
+ @Test
+ fun testProtoBufEnvelopeFormat(){
+ val envelope = Envelope{
+ meta {
+ "a" put 22
+ "node" put {
+ "b" put "DDD"
+ "c" put 11.1
+ "d" put {
+ "d1" put {
+ "d11" put "aaa"
+ "d12" put "bbb"
+ }
+ "d2" put 2
+ }
+ "array" put doubleArrayOf(1.0, 2.0, 3.0)
+ "array2d" put listOf(
+ doubleArrayOf(1.0, 2.0, 3.0).asValue(),
+ doubleArrayOf(1.0, 2.0, 3.0).asValue()
+ ).asValue()
+ }
+ }
+ data {
+ writeString("Hello world!")
+ }
+ }
+
+ val buffer = kotlinx.io.Buffer()
+ ProtoEnvelopeFormat.writeTo(buffer,envelope)
+ val result = ProtoEnvelopeFormat.readFrom(buffer)
+
+ assertEquals(envelope.meta, result.meta)
+ assertContentEquals(envelope.data?.toByteArray(), result.data?.toByteArray())
+ }
+}
\ No newline at end of file
diff --git a/dataforge-io/dataforge-io-proto/src/jvmMain/kotlin/performanceComparison.kt b/dataforge-io/dataforge-io-proto/src/jvmMain/kotlin/performanceComparison.kt
new file mode 100644
index 00000000..74939882
--- /dev/null
+++ b/dataforge-io/dataforge-io-proto/src/jvmMain/kotlin/performanceComparison.kt
@@ -0,0 +1,51 @@
+package pace.kscience.dataforge.io.proto
+
+import kotlinx.io.writeString
+import space.kscience.dataforge.io.Envelope
+import space.kscience.dataforge.meta.asValue
+import kotlin.concurrent.thread
+import kotlin.time.measureTime
+
+public fun main() {
+ val envelope = Envelope {
+ meta {
+ "a" put 22
+ "node" put {
+ "b" put "DDD"
+ "c" put 11.1
+ "d" put {
+ "d1" put {
+ "d11" put "aaa"
+ "d12" put "bbb"
+ }
+ "d2" put 2
+ }
+ "array" put doubleArrayOf(1.0, 2.0, 3.0)
+ "array2d" put listOf(
+ doubleArrayOf(1.0, 2.0, 3.0).asValue(),
+ doubleArrayOf(1.0, 2.0, 3.0).asValue()
+ ).asValue()
+ }
+ }
+ data {
+ writeString("Hello world!")
+ }
+ }
+
+ val format = ProtoEnvelopeFormat
+
+ measureTime {
+ val threads = List(100) {
+ thread {
+ repeat(100000) {
+ val buffer = kotlinx.io.Buffer()
+ format.writeTo(buffer, envelope)
+// println(buffer.size)
+ val r = format.readFrom(buffer)
+ }
+ }
+ }
+
+ threads.forEach { it.join() }
+ }.also { println(it) }
+}
\ No newline at end of file
diff --git a/dataforge-io/dataforge-io-yaml/README.md b/dataforge-io/dataforge-io-yaml/README.md
index 06a0efb4..e296e17e 100644
--- a/dataforge-io/dataforge-io-yaml/README.md
+++ b/dataforge-io/dataforge-io-yaml/README.md
@@ -6,18 +6,16 @@ YAML meta IO
## Artifact:
-The Maven coordinates of this project are `space.kscience:dataforge-io-yaml:0.7.0`.
+The Maven coordinates of this project are `space.kscience:dataforge-io-yaml:0.10.0`.
**Gradle Kotlin DSL:**
```kotlin
repositories {
maven("https://repo.kotlin.link")
- //uncomment to access development builds
- //maven("https://maven.pkg.jetbrains.space/spc/p/sci/dev")
mavenCentral()
}
dependencies {
- implementation("space.kscience:dataforge-io-yaml:0.7.0")
+ implementation("space.kscience:dataforge-io-yaml:0.10.0")
}
```
diff --git a/dataforge-io/dataforge-io-yaml/build.gradle.kts b/dataforge-io/dataforge-io-yaml/build.gradle.kts
index 505ce360..0ae7e9a2 100644
--- a/dataforge-io/dataforge-io-yaml/build.gradle.kts
+++ b/dataforge-io/dataforge-io-yaml/build.gradle.kts
@@ -11,14 +11,14 @@ kscience {
dependencies {
api(projects.dataforgeIo)
}
- useSerialization{
+ useSerialization {
yamlKt()
}
}
-readme{
+readme {
maturity = space.kscience.gradle.Maturity.PROTOTYPE
- description ="""
+ description = """
YAML meta converters and Front Matter envelope format
""".trimIndent()
}
diff --git a/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/Binary.kt b/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/Binary.kt
index f85dd98e..de365519 100644
--- a/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/Binary.kt
+++ b/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/Binary.kt
@@ -71,6 +71,11 @@ internal class ByteArrayBinary(
override fun view(offset: Int, binarySize: Int): ByteArrayBinary =
ByteArrayBinary(array, start + offset, binarySize)
+
+ override fun toString(): String =
+ "ByteArrayBinary(array=$array, start=$start, size=$size)"
+
+
}
public fun ByteArray.asBinary(): Binary = ByteArrayBinary(this)
diff --git a/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/Envelope.kt b/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/Envelope.kt
index 728a0e69..43df520e 100644
--- a/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/Envelope.kt
+++ b/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/Envelope.kt
@@ -2,7 +2,6 @@ package space.kscience.dataforge.io
import space.kscience.dataforge.meta.Laminate
import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.meta.get
import space.kscience.dataforge.meta.string
import space.kscience.dataforge.names.Name
import space.kscience.dataforge.names.asName
@@ -34,7 +33,9 @@ public interface Envelope {
}
}
-internal class SimpleEnvelope(override val meta: Meta, override val data: Binary?) : Envelope
+internal class SimpleEnvelope(override val meta: Meta, override val data: Binary?) : Envelope{
+ override fun toString(): String = "Envelope(meta=$meta, data=$data)"
+}
public fun Envelope(meta: Meta, data: Binary?): Envelope = SimpleEnvelope(meta, data)
diff --git a/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/EnvelopeFormat.kt b/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/EnvelopeFormat.kt
index 0df5ab27..27a192d7 100644
--- a/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/EnvelopeFormat.kt
+++ b/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/EnvelopeFormat.kt
@@ -4,20 +4,17 @@ import kotlinx.io.Source
import space.kscience.dataforge.context.Context
import space.kscience.dataforge.io.EnvelopeFormatFactory.Companion.ENVELOPE_FORMAT_TYPE
import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.misc.DfId
+import space.kscience.dataforge.misc.DfType
import space.kscience.dataforge.names.Name
import space.kscience.dataforge.names.asName
import kotlin.reflect.KType
import kotlin.reflect.typeOf
-public interface EnvelopeFormat : IOFormat<Envelope> {
-
- override val type: KType get() = typeOf<Envelope>()
-}
+public interface EnvelopeFormat : IOFormat<Envelope>
public fun EnvelopeFormat.read(input: Source): Envelope = readFrom(input)
-@DfId(ENVELOPE_FORMAT_TYPE)
+@DfType(ENVELOPE_FORMAT_TYPE)
public interface EnvelopeFormatFactory : IOFormatFactory<Envelope>, EnvelopeFormat {
override val type: KType get() = typeOf<Envelope>()
diff --git a/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/EnvelopeParts.kt b/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/EnvelopeParts.kt
index 183e7b03..88231899 100644
--- a/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/EnvelopeParts.kt
+++ b/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/EnvelopeParts.kt
@@ -11,6 +11,7 @@ import space.kscience.dataforge.io.PartDescriptor.Companion.PARTS_KEY
import space.kscience.dataforge.io.PartDescriptor.Companion.SEPARATOR_KEY
import space.kscience.dataforge.meta.*
import space.kscience.dataforge.names.asName
+import space.kscience.dataforge.names.getIndexedList
import space.kscience.dataforge.names.plus
private class PartDescriptor : Scheme() {
@@ -84,7 +85,7 @@ public fun EnvelopeBuilder.envelopes(
public fun Envelope.parts(): EnvelopeParts {
if (data == null) return emptyList()
//TODO add zip folder reader
- val parts = meta.getIndexed(PARTS_KEY).values.map {
+ val parts = meta.getIndexedList(PARTS_KEY).map {
PartDescriptor.read(it)
}
return if (parts.isEmpty()) {
diff --git a/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/IOFormat.kt b/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/IOFormat.kt
index ffcadf1a..39fa2be1 100644
--- a/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/IOFormat.kt
+++ b/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/IOFormat.kt
@@ -7,7 +7,7 @@ import space.kscience.dataforge.context.Context
import space.kscience.dataforge.context.Factory
import space.kscience.dataforge.io.IOFormatFactory.Companion.IO_FORMAT_TYPE
import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.misc.DfId
+import space.kscience.dataforge.misc.DfType
import space.kscience.dataforge.misc.Named
import space.kscience.dataforge.names.Name
import space.kscience.dataforge.names.asName
@@ -17,11 +17,7 @@ import kotlin.reflect.typeOf
/**
* Reader of a custom object from input
*/
-public interface IOReader<out T> {
- /**
- * The type of object being read
- */
- public val type: KType
+public fun interface IOReader<out T> {
public fun readFrom(source: Source): T
@@ -32,7 +28,6 @@ public interface IOReader<out T> {
* no-op reader for binaries.
*/
public val binary: IOReader<Binary> = object : IOReader<Binary> {
- override val type: KType = typeOf<Binary>()
override fun readFrom(source: Source): Binary = source.readByteArray().asBinary()
@@ -42,8 +37,6 @@ public interface IOReader<out T> {
}
public inline fun <reified T> IOReader(crossinline read: Source.() -> T): IOReader<T> = object : IOReader<T> {
- override val type: KType = typeOf<T>()
-
override fun readFrom(source: Source): T = source.read()
}
@@ -56,24 +49,24 @@ public fun interface IOWriter<in T> {
*/
public interface IOFormat<T> : IOReader<T>, IOWriter<T>
-public fun <T : Any> Source.readWith(format: IOReader<T>): T = format.readFrom(this)
+public fun <T> Source.readWith(format: IOReader<T>): T = format.readFrom(this)
/**
* Read given binary as an object using given format
*/
-public fun <T : Any> Binary.readWith(format: IOReader<T>): T = read {
- readWith(format)
+public fun <T> Binary.readWith(format: IOReader<T>): T = read {
+ this.readWith(format)
}
/**
* Write an object to the [Sink] with given [format]
*/
-public fun <T : Any> Sink.writeWith(format: IOWriter<T>, obj: T): Unit =
+public fun <T> Sink.writeWith(format: IOWriter<T>, obj: T): Unit =
format.writeTo(this, obj)
-@DfId(IO_FORMAT_TYPE)
-public interface IOFormatFactory<T : Any> : Factory<IOFormat<T>>, Named {
+@DfType(IO_FORMAT_TYPE)
+public interface IOFormatFactory<T> : Factory<IOFormat<T>>, Named {
/**
* Explicit type for dynamic type checks
*/
@@ -86,7 +79,7 @@ public interface IOFormatFactory<T : Any> : Factory<IOFormat<T>>, Named {
}
}
-public fun <T : Any> Binary(obj: T, format: IOWriter<T>): Binary = Binary { format.writeTo(this, obj) }
+public fun <T> Binary(obj: T, format: IOWriter<T>): Binary = Binary { format.writeTo(this, obj) }
public object FloatIOFormat : IOFormat<Float>, IOFormatFactory<Float> {
override fun build(context: Context, meta: Meta): IOFormat<Float> = this
diff --git a/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/IOPlugin.kt b/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/IOPlugin.kt
index c3248021..11b5e5e3 100644
--- a/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/IOPlugin.kt
+++ b/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/IOPlugin.kt
@@ -5,30 +5,12 @@ import space.kscience.dataforge.io.EnvelopeFormatFactory.Companion.ENVELOPE_FORM
import space.kscience.dataforge.io.IOFormatFactory.Companion.IO_FORMAT_TYPE
import space.kscience.dataforge.io.MetaFormatFactory.Companion.META_FORMAT_TYPE
import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.meta.get
import space.kscience.dataforge.meta.string
-import space.kscience.dataforge.misc.DFInternal
import space.kscience.dataforge.names.Name
-import kotlin.reflect.KType
-import kotlin.reflect.typeOf
public class IOPlugin(meta: Meta) : AbstractPlugin(meta) {
override val tag: PluginTag get() = Companion.tag
- public val ioFormatFactories: Collection<IOFormatFactory<*>> by lazy {
- context.gather<IOFormatFactory<*>>(IO_FORMAT_TYPE).values
- }
-
- @Suppress("UNCHECKED_CAST")
- @DFInternal
- public fun <T : Any> resolveIOFormat(type: KType, meta: Meta): IOFormat<T>? =
- ioFormatFactories.singleOrNull { it.type == type }?.build(context, meta) as? IOFormat<T>
-
- @OptIn(DFInternal::class)
- public inline fun <reified T : Any> resolveIOFormat(meta: Meta = Meta.EMPTY): IOFormat<T>? =
- resolveIOFormat(typeOf<T>(), meta)
-
-
public val metaFormatFactories: Collection<MetaFormatFactory> by lazy {
context.gather<MetaFormatFactory>(META_FORMAT_TYPE).values
}
diff --git a/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/MetaFormat.kt b/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/MetaFormat.kt
index cadf87ca..b36ddc2a 100644
--- a/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/MetaFormat.kt
+++ b/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/MetaFormat.kt
@@ -9,7 +9,7 @@ import space.kscience.dataforge.context.Global
import space.kscience.dataforge.io.MetaFormatFactory.Companion.META_FORMAT_TYPE
import space.kscience.dataforge.meta.Meta
import space.kscience.dataforge.meta.descriptors.MetaDescriptor
-import space.kscience.dataforge.misc.DfId
+import space.kscience.dataforge.misc.DfType
import space.kscience.dataforge.names.Name
import space.kscience.dataforge.names.asName
import space.kscience.dataforge.names.plus
@@ -21,8 +21,6 @@ import kotlin.reflect.typeOf
*/
public interface MetaFormat : IOFormat<Meta> {
- override val type: KType get() = typeOf<Meta>()
-
override fun writeTo(sink: Sink, obj: Meta) {
writeMeta(sink, obj, null)
}
@@ -38,7 +36,7 @@ public interface MetaFormat : IOFormat<Meta> {
public fun readMeta(source: Source, descriptor: MetaDescriptor? = null): Meta
}
-@DfId(META_FORMAT_TYPE)
+@DfType(META_FORMAT_TYPE)
public interface MetaFormatFactory : IOFormatFactory<Meta>, MetaFormat {
public val shortName: String
diff --git a/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/Responder.kt b/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/Responder.kt
deleted file mode 100644
index 2a64966f..00000000
--- a/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/Responder.kt
+++ /dev/null
@@ -1,12 +0,0 @@
-package space.kscience.dataforge.io
-
-/**
- * An object that could respond to external messages asynchronously
- */
-public interface Responder {
- /**
- * Send a request and wait for response for this specific request
- */
- public suspend fun respond(request: Envelope): Envelope
-}
-
diff --git a/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/ioMisc.kt b/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/ioMisc.kt
index 93fd210a..b5e56dfe 100644
--- a/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/ioMisc.kt
+++ b/dataforge-io/src/commonMain/kotlin/space/kscience/dataforge/io/ioMisc.kt
@@ -5,7 +5,6 @@ import kotlinx.io.bytestring.ByteString
import kotlinx.io.bytestring.decodeToString
import kotlinx.io.bytestring.encodeToByteString
import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.misc.DFExperimental
import kotlin.math.min
/**
@@ -52,7 +51,6 @@ public fun IOPlugin.peekBinaryEnvelopeFormat(binary: Binary): EnvelopeFormat? {
/**
* A zero-copy read from
*/
-@DFExperimental
public fun IOPlugin.readEnvelope(
binary: Binary,
readNonEnvelopes: Boolean = false,
@@ -62,7 +60,6 @@ public fun IOPlugin.readEnvelope(
Envelope(Meta.EMPTY, binary)
} else error("Can't infer format for $binary")
-@DFExperimental
public fun IOPlugin.readEnvelope(
string: String,
readNonEnvelopes: Boolean = false,
diff --git a/dataforge-io/src/jvmMain/kotlin/space/kscience/dataforge/io/fileIO.kt b/dataforge-io/src/jvmMain/kotlin/space/kscience/dataforge/io/fileIO.kt
index c15280f3..2d54e061 100644
--- a/dataforge-io/src/jvmMain/kotlin/space/kscience/dataforge/io/fileIO.kt
+++ b/dataforge-io/src/jvmMain/kotlin/space/kscience/dataforge/io/fileIO.kt
@@ -15,8 +15,6 @@ import java.nio.file.Path
import java.nio.file.StandardOpenOption
import kotlin.io.path.inputStream
import kotlin.math.min
-import kotlin.reflect.full.isSupertypeOf
-import kotlin.reflect.typeOf
import kotlin.streams.asSequence
@@ -77,18 +75,8 @@ public fun Path.rewrite(block: Sink.() -> Unit): Unit {
stream.asSink().buffered().use(block)
}
-@DFExperimental
public fun EnvelopeFormat.readFile(path: Path): Envelope = readFrom(path.asBinary())
-/**
- * Resolve IOFormat based on type
- */
-@Suppress("UNCHECKED_CAST")
-@DFExperimental
-public inline fun <reified T : Any> IOPlugin.resolveIOFormat(): IOFormat<T>? =
- ioFormatFactories.find { it.type.isSupertypeOf(typeOf<T>()) } as IOFormat<T>?
-
-
public val IOPlugin.Companion.META_FILE_NAME: String get() = "@meta"
public val IOPlugin.Companion.DATA_FILE_NAME: String get() = "@data"
@@ -192,7 +180,6 @@ public fun IOPlugin.peekFileEnvelopeFormat(path: Path): EnvelopeFormat? {
*
* Return null otherwise.
*/
-@DFExperimental
public fun IOPlugin.readEnvelopeFile(
path: Path,
readNonEnvelopes: Boolean = false,
diff --git a/dataforge-meta/README.md b/dataforge-meta/README.md
index 5f214640..7eb186ed 100644
--- a/dataforge-meta/README.md
+++ b/dataforge-meta/README.md
@@ -2,22 +2,27 @@
Meta definition and basic operations on meta
+## Features
+
+ - [Meta](src/commonMain/kotlin/space/kscience/dataforge/meta/Meta.kt) : **Meta** is the representation of basic DataForge concept: Metadata, but it also could be called meta-value tree.
+ - [Value](src/commonMain/kotlin/space/kscience/dataforge/meta/Value.kt) : **Value** a sum type for different meta values.
+ - [Name](src/commonMain/kotlin/space/kscience/dataforge/names/Name.kt) : **Name** is an identifier to access tree-like structure.
+
+
## Usage
## Artifact:
-The Maven coordinates of this project are `space.kscience:dataforge-meta:0.7.0`.
+The Maven coordinates of this project are `space.kscience:dataforge-meta:0.10.0`.
**Gradle Kotlin DSL:**
```kotlin
repositories {
maven("https://repo.kotlin.link")
- //uncomment to access development builds
- //maven("https://maven.pkg.jetbrains.space/spc/p/sci/dev")
mavenCentral()
}
dependencies {
- implementation("space.kscience:dataforge-meta:0.7.0")
+ implementation("space.kscience:dataforge-meta:0.10.0")
}
```
diff --git a/dataforge-meta/api/dataforge-meta.api b/dataforge-meta/api/dataforge-meta.api
index 7daa7540..b282c77f 100644
--- a/dataforge-meta/api/dataforge-meta.api
+++ b/dataforge-meta/api/dataforge-meta.api
@@ -1,3 +1,15 @@
+public final class space/kscience/dataforge/meta/ByteArrayValue : java/lang/Iterable, kotlin/jvm/internal/markers/KMappedMarker, space/kscience/dataforge/meta/Value {
+ public fun <init> ([B)V
+ public fun equals (Ljava/lang/Object;)Z
+ public fun getList ()Ljava/util/List;
+ public fun getType ()Lspace/kscience/dataforge/meta/ValueType;
+ public synthetic fun getValue ()Ljava/lang/Object;
+ public fun getValue ()[B
+ public fun hashCode ()I
+ public fun iterator ()Ljava/util/Iterator;
+ public fun toString ()Ljava/lang/String;
+}
+
public abstract interface class space/kscience/dataforge/meta/Configurable {
public abstract fun getMeta ()Lspace/kscience/dataforge/meta/MutableMeta;
}
@@ -30,7 +42,22 @@ public final class space/kscience/dataforge/meta/EnumValue : space/kscience/data
}
public final class space/kscience/dataforge/meta/ExoticValuesKt {
+ public static final fun asValue ([B)Lspace/kscience/dataforge/meta/Value;
public static final fun asValue ([D)Lspace/kscience/dataforge/meta/Value;
+ public static final fun byteArray (Lspace/kscience/dataforge/meta/MetaProvider;[BLspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadOnlyProperty;
+ public static final fun byteArray (Lspace/kscience/dataforge/meta/MutableMetaProvider;[BLspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
+ public static synthetic fun byteArray$default (Lspace/kscience/dataforge/meta/MetaProvider;[BLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
+ public static synthetic fun byteArray$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;[BLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
+ public static final fun doubleArray (Lspace/kscience/dataforge/meta/MetaProvider;[DLspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadOnlyProperty;
+ public static final fun doubleArray (Lspace/kscience/dataforge/meta/MutableMetaProvider;[DLspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
+ public static synthetic fun doubleArray$default (Lspace/kscience/dataforge/meta/MetaProvider;[DLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
+ public static synthetic fun doubleArray$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;[DLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
+ public static final fun getByteArray (Lspace/kscience/dataforge/meta/Meta;)[B
+ public static final fun getByteArray (Lspace/kscience/dataforge/meta/MetaConverter$Companion;)Lspace/kscience/dataforge/meta/MetaConverter;
+ public static final fun getByteArray (Lspace/kscience/dataforge/meta/Value;)[B
+ public static final fun getDoubleArray (Lspace/kscience/dataforge/meta/Meta;)[D
+ public static final fun getDoubleArray (Lspace/kscience/dataforge/meta/MetaConverter$Companion;)Lspace/kscience/dataforge/meta/MetaConverter;
+ public static final fun getDoubleArray (Lspace/kscience/dataforge/meta/Value;)[D
public static final fun lazyParseValue (Ljava/lang/String;)Lspace/kscience/dataforge/meta/LazyParsedValue;
}
@@ -56,6 +83,20 @@ public final class space/kscience/dataforge/meta/JsonMetaKt {
public static final fun toValue (Lkotlinx/serialization/json/JsonPrimitive;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;)Lspace/kscience/dataforge/meta/Value;
}
+public final class space/kscience/dataforge/meta/KeepTransformationRule : space/kscience/dataforge/meta/TransformationRule {
+ public fun <init> (Lkotlin/jvm/functions/Function1;)V
+ public final fun component1 ()Lkotlin/jvm/functions/Function1;
+ public final fun copy (Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/KeepTransformationRule;
+ public static synthetic fun copy$default (Lspace/kscience/dataforge/meta/KeepTransformationRule;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/KeepTransformationRule;
+ public fun equals (Ljava/lang/Object;)Z
+ public final fun getSelector ()Lkotlin/jvm/functions/Function1;
+ public fun hashCode ()I
+ public fun matches (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/Meta;)Z
+ public fun selectItems (Lspace/kscience/dataforge/meta/Meta;)Lkotlin/sequences/Sequence;
+ public fun toString ()Ljava/lang/String;
+ public fun transformItem (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/Meta;Lspace/kscience/dataforge/meta/MutableMeta;)V
+}
+
public final class space/kscience/dataforge/meta/Laminate : space/kscience/dataforge/meta/TypedMeta {
public static final field Companion Lspace/kscience/dataforge/meta/Laminate$Companion;
public fun equals (Ljava/lang/Object;)Z
@@ -64,6 +105,8 @@ public final class space/kscience/dataforge/meta/Laminate : space/kscience/dataf
public synthetic fun get (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/TypedMeta;
public fun getItems ()Ljava/util/Map;
public final fun getLayers ()Ljava/util/List;
+ public fun getSelf ()Lspace/kscience/dataforge/meta/Laminate;
+ public synthetic fun getSelf ()Lspace/kscience/dataforge/meta/TypedMeta;
public fun getValue ()Lspace/kscience/dataforge/meta/Value;
public fun hashCode ()I
public final fun merge ()Lspace/kscience/dataforge/meta/SealedMeta;
@@ -159,49 +202,89 @@ public final class space/kscience/dataforge/meta/MetaBuilder : space/kscience/da
public abstract interface annotation class space/kscience/dataforge/meta/MetaBuilderMarker : java/lang/annotation/Annotation {
}
+public abstract interface class space/kscience/dataforge/meta/MetaConverter : space/kscience/dataforge/meta/MetaReader {
+ public static final field Companion Lspace/kscience/dataforge/meta/MetaConverter$Companion;
+ public abstract fun convert (Ljava/lang/Object;)Lspace/kscience/dataforge/meta/Meta;
+ public fun getDescriptor ()Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;
+ public fun read (Lspace/kscience/dataforge/meta/Meta;)Ljava/lang/Object;
+ public abstract fun readOrNull (Lspace/kscience/dataforge/meta/Meta;)Ljava/lang/Object;
+}
+
+public final class space/kscience/dataforge/meta/MetaConverter$Companion {
+ public final fun getBoolean ()Lspace/kscience/dataforge/meta/MetaConverter;
+ public final fun getDouble ()Lspace/kscience/dataforge/meta/MetaConverter;
+ public final fun getFloat ()Lspace/kscience/dataforge/meta/MetaConverter;
+ public final fun getInt ()Lspace/kscience/dataforge/meta/MetaConverter;
+ public final fun getLong ()Lspace/kscience/dataforge/meta/MetaConverter;
+ public final fun getMeta ()Lspace/kscience/dataforge/meta/MetaConverter;
+ public final fun getNumber ()Lspace/kscience/dataforge/meta/MetaConverter;
+ public final fun getString ()Lspace/kscience/dataforge/meta/MetaConverter;
+ public final fun getStringList ()Lspace/kscience/dataforge/meta/MetaConverter;
+ public final fun getValue ()Lspace/kscience/dataforge/meta/MetaConverter;
+ public final fun valueList (Lkotlin/jvm/functions/Function1;Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/MetaConverter;
+ public static synthetic fun valueList$default (Lspace/kscience/dataforge/meta/MetaConverter$Companion;Lkotlin/jvm/functions/Function1;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaConverter;
+}
+
+public final class space/kscience/dataforge/meta/MetaConverterKt {
+ public static final fun convertNullable (Lspace/kscience/dataforge/meta/MetaConverter;Ljava/lang/Object;)Lspace/kscience/dataforge/meta/Meta;
+}
+
+public abstract interface class space/kscience/dataforge/meta/MetaDelegate : kotlin/properties/ReadOnlyProperty, space/kscience/dataforge/meta/descriptors/Described {
+}
+
public final class space/kscience/dataforge/meta/MetaDelegateKt {
- public static final fun boolean (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadOnlyProperty;
- public static final fun boolean (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;)Lkotlin/properties/ReadOnlyProperty;
- public static final fun boolean (Lspace/kscience/dataforge/meta/MetaProvider;ZLspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun boolean$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun boolean$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun boolean$default (Lspace/kscience/dataforge/meta/MetaProvider;ZLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
- public static final fun double (Lspace/kscience/dataforge/meta/MetaProvider;DLspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadOnlyProperty;
- public static final fun double (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun double$default (Lspace/kscience/dataforge/meta/MetaProvider;DLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun double$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
- public static final fun float (Lspace/kscience/dataforge/meta/MetaProvider;FLspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadOnlyProperty;
- public static final fun float (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun float$default (Lspace/kscience/dataforge/meta/MetaProvider;FLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun float$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
- public static final fun int (Lspace/kscience/dataforge/meta/MetaProvider;ILspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadOnlyProperty;
- public static final fun int (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun int$default (Lspace/kscience/dataforge/meta/MetaProvider;ILspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun int$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
- public static final fun long (Lspace/kscience/dataforge/meta/MetaProvider;JLspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadOnlyProperty;
- public static final fun long (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun long$default (Lspace/kscience/dataforge/meta/MetaProvider;JLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun long$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
- public static final fun node (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadOnlyProperty;
- public static final fun node (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/transformations/MetaConverter;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun node$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun node$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/transformations/MetaConverter;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
- public static final fun number (Lspace/kscience/dataforge/meta/MetaProvider;Ljava/lang/Number;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadOnlyProperty;
- public static final fun number (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadOnlyProperty;
- public static final fun number (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun number$default (Lspace/kscience/dataforge/meta/MetaProvider;Ljava/lang/Number;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun number$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun number$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
- public static final fun string (Lspace/kscience/dataforge/meta/MetaProvider;Ljava/lang/String;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadOnlyProperty;
- public static final fun string (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadOnlyProperty;
- public static final fun string (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun string$default (Lspace/kscience/dataforge/meta/MetaProvider;Ljava/lang/String;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun string$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun string$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
- public static final fun value (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadOnlyProperty;
- public static final fun value (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function1;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun value$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
- public static synthetic fun value$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
+ public static final fun boolean (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun boolean (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun boolean (Lspace/kscience/dataforge/meta/MetaProvider;ZLspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun boolean$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun boolean$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun boolean$default (Lspace/kscience/dataforge/meta/MetaProvider;ZLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun double (Lspace/kscience/dataforge/meta/MetaProvider;DLspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun double (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun double$default (Lspace/kscience/dataforge/meta/MetaProvider;DLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun double$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun float (Lspace/kscience/dataforge/meta/MetaProvider;FLspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun float (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun float$default (Lspace/kscience/dataforge/meta/MetaProvider;FLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun float$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun int (Lspace/kscience/dataforge/meta/MetaProvider;ILspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun int (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun int$default (Lspace/kscience/dataforge/meta/MetaProvider;ILspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun int$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun listOfReadable (Lspace/kscience/dataforge/meta/Meta;Lspace/kscience/dataforge/meta/MetaReader;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun listOfReadable$default (Lspace/kscience/dataforge/meta/Meta;Lspace/kscience/dataforge/meta/MetaReader;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun listOfSpec (Lspace/kscience/dataforge/meta/Meta;Lspace/kscience/dataforge/meta/MetaReader;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun listOfSpec$default (Lspace/kscience/dataforge/meta/Meta;Lspace/kscience/dataforge/meta/MetaReader;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun long (Lspace/kscience/dataforge/meta/MetaProvider;JLspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun long (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun long$default (Lspace/kscience/dataforge/meta/MetaProvider;JLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun long$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun node (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/MetaReader;)Lkotlin/properties/ReadOnlyProperty;
+ public static final fun node (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun node$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/MetaReader;ILjava/lang/Object;)Lkotlin/properties/ReadOnlyProperty;
+ public static synthetic fun node$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun number (Lspace/kscience/dataforge/meta/MetaProvider;Ljava/lang/Number;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun number (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun number (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun number$default (Lspace/kscience/dataforge/meta/MetaProvider;Ljava/lang/Number;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun number$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun number$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun readable (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/meta/MetaReader;Ljava/lang/Object;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun readable (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/meta/MetaReader;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun readable$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/meta/MetaReader;Ljava/lang/Object;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun readable$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/meta/MetaReader;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun spec (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/meta/MetaReader;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun spec$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/meta/MetaReader;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun string (Lspace/kscience/dataforge/meta/MetaProvider;Ljava/lang/String;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun string (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun string (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun string$default (Lspace/kscience/dataforge/meta/MetaProvider;Ljava/lang/String;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun string$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun string$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun value (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static final fun value (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun value$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
+ public static synthetic fun value$default (Lspace/kscience/dataforge/meta/MetaProvider;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MetaDelegate;
}
public final class space/kscience/dataforge/meta/MetaKt {
@@ -222,7 +305,6 @@ public final class space/kscience/dataforge/meta/MetaKt {
public static final fun getLong (Lspace/kscience/dataforge/meta/Meta;)Ljava/lang/Long;
public static final synthetic fun getNonNullable (Lspace/kscience/dataforge/meta/Meta;Lspace/kscience/dataforge/names/NameToken;)Lspace/kscience/dataforge/meta/Meta;
public static final fun getNumber (Lspace/kscience/dataforge/meta/Meta;)Ljava/lang/Number;
- public static final fun getSelf (Lspace/kscience/dataforge/meta/TypedMeta;)Lspace/kscience/dataforge/meta/TypedMeta;
public static final fun getShort (Lspace/kscience/dataforge/meta/Meta;)Ljava/lang/Short;
public static final fun getString (Lspace/kscience/dataforge/meta/Meta;)Ljava/lang/String;
public static final fun getStringList (Lspace/kscience/dataforge/meta/Meta;)Ljava/util/List;
@@ -239,6 +321,20 @@ public abstract interface class space/kscience/dataforge/meta/MetaProvider : spa
public fun getValue (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/Value;
}
+public abstract interface class space/kscience/dataforge/meta/MetaReader : space/kscience/dataforge/meta/descriptors/Described {
+ public fun read (Lspace/kscience/dataforge/meta/Meta;)Ljava/lang/Object;
+ public abstract fun readOrNull (Lspace/kscience/dataforge/meta/Meta;)Ljava/lang/Object;
+}
+
+public final class space/kscience/dataforge/meta/MetaReaderKt {
+ public static final fun readNullable (Lspace/kscience/dataforge/meta/MetaReader;Lspace/kscience/dataforge/meta/Meta;)Ljava/lang/Object;
+ public static final fun readValue (Lspace/kscience/dataforge/meta/MetaReader;Lspace/kscience/dataforge/meta/Value;)Ljava/lang/Object;
+}
+
+public abstract interface class space/kscience/dataforge/meta/MetaRefStore : space/kscience/dataforge/meta/descriptors/Described {
+ public abstract fun getRefs ()Ljava/util/List;
+}
+
public abstract interface class space/kscience/dataforge/meta/MetaRepr {
public abstract fun toMeta ()Lspace/kscience/dataforge/meta/Meta;
}
@@ -252,6 +348,37 @@ public final class space/kscience/dataforge/meta/MetaSerializer : kotlinx/serial
public fun serialize (Lkotlinx/serialization/encoding/Encoder;Lspace/kscience/dataforge/meta/Meta;)V
}
+public final class space/kscience/dataforge/meta/MetaTransformation {
+ public static final field Companion Lspace/kscience/dataforge/meta/MetaTransformation$Companion;
+ public static final fun apply-impl (Ljava/util/Collection;Lspace/kscience/dataforge/meta/Meta;)Lspace/kscience/dataforge/meta/Meta;
+ public static final fun bind-impl (Ljava/util/Collection;Lspace/kscience/dataforge/meta/ObservableMeta;Lspace/kscience/dataforge/meta/MutableMeta;)V
+ public static final synthetic fun box-impl (Ljava/util/Collection;)Lspace/kscience/dataforge/meta/MetaTransformation;
+ public static fun constructor-impl (Ljava/util/Collection;)Ljava/util/Collection;
+ public fun equals (Ljava/lang/Object;)Z
+ public static fun equals-impl (Ljava/util/Collection;Ljava/lang/Object;)Z
+ public static final fun equals-impl0 (Ljava/util/Collection;Ljava/util/Collection;)Z
+ public static final fun generate-impl (Ljava/util/Collection;Lspace/kscience/dataforge/meta/Meta;)Lspace/kscience/dataforge/meta/Meta;
+ public fun hashCode ()I
+ public static fun hashCode-impl (Ljava/util/Collection;)I
+ public fun toString ()Ljava/lang/String;
+ public static fun toString-impl (Ljava/util/Collection;)Ljava/lang/String;
+ public final synthetic fun unbox-impl ()Ljava/util/Collection;
+}
+
+public final class space/kscience/dataforge/meta/MetaTransformation$Companion {
+ public final fun make--mWxz5M (Lkotlin/jvm/functions/Function1;)Ljava/util/Collection;
+}
+
+public final class space/kscience/dataforge/meta/MetaTransformationBuilder {
+ public fun <init> ()V
+ public final fun build-m6Fha10 ()Ljava/util/Collection;
+ public final fun keep (Ljava/lang/String;)V
+ public final fun keep (Lkotlin/jvm/functions/Function1;)V
+ public final fun keep (Lspace/kscience/dataforge/names/Name;)V
+ public final fun move (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function1;)V
+ public static synthetic fun move$default (Lspace/kscience/dataforge/meta/MetaTransformationBuilder;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)V
+}
+
public abstract interface class space/kscience/dataforge/meta/MutableMeta : space/kscience/dataforge/meta/Meta, space/kscience/dataforge/meta/MutableMetaProvider {
public static final field Companion Lspace/kscience/dataforge/meta/MutableMeta$Companion;
public synthetic fun get (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/Meta;
@@ -267,6 +394,7 @@ public abstract interface class space/kscience/dataforge/meta/MutableMeta : spac
public fun put (Ljava/lang/String;Lspace/kscience/dataforge/meta/MetaRepr;)V
public fun put (Ljava/lang/String;Lspace/kscience/dataforge/meta/Value;)V
public fun put (Ljava/lang/String;Z)V
+ public fun put (Ljava/lang/String;[B)V
public fun put (Ljava/lang/String;[D)V
public fun put (Lspace/kscience/dataforge/names/Name;Ljava/lang/Enum;)V
public fun put (Lspace/kscience/dataforge/names/Name;Ljava/lang/Number;)V
@@ -286,64 +414,70 @@ public final class space/kscience/dataforge/meta/MutableMeta$Companion {
public final fun serializer ()Lkotlinx/serialization/KSerializer;
}
+public abstract interface class space/kscience/dataforge/meta/MutableMetaDelegate : kotlin/properties/ReadWriteProperty, space/kscience/dataforge/meta/descriptors/Described {
+}
+
public final class space/kscience/dataforge/meta/MutableMetaDelegateKt {
- public static final fun boolean (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static final fun boolean (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;)Lkotlin/properties/ReadWriteProperty;
- public static final fun boolean (Lspace/kscience/dataforge/meta/MutableMetaProvider;ZLspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun boolean$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun boolean$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun boolean$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;ZLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static final fun double (Lspace/kscience/dataforge/meta/MutableMetaProvider;DLspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static final fun double (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun double$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;DLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun double$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static final fun doubleArray (Lspace/kscience/dataforge/meta/MutableMetaProvider;[DLspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun doubleArray$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;[DLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static final fun float (Lspace/kscience/dataforge/meta/MutableMetaProvider;FLspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static final fun float (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun float$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;FLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun float$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static final fun int (Lspace/kscience/dataforge/meta/MutableMetaProvider;ILspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static final fun int (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun int$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;ILspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun int$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static final fun listValue (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function1;Lkotlin/jvm/functions/Function1;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun listValue$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function1;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static final fun long (Lspace/kscience/dataforge/meta/MutableMetaProvider;JLspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static final fun long (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun long$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;JLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun long$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static final fun node (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static final fun node (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/transformations/MetaConverter;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun node$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun node$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/transformations/MetaConverter;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static final fun number (Lspace/kscience/dataforge/meta/MutableMetaProvider;Ljava/lang/Number;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static final fun number (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static final fun number (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun number$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Ljava/lang/Number;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun number$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun number$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static final fun numberList (Lspace/kscience/dataforge/meta/MutableMetaProvider;[Ljava/lang/Number;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun numberList$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;[Ljava/lang/Number;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static final fun string (Lspace/kscience/dataforge/meta/MutableMetaProvider;Ljava/lang/String;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static final fun string (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static final fun string (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun string$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Ljava/lang/String;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun string$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun string$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static final fun stringList (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static final fun stringList (Lspace/kscience/dataforge/meta/MutableMetaProvider;[Ljava/lang/String;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun stringList$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun stringList$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;[Ljava/lang/String;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static final fun value (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static final fun value (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function1;Lkotlin/jvm/functions/Function1;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun value$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun value$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function1;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
+ public static final fun boolean (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun boolean (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun boolean (Lspace/kscience/dataforge/meta/MutableMetaProvider;ZLspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun boolean$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun boolean$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun boolean$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;ZLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun convertable (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/meta/MetaConverter;Ljava/lang/Object;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun convertable (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/meta/MetaConverter;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun convertable$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/meta/MetaConverter;Ljava/lang/Object;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun convertable$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/meta/MetaConverter;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun double (Lspace/kscience/dataforge/meta/MutableMetaProvider;DLspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun double (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun double$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;DLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun double$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun float (Lspace/kscience/dataforge/meta/MutableMetaProvider;FLspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun float (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun float$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;FLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun float$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun int (Lspace/kscience/dataforge/meta/MutableMetaProvider;ILspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun int (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun int$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;ILspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun int$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun listOfConvertable (Lspace/kscience/dataforge/meta/MutableMeta;Lspace/kscience/dataforge/meta/MetaConverter;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun listOfConvertable$default (Lspace/kscience/dataforge/meta/MutableMeta;Lspace/kscience/dataforge/meta/MetaConverter;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun listValue (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function1;Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun listValue$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function1;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun long (Lspace/kscience/dataforge/meta/MutableMetaProvider;JLspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun long (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun long$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;JLspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun long$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun node (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun node$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun number (Lspace/kscience/dataforge/meta/MutableMetaProvider;Ljava/lang/Number;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun number (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun number (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun number$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Ljava/lang/Number;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun number$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun number$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun numberList (Lspace/kscience/dataforge/meta/MutableMetaProvider;[Ljava/lang/Number;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun numberList$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;[Ljava/lang/Number;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun string (Lspace/kscience/dataforge/meta/MutableMetaProvider;Ljava/lang/String;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun string (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun string (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun string$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Ljava/lang/String;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun string$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun string$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function0;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun stringList (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun stringList (Lspace/kscience/dataforge/meta/MutableMetaProvider;[Ljava/lang/String;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun stringList$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun stringList$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;[Ljava/lang/String;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun value (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function1;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static final fun value (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun value$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function1;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
+ public static synthetic fun value$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/MutableMetaDelegate;
}
public final class space/kscience/dataforge/meta/MutableMetaKt {
public static final fun ObservableMutableMeta ()Lspace/kscience/dataforge/meta/ObservableMutableMeta;
public static final fun ObservableMutableMeta (Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/ObservableMutableMeta;
+ public static final fun ObservableMutableMeta (Lspace/kscience/dataforge/meta/Meta;)Lspace/kscience/dataforge/meta/ObservableMutableMeta;
public static synthetic fun ObservableMutableMeta$default (Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/ObservableMutableMeta;
public static final fun append (Lspace/kscience/dataforge/meta/MutableMeta;Ljava/lang/String;Lspace/kscience/dataforge/meta/Meta;)V
public static final fun append (Lspace/kscience/dataforge/meta/MutableMeta;Ljava/lang/String;Lspace/kscience/dataforge/meta/Value;)V
@@ -357,6 +491,7 @@ public final class space/kscience/dataforge/meta/MutableMetaKt {
public static final fun getOrCreate (Lspace/kscience/dataforge/meta/MutableTypedMeta;Ljava/lang/String;)Lspace/kscience/dataforge/meta/MutableTypedMeta;
public static final fun remove (Lspace/kscience/dataforge/meta/MutableMetaProvider;Ljava/lang/String;)V
public static final fun remove (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;)V
+ public static final fun reset (Lspace/kscience/dataforge/meta/MutableMeta;Lspace/kscience/dataforge/meta/Meta;)V
public static final fun set (Lspace/kscience/dataforge/meta/MutableMetaProvider;Ljava/lang/String;Ljava/lang/Iterable;)V
public static final fun set (Lspace/kscience/dataforge/meta/MutableMetaProvider;Ljava/lang/String;Lspace/kscience/dataforge/meta/Meta;)V
public static final fun set (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Ljava/lang/Iterable;)V
@@ -365,7 +500,7 @@ public final class space/kscience/dataforge/meta/MutableMetaKt {
public static final fun set (Lspace/kscience/dataforge/meta/MutableValueProvider;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/Value;)V
public static final fun setIndexed (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Ljava/lang/Iterable;Lkotlin/jvm/functions/Function2;)V
public static synthetic fun setIndexed$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/names/Name;Ljava/lang/Iterable;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)V
- public static final fun toMutableMeta (Lspace/kscience/dataforge/meta/Meta;)Lspace/kscience/dataforge/meta/ObservableMutableMeta;
+ public static final fun toMutableMeta (Lspace/kscience/dataforge/meta/Meta;)Lspace/kscience/dataforge/meta/MutableMeta;
public static final fun update (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/meta/Meta;)V
public static final fun withDefault (Lspace/kscience/dataforge/meta/MutableMeta;Lspace/kscience/dataforge/meta/MetaProvider;)Lspace/kscience/dataforge/meta/MutableMeta;
}
@@ -385,8 +520,16 @@ public final class space/kscience/dataforge/meta/MutableMetaSerializer : kotlinx
public fun serialize (Lkotlinx/serialization/encoding/Encoder;Lspace/kscience/dataforge/meta/MutableMeta;)V
}
+public final class space/kscience/dataforge/meta/MutableMetaViewKt {
+ public static final fun view (Lspace/kscience/dataforge/meta/MutableMeta;Ljava/lang/String;)Lspace/kscience/dataforge/meta/MutableMeta;
+ public static final fun view (Lspace/kscience/dataforge/meta/MutableMeta;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMeta;
+}
+
public abstract interface class space/kscience/dataforge/meta/MutableTypedMeta : space/kscience/dataforge/meta/MutableMeta, space/kscience/dataforge/meta/TypedMeta {
- public abstract fun get (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableTypedMeta;
+ public synthetic fun get (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/Meta;
+ public synthetic fun get (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMeta;
+ public fun get (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableTypedMeta;
+ public synthetic fun get (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/TypedMeta;
public abstract fun getOrCreate (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableTypedMeta;
}
@@ -419,32 +562,48 @@ public abstract interface class space/kscience/dataforge/meta/ObservableMeta : s
public abstract fun removeListener (Ljava/lang/Object;)V
}
-public final class space/kscience/dataforge/meta/ObservableMetaKt {
- public static final fun useProperty (Lspace/kscience/dataforge/meta/Scheme;Lkotlin/reflect/KProperty1;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)V
- public static synthetic fun useProperty$default (Lspace/kscience/dataforge/meta/Scheme;Lkotlin/reflect/KProperty1;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)V
-}
-
public final class space/kscience/dataforge/meta/ObservableMetaWrapperKt {
public static final fun asObservable (Lspace/kscience/dataforge/meta/MutableMeta;)Lspace/kscience/dataforge/meta/ObservableMutableMeta;
}
public abstract interface class space/kscience/dataforge/meta/ObservableMutableMeta : space/kscience/dataforge/meta/MutableMeta, space/kscience/dataforge/meta/MutableTypedMeta, space/kscience/dataforge/meta/ObservableMeta {
- public synthetic fun get (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/Meta;
- public synthetic fun get (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMeta;
- public synthetic fun get (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableTypedMeta;
- public fun get (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/ObservableMutableMeta;
- public synthetic fun get (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/TypedMeta;
- public abstract fun getOrCreate (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/ObservableMutableMeta;
+ public static final field Companion Lspace/kscience/dataforge/meta/ObservableMutableMeta$Companion;
+ public fun getSelf ()Lspace/kscience/dataforge/meta/ObservableMutableMeta;
+ public synthetic fun getSelf ()Lspace/kscience/dataforge/meta/TypedMeta;
}
-public abstract interface class space/kscience/dataforge/meta/ReadOnlySpecification : space/kscience/dataforge/meta/descriptors/Described {
- public abstract fun empty ()Ljava/lang/Object;
- public fun invoke (Lkotlin/jvm/functions/Function1;)Ljava/lang/Object;
- public abstract fun read (Lspace/kscience/dataforge/meta/Meta;)Ljava/lang/Object;
+public final class space/kscience/dataforge/meta/ObservableMutableMeta$Companion {
+ public final fun serializer ()Lkotlinx/serialization/KSerializer;
+}
+
+public final class space/kscience/dataforge/meta/ObservableMutableMetaSerializer : kotlinx/serialization/KSerializer {
+ public static final field INSTANCE Lspace/kscience/dataforge/meta/ObservableMutableMetaSerializer;
+ public synthetic fun deserialize (Lkotlinx/serialization/encoding/Decoder;)Ljava/lang/Object;
+ public fun deserialize (Lkotlinx/serialization/encoding/Decoder;)Lspace/kscience/dataforge/meta/ObservableMutableMeta;
+ public fun getDescriptor ()Lkotlinx/serialization/descriptors/SerialDescriptor;
+ public synthetic fun serialize (Lkotlinx/serialization/encoding/Encoder;Ljava/lang/Object;)V
+ public fun serialize (Lkotlinx/serialization/encoding/Encoder;Lspace/kscience/dataforge/meta/ObservableMutableMeta;)V
+}
+
+public final class space/kscience/dataforge/meta/RegexItemTransformationRule : space/kscience/dataforge/meta/TransformationRule {
+ public fun <init> (Lkotlin/text/Regex;Lkotlin/jvm/functions/Function4;)V
+ public final fun component1 ()Lkotlin/text/Regex;
+ public final fun component2 ()Lkotlin/jvm/functions/Function4;
+ public final fun copy (Lkotlin/text/Regex;Lkotlin/jvm/functions/Function4;)Lspace/kscience/dataforge/meta/RegexItemTransformationRule;
+ public static synthetic fun copy$default (Lspace/kscience/dataforge/meta/RegexItemTransformationRule;Lkotlin/text/Regex;Lkotlin/jvm/functions/Function4;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/RegexItemTransformationRule;
+ public fun equals (Ljava/lang/Object;)Z
+ public final fun getFrom ()Lkotlin/text/Regex;
+ public final fun getTransform ()Lkotlin/jvm/functions/Function4;
+ public fun hashCode ()I
+ public fun matches (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/Meta;)Z
+ public fun toString ()Ljava/lang/String;
+ public fun transformItem (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/Meta;Lspace/kscience/dataforge/meta/MutableMeta;)V
}
public class space/kscience/dataforge/meta/Scheme : space/kscience/dataforge/meta/Configurable, space/kscience/dataforge/meta/MetaRepr, space/kscience/dataforge/meta/MutableMetaProvider, space/kscience/dataforge/meta/descriptors/Described {
public fun <init> ()V
+ public fun <init> (Lspace/kscience/dataforge/meta/Meta;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;)V
+ public synthetic fun <init> (Lspace/kscience/dataforge/meta/Meta;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;ILkotlin/jvm/internal/DefaultConstructorMarker;)V
public synthetic fun get (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/Meta;
public fun get (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/MutableMeta;
public final fun getDescriptor ()Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;
@@ -454,6 +613,7 @@ public class space/kscience/dataforge/meta/Scheme : space/kscience/dataforge/met
public fun setValue (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/Value;)V
public fun toMeta ()Lspace/kscience/dataforge/meta/Laminate;
public synthetic fun toMeta ()Lspace/kscience/dataforge/meta/Meta;
+ public fun toString ()Ljava/lang/String;
public fun validate (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/Meta;)Z
}
@@ -461,20 +621,34 @@ public final class space/kscience/dataforge/meta/SchemeKt {
public static final fun copy (Lspace/kscience/dataforge/meta/Scheme;Lspace/kscience/dataforge/meta/SchemeSpec;Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/Scheme;
public static synthetic fun copy$default (Lspace/kscience/dataforge/meta/Scheme;Lspace/kscience/dataforge/meta/SchemeSpec;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/Scheme;
public static final fun invoke (Lspace/kscience/dataforge/meta/Scheme;Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/Scheme;
- public static final fun retarget (Lspace/kscience/dataforge/meta/Scheme;Lspace/kscience/dataforge/meta/MutableMeta;)Lspace/kscience/dataforge/meta/Scheme;
+ public static final fun listOfScheme (Lspace/kscience/dataforge/meta/MutableMeta;Lspace/kscience/dataforge/meta/SchemeSpec;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
+ public static final fun listOfScheme (Lspace/kscience/dataforge/meta/Scheme;Lspace/kscience/dataforge/meta/SchemeSpec;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
+ public static synthetic fun listOfScheme$default (Lspace/kscience/dataforge/meta/MutableMeta;Lspace/kscience/dataforge/meta/SchemeSpec;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
+ public static synthetic fun listOfScheme$default (Lspace/kscience/dataforge/meta/Scheme;Lspace/kscience/dataforge/meta/SchemeSpec;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
+ public static final fun scheme (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/meta/SchemeSpec;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
+ public static final fun scheme (Lspace/kscience/dataforge/meta/Scheme;Lspace/kscience/dataforge/meta/SchemeSpec;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
+ public static synthetic fun scheme$default (Lspace/kscience/dataforge/meta/MutableMetaProvider;Lspace/kscience/dataforge/meta/SchemeSpec;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
+ public static synthetic fun scheme$default (Lspace/kscience/dataforge/meta/Scheme;Lspace/kscience/dataforge/meta/SchemeSpec;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
+ public static final fun schemeOrNull (Lspace/kscience/dataforge/meta/MutableMeta;Lspace/kscience/dataforge/meta/SchemeSpec;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
+ public static final fun schemeOrNull (Lspace/kscience/dataforge/meta/Scheme;Lspace/kscience/dataforge/meta/SchemeSpec;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
+ public static synthetic fun schemeOrNull$default (Lspace/kscience/dataforge/meta/MutableMeta;Lspace/kscience/dataforge/meta/SchemeSpec;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
+ public static synthetic fun schemeOrNull$default (Lspace/kscience/dataforge/meta/Scheme;Lspace/kscience/dataforge/meta/SchemeSpec;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
+ public static final fun updateWith (Lspace/kscience/dataforge/meta/Configurable;Lspace/kscience/dataforge/meta/SchemeSpec;Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/Scheme;
+ public static final fun updateWith (Lspace/kscience/dataforge/meta/MutableMeta;Lspace/kscience/dataforge/meta/SchemeSpec;Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/Scheme;
+ public static final fun useProperty (Lspace/kscience/dataforge/meta/Scheme;Lkotlin/reflect/KProperty1;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;)V
+ public static synthetic fun useProperty$default (Lspace/kscience/dataforge/meta/Scheme;Lkotlin/reflect/KProperty1;Ljava/lang/Object;Lkotlin/jvm/functions/Function2;ILjava/lang/Object;)V
}
-public class space/kscience/dataforge/meta/SchemeSpec : space/kscience/dataforge/meta/Specification {
+public class space/kscience/dataforge/meta/SchemeSpec : space/kscience/dataforge/meta/MetaConverter {
public fun <init> (Lkotlin/jvm/functions/Function0;)V
- public synthetic fun empty ()Ljava/lang/Object;
- public fun empty ()Lspace/kscience/dataforge/meta/Scheme;
+ public synthetic fun convert (Ljava/lang/Object;)Lspace/kscience/dataforge/meta/Meta;
+ public fun convert (Lspace/kscience/dataforge/meta/Scheme;)Lspace/kscience/dataforge/meta/Meta;
+ public final fun empty ()Lspace/kscience/dataforge/meta/Scheme;
public fun getDescriptor ()Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;
- public synthetic fun invoke (Lkotlin/jvm/functions/Function1;)Ljava/lang/Object;
public final fun invoke (Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/Scheme;
- public synthetic fun read (Lspace/kscience/dataforge/meta/Meta;)Ljava/lang/Object;
- public fun read (Lspace/kscience/dataforge/meta/Meta;)Lspace/kscience/dataforge/meta/Scheme;
- public synthetic fun write (Lspace/kscience/dataforge/meta/MutableMeta;)Ljava/lang/Object;
- public fun write (Lspace/kscience/dataforge/meta/MutableMeta;)Lspace/kscience/dataforge/meta/Scheme;
+ public synthetic fun readOrNull (Lspace/kscience/dataforge/meta/Meta;)Ljava/lang/Object;
+ public fun readOrNull (Lspace/kscience/dataforge/meta/Meta;)Lspace/kscience/dataforge/meta/Scheme;
+ public final fun write (Lspace/kscience/dataforge/meta/MutableMeta;)Lspace/kscience/dataforge/meta/Scheme;
}
public final class space/kscience/dataforge/meta/SealedMeta : space/kscience/dataforge/meta/TypedMeta {
@@ -482,19 +656,21 @@ public final class space/kscience/dataforge/meta/SealedMeta : space/kscience/dat
public fun <init> (Lspace/kscience/dataforge/meta/Value;Ljava/util/Map;)V
public fun equals (Ljava/lang/Object;)Z
public fun getItems ()Ljava/util/Map;
+ public fun getSelf ()Lspace/kscience/dataforge/meta/SealedMeta;
+ public synthetic fun getSelf ()Lspace/kscience/dataforge/meta/TypedMeta;
public fun getValue ()Lspace/kscience/dataforge/meta/Value;
public fun hashCode ()I
public fun toString ()Ljava/lang/String;
}
-public final class space/kscience/dataforge/meta/SealedMeta$$serializer : kotlinx/serialization/internal/GeneratedSerializer {
+public synthetic class space/kscience/dataforge/meta/SealedMeta$$serializer : kotlinx/serialization/internal/GeneratedSerializer {
public static final field INSTANCE Lspace/kscience/dataforge/meta/SealedMeta$$serializer;
- public fun childSerializers ()[Lkotlinx/serialization/KSerializer;
+ public final fun childSerializers ()[Lkotlinx/serialization/KSerializer;
public synthetic fun deserialize (Lkotlinx/serialization/encoding/Decoder;)Ljava/lang/Object;
- public fun deserialize (Lkotlinx/serialization/encoding/Decoder;)Lspace/kscience/dataforge/meta/SealedMeta;
- public fun getDescriptor ()Lkotlinx/serialization/descriptors/SerialDescriptor;
+ public final fun deserialize (Lkotlinx/serialization/encoding/Decoder;)Lspace/kscience/dataforge/meta/SealedMeta;
+ public final fun getDescriptor ()Lkotlinx/serialization/descriptors/SerialDescriptor;
public synthetic fun serialize (Lkotlinx/serialization/encoding/Encoder;Ljava/lang/Object;)V
- public fun serialize (Lkotlinx/serialization/encoding/Encoder;Lspace/kscience/dataforge/meta/SealedMeta;)V
+ public final fun serialize (Lkotlinx/serialization/encoding/Encoder;Lspace/kscience/dataforge/meta/SealedMeta;)V
public fun typeParametersSerializers ()[Lkotlinx/serialization/KSerializer;
}
@@ -515,21 +691,20 @@ public final class space/kscience/dataforge/meta/SealedMetaKt {
public static final fun seal (Lspace/kscience/dataforge/meta/Meta;)Lspace/kscience/dataforge/meta/SealedMeta;
}
-public abstract interface class space/kscience/dataforge/meta/Specification : space/kscience/dataforge/meta/ReadOnlySpecification {
- public abstract fun write (Lspace/kscience/dataforge/meta/MutableMeta;)Ljava/lang/Object;
-}
-
-public final class space/kscience/dataforge/meta/SpecificationKt {
- public static final fun spec (Lspace/kscience/dataforge/meta/MutableMeta;Lspace/kscience/dataforge/meta/Specification;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static final fun spec (Lspace/kscience/dataforge/meta/Scheme;Lspace/kscience/dataforge/meta/Specification;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun spec$default (Lspace/kscience/dataforge/meta/MutableMeta;Lspace/kscience/dataforge/meta/Specification;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun spec$default (Lspace/kscience/dataforge/meta/Scheme;Lspace/kscience/dataforge/meta/Specification;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static final fun specOrNull (Lspace/kscience/dataforge/meta/MutableMeta;Lspace/kscience/dataforge/meta/Specification;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static final fun specOrNull (Lspace/kscience/dataforge/meta/Scheme;Lspace/kscience/dataforge/meta/Specification;Lspace/kscience/dataforge/names/Name;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun specOrNull$default (Lspace/kscience/dataforge/meta/MutableMeta;Lspace/kscience/dataforge/meta/Specification;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static synthetic fun specOrNull$default (Lspace/kscience/dataforge/meta/Scheme;Lspace/kscience/dataforge/meta/Specification;Lspace/kscience/dataforge/names/Name;ILjava/lang/Object;)Lkotlin/properties/ReadWriteProperty;
- public static final fun updateWith (Lspace/kscience/dataforge/meta/Configurable;Lspace/kscience/dataforge/meta/Specification;Lkotlin/jvm/functions/Function1;)Ljava/lang/Object;
- public static final fun updateWith (Lspace/kscience/dataforge/meta/MutableMeta;Lspace/kscience/dataforge/meta/Specification;Lkotlin/jvm/functions/Function1;)Ljava/lang/Object;
+public final class space/kscience/dataforge/meta/SingleItemTransformationRule : space/kscience/dataforge/meta/TransformationRule {
+ public fun <init> (Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function3;)V
+ public final fun component1 ()Lspace/kscience/dataforge/names/Name;
+ public final fun component2 ()Lkotlin/jvm/functions/Function3;
+ public final fun copy (Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function3;)Lspace/kscience/dataforge/meta/SingleItemTransformationRule;
+ public static synthetic fun copy$default (Lspace/kscience/dataforge/meta/SingleItemTransformationRule;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function3;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/SingleItemTransformationRule;
+ public fun equals (Ljava/lang/Object;)Z
+ public final fun getFrom ()Lspace/kscience/dataforge/names/Name;
+ public final fun getTransform ()Lkotlin/jvm/functions/Function3;
+ public fun hashCode ()I
+ public fun matches (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/Meta;)Z
+ public fun selectItems (Lspace/kscience/dataforge/meta/Meta;)Lkotlin/sequences/Sequence;
+ public fun toString ()Ljava/lang/String;
+ public fun transformItem (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/Meta;Lspace/kscience/dataforge/meta/MutableMeta;)V
}
public final class space/kscience/dataforge/meta/StringValue : space/kscience/dataforge/meta/Value {
@@ -550,6 +725,12 @@ public final class space/kscience/dataforge/meta/StringValue : space/kscience/da
public final synthetic fun unbox-impl ()Ljava/lang/String;
}
+public abstract interface class space/kscience/dataforge/meta/TransformationRule {
+ public abstract fun matches (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/Meta;)Z
+ public fun selectItems (Lspace/kscience/dataforge/meta/Meta;)Lkotlin/sequences/Sequence;
+ public abstract fun transformItem (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/Meta;Lspace/kscience/dataforge/meta/MutableMeta;)V
+}
+
public final class space/kscience/dataforge/meta/True : space/kscience/dataforge/meta/Value {
public static final field INSTANCE Lspace/kscience/dataforge/meta/True;
public fun equals (Ljava/lang/Object;)Z
@@ -563,6 +744,7 @@ public abstract interface class space/kscience/dataforge/meta/TypedMeta : space/
public synthetic fun get (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/Meta;
public fun get (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/meta/TypedMeta;
public abstract fun getItems ()Ljava/util/Map;
+ public abstract fun getSelf ()Lspace/kscience/dataforge/meta/TypedMeta;
public fun toMeta ()Lspace/kscience/dataforge/meta/Meta;
}
@@ -587,7 +769,6 @@ public final class space/kscience/dataforge/meta/Value$Companion {
public final class space/kscience/dataforge/meta/ValueExtensionsKt {
public static final fun getBoolean (Lspace/kscience/dataforge/meta/Value;)Z
public static final fun getDouble (Lspace/kscience/dataforge/meta/Value;)D
- public static final fun getDoubleArray (Lspace/kscience/dataforge/meta/Value;)[D
public static final fun getFloat (Lspace/kscience/dataforge/meta/Value;)F
public static final fun getInt (Lspace/kscience/dataforge/meta/Value;)I
public static final fun getLong (Lspace/kscience/dataforge/meta/Value;)J
@@ -606,7 +787,6 @@ public final class space/kscience/dataforge/meta/ValueKt {
public static final fun asValue (Ljava/lang/Number;)Lspace/kscience/dataforge/meta/Value;
public static final fun asValue (Ljava/lang/String;)Lspace/kscience/dataforge/meta/Value;
public static final fun asValue (Z)Lspace/kscience/dataforge/meta/Value;
- public static final fun asValue ([B)Lspace/kscience/dataforge/meta/Value;
public static final fun asValue ([F)Lspace/kscience/dataforge/meta/Value;
public static final fun asValue ([I)Lspace/kscience/dataforge/meta/Value;
public static final fun asValue ([J)Lspace/kscience/dataforge/meta/Value;
@@ -692,20 +872,21 @@ public final class space/kscience/dataforge/meta/descriptors/MetaDescriptor {
public final fun getDescription ()Ljava/lang/String;
public final fun getIndexKey ()Ljava/lang/String;
public final fun getMultiple ()Z
+ public final fun getNodes ()Ljava/util/Map;
public final fun getValueRestriction ()Lspace/kscience/dataforge/meta/descriptors/ValueRestriction;
public final fun getValueTypes ()Ljava/util/List;
public fun hashCode ()I
public fun toString ()Ljava/lang/String;
}
-public final class space/kscience/dataforge/meta/descriptors/MetaDescriptor$$serializer : kotlinx/serialization/internal/GeneratedSerializer {
+public synthetic class space/kscience/dataforge/meta/descriptors/MetaDescriptor$$serializer : kotlinx/serialization/internal/GeneratedSerializer {
public static final field INSTANCE Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor$$serializer;
- public fun childSerializers ()[Lkotlinx/serialization/KSerializer;
+ public final fun childSerializers ()[Lkotlinx/serialization/KSerializer;
public synthetic fun deserialize (Lkotlinx/serialization/encoding/Decoder;)Ljava/lang/Object;
- public fun deserialize (Lkotlinx/serialization/encoding/Decoder;)Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;
- public fun getDescriptor ()Lkotlinx/serialization/descriptors/SerialDescriptor;
+ public final fun deserialize (Lkotlinx/serialization/encoding/Decoder;)Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;
+ public final fun getDescriptor ()Lkotlinx/serialization/descriptors/SerialDescriptor;
public synthetic fun serialize (Lkotlinx/serialization/encoding/Encoder;Ljava/lang/Object;)V
- public fun serialize (Lkotlinx/serialization/encoding/Encoder;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;)V
+ public final fun serialize (Lkotlinx/serialization/encoding/Encoder;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;)V
public fun typeParametersSerializers ()[Lkotlinx/serialization/KSerializer;
}
@@ -720,25 +901,23 @@ public final class space/kscience/dataforge/meta/descriptors/MetaDescriptorBuild
public final fun attributes (Lkotlin/jvm/functions/Function1;)V
public final fun build ()Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;
public final fun default (Ljava/lang/Object;)V
+ public final fun from (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;)V
public final fun getAllowedValues ()Ljava/util/List;
public final fun getAttributes ()Lspace/kscience/dataforge/meta/MutableMeta;
public final fun getChildren ()Ljava/util/Map;
public final fun getDefault ()Lspace/kscience/dataforge/meta/Value;
+ public final fun getDescription ()Ljava/lang/String;
public final fun getIndexKey ()Ljava/lang/String;
- public final fun getInfo ()Ljava/lang/String;
public final fun getMultiple ()Z
public final fun getValueRestriction ()Lspace/kscience/dataforge/meta/descriptors/ValueRestriction;
public final fun getValueTypes ()Ljava/util/List;
- public final fun item (Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;
- public static synthetic fun item$default (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;
- public final fun node (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;
- public static synthetic fun node$default (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;
+ public final fun node (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;)V
public final fun setAllowedValues (Ljava/util/List;)V
public final fun setAttributes (Lspace/kscience/dataforge/meta/MutableMeta;)V
public final fun setChildren (Ljava/util/Map;)V
public final fun setDefault (Lspace/kscience/dataforge/meta/Value;)V
+ public final fun setDescription (Ljava/lang/String;)V
public final fun setIndexKey (Ljava/lang/String;)V
- public final fun setInfo (Ljava/lang/String;)V
public final fun setMultiple (Z)V
public final fun setValueRestriction (Lspace/kscience/dataforge/meta/descriptors/ValueRestriction;)V
public final fun setValueTypes (Ljava/util/List;)V
@@ -749,16 +928,16 @@ public final class space/kscience/dataforge/meta/descriptors/MetaDescriptorBuild
public static final fun MetaDescriptor (Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;
public static final fun copy (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;
public static synthetic fun copy$default (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;
- public static final fun item (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;Ljava/lang/String;Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;
public static final fun node (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;Ljava/lang/String;Lkotlin/jvm/functions/Function1;)V
public static final fun node (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;Ljava/lang/String;Lspace/kscience/dataforge/meta/descriptors/Described;Lkotlin/jvm/functions/Function1;)V
- public static final fun node (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;
+ public static final fun node (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;Ljava/lang/String;Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;)V
+ public static final fun node (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function1;)V
public static synthetic fun node$default (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;Ljava/lang/String;Lspace/kscience/dataforge/meta/descriptors/Described;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)V
public static final fun required (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;)V
- public static final fun value (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;Ljava/lang/String;Lspace/kscience/dataforge/meta/ValueType;[Lspace/kscience/dataforge/meta/ValueType;Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;
- public static final fun value (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/ValueType;[Lspace/kscience/dataforge/meta/ValueType;Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;
- public static synthetic fun value$default (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;Ljava/lang/String;Lspace/kscience/dataforge/meta/ValueType;[Lspace/kscience/dataforge/meta/ValueType;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;
- public static synthetic fun value$default (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/ValueType;[Lspace/kscience/dataforge/meta/ValueType;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;
+ public static final fun value (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;Ljava/lang/String;Lspace/kscience/dataforge/meta/ValueType;[Lspace/kscience/dataforge/meta/ValueType;Lkotlin/jvm/functions/Function1;)V
+ public static final fun value (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/ValueType;[Lspace/kscience/dataforge/meta/ValueType;Lkotlin/jvm/functions/Function1;)V
+ public static synthetic fun value$default (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;Ljava/lang/String;Lspace/kscience/dataforge/meta/ValueType;[Lspace/kscience/dataforge/meta/ValueType;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)V
+ public static synthetic fun value$default (Lspace/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/ValueType;[Lspace/kscience/dataforge/meta/ValueType;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)V
}
public final class space/kscience/dataforge/meta/descriptors/MetaDescriptorKt {
@@ -772,6 +951,7 @@ public final class space/kscience/dataforge/meta/descriptors/MetaDescriptorKt {
public final class space/kscience/dataforge/meta/descriptors/ValueRestriction : java/lang/Enum {
public static final field ABSENT Lspace/kscience/dataforge/meta/descriptors/ValueRestriction;
+ public static final field Companion Lspace/kscience/dataforge/meta/descriptors/ValueRestriction$Companion;
public static final field NONE Lspace/kscience/dataforge/meta/descriptors/ValueRestriction;
public static final field REQUIRED Lspace/kscience/dataforge/meta/descriptors/ValueRestriction;
public static fun getEntries ()Lkotlin/enums/EnumEntries;
@@ -779,119 +959,8 @@ public final class space/kscience/dataforge/meta/descriptors/ValueRestriction :
public static fun values ()[Lspace/kscience/dataforge/meta/descriptors/ValueRestriction;
}
-public final class space/kscience/dataforge/meta/transformations/KeepTransformationRule : space/kscience/dataforge/meta/transformations/TransformationRule {
- public fun <init> (Lkotlin/jvm/functions/Function1;)V
- public final fun component1 ()Lkotlin/jvm/functions/Function1;
- public final fun copy (Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/transformations/KeepTransformationRule;
- public static synthetic fun copy$default (Lspace/kscience/dataforge/meta/transformations/KeepTransformationRule;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/transformations/KeepTransformationRule;
- public fun equals (Ljava/lang/Object;)Z
- public final fun getSelector ()Lkotlin/jvm/functions/Function1;
- public fun hashCode ()I
- public fun matches (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/Meta;)Z
- public fun selectItems (Lspace/kscience/dataforge/meta/Meta;)Lkotlin/sequences/Sequence;
- public fun toString ()Ljava/lang/String;
- public fun transformItem (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/Meta;Lspace/kscience/dataforge/meta/MutableMeta;)V
-}
-
-public abstract interface class space/kscience/dataforge/meta/transformations/MetaConverter {
- public static final field Companion Lspace/kscience/dataforge/meta/transformations/MetaConverter$Companion;
- public fun getDescriptor ()Lspace/kscience/dataforge/meta/descriptors/MetaDescriptor;
- public abstract fun getType ()Lkotlin/reflect/KType;
- public fun metaToObject (Lspace/kscience/dataforge/meta/Meta;)Ljava/lang/Object;
- public abstract fun metaToObjectOrNull (Lspace/kscience/dataforge/meta/Meta;)Ljava/lang/Object;
- public abstract fun objectToMeta (Ljava/lang/Object;)Lspace/kscience/dataforge/meta/Meta;
-}
-
-public final class space/kscience/dataforge/meta/transformations/MetaConverter$Companion {
- public final fun getBoolean ()Lspace/kscience/dataforge/meta/transformations/MetaConverter;
- public final fun getDouble ()Lspace/kscience/dataforge/meta/transformations/MetaConverter;
- public final fun getFloat ()Lspace/kscience/dataforge/meta/transformations/MetaConverter;
- public final fun getInt ()Lspace/kscience/dataforge/meta/transformations/MetaConverter;
- public final fun getLong ()Lspace/kscience/dataforge/meta/transformations/MetaConverter;
- public final fun getMeta ()Lspace/kscience/dataforge/meta/transformations/MetaConverter;
- public final fun getNumber ()Lspace/kscience/dataforge/meta/transformations/MetaConverter;
- public final fun getString ()Lspace/kscience/dataforge/meta/transformations/MetaConverter;
- public final fun getValue ()Lspace/kscience/dataforge/meta/transformations/MetaConverter;
- public final fun valueList (Lkotlin/jvm/functions/Function1;Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/meta/transformations/MetaConverter;
- public static synthetic fun valueList$default (Lspace/kscience/dataforge/meta/transformations/MetaConverter$Companion;Lkotlin/jvm/functions/Function1;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/transformations/MetaConverter;
-}
-
-public final class space/kscience/dataforge/meta/transformations/MetaConverterKt {
- public static final fun nullableMetaToObject (Lspace/kscience/dataforge/meta/transformations/MetaConverter;Lspace/kscience/dataforge/meta/Meta;)Ljava/lang/Object;
- public static final fun nullableObjectToMeta (Lspace/kscience/dataforge/meta/transformations/MetaConverter;Ljava/lang/Object;)Lspace/kscience/dataforge/meta/Meta;
- public static final fun valueToObject (Lspace/kscience/dataforge/meta/transformations/MetaConverter;Lspace/kscience/dataforge/meta/Value;)Ljava/lang/Object;
-}
-
-public final class space/kscience/dataforge/meta/transformations/MetaTransformation {
- public static final field Companion Lspace/kscience/dataforge/meta/transformations/MetaTransformation$Companion;
- public static final fun apply-impl (Ljava/util/Collection;Lspace/kscience/dataforge/meta/Meta;)Lspace/kscience/dataforge/meta/Meta;
- public static final fun bind-impl (Ljava/util/Collection;Lspace/kscience/dataforge/meta/ObservableMeta;Lspace/kscience/dataforge/meta/MutableMeta;)V
- public static final synthetic fun box-impl (Ljava/util/Collection;)Lspace/kscience/dataforge/meta/transformations/MetaTransformation;
- public static fun constructor-impl (Ljava/util/Collection;)Ljava/util/Collection;
- public fun equals (Ljava/lang/Object;)Z
- public static fun equals-impl (Ljava/util/Collection;Ljava/lang/Object;)Z
- public static final fun equals-impl0 (Ljava/util/Collection;Ljava/util/Collection;)Z
- public static final fun generate-impl (Ljava/util/Collection;Lspace/kscience/dataforge/meta/Meta;)Lspace/kscience/dataforge/meta/Meta;
- public fun hashCode ()I
- public static fun hashCode-impl (Ljava/util/Collection;)I
- public fun toString ()Ljava/lang/String;
- public static fun toString-impl (Ljava/util/Collection;)Ljava/lang/String;
- public final synthetic fun unbox-impl ()Ljava/util/Collection;
-}
-
-public final class space/kscience/dataforge/meta/transformations/MetaTransformation$Companion {
- public final fun make-XNaMui4 (Lkotlin/jvm/functions/Function1;)Ljava/util/Collection;
-}
-
-public final class space/kscience/dataforge/meta/transformations/MetaTransformationBuilder {
- public fun <init> ()V
- public final fun build-050menU ()Ljava/util/Collection;
- public final fun keep (Ljava/lang/String;)V
- public final fun keep (Lkotlin/jvm/functions/Function1;)V
- public final fun keep (Lspace/kscience/dataforge/names/Name;)V
- public final fun move (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function1;)V
- public static synthetic fun move$default (Lspace/kscience/dataforge/meta/transformations/MetaTransformationBuilder;Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)V
-}
-
-public final class space/kscience/dataforge/meta/transformations/RegexItemTransformationRule : space/kscience/dataforge/meta/transformations/TransformationRule {
- public fun <init> (Lkotlin/text/Regex;Lkotlin/jvm/functions/Function4;)V
- public final fun component1 ()Lkotlin/text/Regex;
- public final fun component2 ()Lkotlin/jvm/functions/Function4;
- public final fun copy (Lkotlin/text/Regex;Lkotlin/jvm/functions/Function4;)Lspace/kscience/dataforge/meta/transformations/RegexItemTransformationRule;
- public static synthetic fun copy$default (Lspace/kscience/dataforge/meta/transformations/RegexItemTransformationRule;Lkotlin/text/Regex;Lkotlin/jvm/functions/Function4;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/transformations/RegexItemTransformationRule;
- public fun equals (Ljava/lang/Object;)Z
- public final fun getFrom ()Lkotlin/text/Regex;
- public final fun getTransform ()Lkotlin/jvm/functions/Function4;
- public fun hashCode ()I
- public fun matches (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/Meta;)Z
- public fun toString ()Ljava/lang/String;
- public fun transformItem (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/Meta;Lspace/kscience/dataforge/meta/MutableMeta;)V
-}
-
-public final class space/kscience/dataforge/meta/transformations/SingleItemTransformationRule : space/kscience/dataforge/meta/transformations/TransformationRule {
- public fun <init> (Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function3;)V
- public final fun component1 ()Lspace/kscience/dataforge/names/Name;
- public final fun component2 ()Lkotlin/jvm/functions/Function3;
- public final fun copy (Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function3;)Lspace/kscience/dataforge/meta/transformations/SingleItemTransformationRule;
- public static synthetic fun copy$default (Lspace/kscience/dataforge/meta/transformations/SingleItemTransformationRule;Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function3;ILjava/lang/Object;)Lspace/kscience/dataforge/meta/transformations/SingleItemTransformationRule;
- public fun equals (Ljava/lang/Object;)Z
- public final fun getFrom ()Lspace/kscience/dataforge/names/Name;
- public final fun getTransform ()Lkotlin/jvm/functions/Function3;
- public fun hashCode ()I
- public fun matches (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/Meta;)Z
- public fun selectItems (Lspace/kscience/dataforge/meta/Meta;)Lkotlin/sequences/Sequence;
- public fun toString ()Ljava/lang/String;
- public fun transformItem (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/Meta;Lspace/kscience/dataforge/meta/MutableMeta;)V
-}
-
-public abstract interface class space/kscience/dataforge/meta/transformations/TransformationRule {
- public abstract fun matches (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/Meta;)Z
- public fun selectItems (Lspace/kscience/dataforge/meta/Meta;)Lkotlin/sequences/Sequence;
- public abstract fun transformItem (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/meta/Meta;Lspace/kscience/dataforge/meta/MutableMeta;)V
-}
-
-public final class space/kscience/dataforge/misc/CastJvmKt {
- public static final fun unsafeCast (Ljava/lang/Object;)Ljava/lang/Object;
+public final class space/kscience/dataforge/meta/descriptors/ValueRestriction$Companion {
+ public final fun serializer ()Lkotlinx/serialization/KSerializer;
}
public abstract interface annotation class space/kscience/dataforge/misc/DFBuilder : java/lang/annotation/Annotation {
@@ -903,7 +972,7 @@ public abstract interface annotation class space/kscience/dataforge/misc/DFExper
public abstract interface annotation class space/kscience/dataforge/misc/DFInternal : java/lang/annotation/Annotation {
}
-public abstract interface annotation class space/kscience/dataforge/misc/DfId : java/lang/annotation/Annotation {
+public abstract interface annotation class space/kscience/dataforge/misc/DfType : java/lang/annotation/Annotation {
public abstract fun id ()Ljava/lang/String;
}
@@ -920,6 +989,9 @@ public final class space/kscience/dataforge/misc/NamedKt {
public static final fun isAnonymous (Lspace/kscience/dataforge/misc/Named;)Z
}
+public abstract interface annotation class space/kscience/dataforge/misc/UnsafeKType : java/lang/annotation/Annotation {
+}
+
public final class space/kscience/dataforge/names/Name {
public static final field Companion Lspace/kscience/dataforge/names/Name$Companion;
public static final field NAME_SEPARATOR Ljava/lang/String;
@@ -937,6 +1009,16 @@ public final class space/kscience/dataforge/names/Name$Companion {
public final fun serializer ()Lkotlinx/serialization/KSerializer;
}
+public final class space/kscience/dataforge/names/NameIndexComparator : java/util/Comparator {
+ public static final field INSTANCE Lspace/kscience/dataforge/names/NameIndexComparator;
+ public synthetic fun compare (Ljava/lang/Object;Ljava/lang/Object;)I
+ public fun compare (Ljava/lang/String;Ljava/lang/String;)I
+}
+
+public final class space/kscience/dataforge/names/NameIndexComparatorKt {
+ public static final fun getIndexedList (Lspace/kscience/dataforge/meta/Meta;Lspace/kscience/dataforge/names/Name;)Ljava/util/List;
+}
+
public final class space/kscience/dataforge/names/NameKt {
public static final fun appendFirst (Lspace/kscience/dataforge/names/Name;Ljava/lang/String;)Lspace/kscience/dataforge/names/Name;
public static final fun appendLeft (Lspace/kscience/dataforge/names/Name;Ljava/lang/String;)Lspace/kscience/dataforge/names/Name;
@@ -944,6 +1026,7 @@ public final class space/kscience/dataforge/names/NameKt {
public static final fun asName (Lspace/kscience/dataforge/names/NameToken;)Lspace/kscience/dataforge/names/Name;
public static final fun cutFirst (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/names/Name;
public static final fun cutLast (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/names/Name;
+ public static final fun endsWith (Lspace/kscience/dataforge/names/Name;Ljava/lang/String;)Z
public static final fun endsWith (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/names/Name;)Z
public static final fun endsWith (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/names/NameToken;)Z
public static final fun first (Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/names/NameToken;
@@ -966,8 +1049,10 @@ public final class space/kscience/dataforge/names/NameKt {
public static final fun removeHeadOrNull (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/names/Name;)Lspace/kscience/dataforge/names/Name;
public static final fun replaceLast (Lspace/kscience/dataforge/names/Name;Lkotlin/jvm/functions/Function1;)Lspace/kscience/dataforge/names/Name;
public static final fun set (Ljava/util/Map;Ljava/lang/String;Ljava/lang/Object;)V
+ public static final fun startsWith (Lspace/kscience/dataforge/names/Name;Ljava/lang/String;)Z
public static final fun startsWith (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/names/Name;)Z
public static final fun startsWith (Lspace/kscience/dataforge/names/Name;Lspace/kscience/dataforge/names/NameToken;)Z
+ public static final fun toStringUnescaped (Lspace/kscience/dataforge/names/Name;)Ljava/lang/String;
public static final fun withIndex (Lspace/kscience/dataforge/names/Name;Ljava/lang/String;)Lspace/kscience/dataforge/names/Name;
}
diff --git a/dataforge-meta/build.gradle.kts b/dataforge-meta/build.gradle.kts
index 51b07113..49d8bdc5 100644
--- a/dataforge-meta/build.gradle.kts
+++ b/dataforge-meta/build.gradle.kts
@@ -6,13 +6,58 @@ kscience {
jvm()
js()
native()
- useSerialization{
+ wasm()
+ useSerialization {
json()
}
}
description = "Meta definition and basic operations on meta"
-readme{
+readme {
maturity = space.kscience.gradle.Maturity.DEVELOPMENT
+
+ description = """
+ Core Meta and Name manipulation module
+ """.trimIndent()
+
+ feature(
+ "meta",
+ ref = "src/commonMain/kotlin/space/kscience/dataforge/meta/Meta.kt",
+ name = "Meta"
+ ) {
+ """
+ **Meta** is the representation of basic DataForge concept: Metadata, but it also could be called meta-value tree.
+
+ Each Meta node could hava a node Value as well as a map of named child items.
+
+ """.trimIndent()
+ }
+
+ feature(
+ "value",
+ ref = "src/commonMain/kotlin/space/kscience/dataforge/meta/Value.kt",
+ name = "Value"
+ ) {
+ """
+ **Value** a sum type for different meta values.
+
+ The following types are implemented in core (custom ones are also available):
+ * null
+ * boolean
+ * number
+ * string
+ * list of values
+ """.trimIndent()
+ }
+
+ feature(
+ "name",
+ ref = "src/commonMain/kotlin/space/kscience/dataforge/names/Name.kt",
+ name = "Name"
+ ) {
+ """
+ **Name** is an identifier to access tree-like structure.
+ """.trimIndent()
+ }
}
\ No newline at end of file
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/JsonMeta.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/JsonMeta.kt
index 36373582..1817ba64 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/JsonMeta.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/JsonMeta.kt
@@ -31,12 +31,12 @@ private fun Meta.toJsonWithIndex(descriptor: MetaDescriptor?, index: String?): J
val pairs: MutableList<Pair<String, JsonElement>> = items.entries.groupBy {
it.key.body
}.mapTo(ArrayList()) { (body, list) ->
- val childDescriptor = descriptor?.children?.get(body)
+ val childDescriptor = descriptor?.nodes?.get(body)
if (list.size == 1) {
val (token, element) = list.first()
- //do not add an empty element
- val child: JsonElement = element.toJsonWithIndex(childDescriptor, token.index)
- if(token.index == null) {
+ //do not add an empty element
+ val child: JsonElement = element.toJsonWithIndex(childDescriptor, token.index)
+ if (token.index == null) {
body to child
} else {
body to JsonArray(listOf(child))
@@ -106,7 +106,7 @@ private fun JsonElement.toValueOrNull(descriptor: MetaDescriptor?): Value? = whe
private fun MutableMap<NameToken, SealedMeta>.addJsonElement(
key: String,
element: JsonElement,
- descriptor: MetaDescriptor?
+ descriptor: MetaDescriptor?,
) {
when (element) {
is JsonPrimitive -> put(NameToken(key), Meta(element.toValue(descriptor)))
@@ -117,8 +117,11 @@ private fun MutableMap<NameToken, SealedMeta>.addJsonElement(
} else {
val indexKey = descriptor?.indexKey ?: Meta.INDEX_KEY
element.forEachIndexed { serial, childElement ->
- val index = (childElement as? JsonObject)?.get(indexKey)?.jsonPrimitive?.content
+
+ val index = (childElement as? JsonObject)
+ ?.get(indexKey)?.jsonPrimitive?.content
?: serial.toString()
+
val child: SealedMeta = when (childElement) {
is JsonObject -> childElement.toMeta(descriptor)
is JsonArray -> {
@@ -133,12 +136,14 @@ private fun MutableMap<NameToken, SealedMeta>.addJsonElement(
Meta(childValue)
}
}
+
is JsonPrimitive -> Meta(childElement.toValue(null))
}
put(NameToken(key, index), child)
}
}
}
+
is JsonObject -> {
val indexKey = descriptor?.indexKey ?: Meta.INDEX_KEY
val index = element[indexKey]?.jsonPrimitive?.content
@@ -160,11 +165,15 @@ public fun JsonObject.toMeta(descriptor: MetaDescriptor? = null): SealedMeta {
public fun JsonElement.toMeta(descriptor: MetaDescriptor? = null): SealedMeta = when (this) {
is JsonPrimitive -> Meta(toValue(descriptor))
is JsonObject -> toMeta(descriptor)
- is JsonArray -> SealedMeta(null,
- linkedMapOf<NameToken, SealedMeta>().apply {
- addJsonElement(Meta.JSON_ARRAY_KEY, this@toMeta, null)
- }
- )
+ is JsonArray -> if (all { it is JsonPrimitive }) {
+ Meta(map { it.toValueOrNull(descriptor) ?: error("Unreachable: should not contain objects") }.asValue())
+ } else {
+ SealedMeta(null,
+ linkedMapOf<NameToken, SealedMeta>().apply {
+ addJsonElement(Meta.JSON_ARRAY_KEY, this@toMeta, null)
+ }
+ )
+ }
}
//
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Laminate.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Laminate.kt
index 87284107..0ae84bb6 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Laminate.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Laminate.kt
@@ -9,6 +9,8 @@ import space.kscience.dataforge.names.NameToken
*/
public class Laminate internal constructor(public val layers: List<Meta>) : TypedMeta<Laminate> {
+ override val self: Laminate get() = this
+
override val value: Value? = layers.firstNotNullOfOrNull { it.value }
override val items: Map<NameToken, Laminate> by lazy {
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Meta.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Meta.kt
index 420625ca..fd953085 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Meta.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Meta.kt
@@ -2,8 +2,7 @@ package space.kscience.dataforge.meta
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.Json
-import space.kscience.dataforge.misc.DfId
-import space.kscience.dataforge.misc.unsafeCast
+import space.kscience.dataforge.misc.DfType
import space.kscience.dataforge.names.*
import kotlin.jvm.JvmName
@@ -31,7 +30,7 @@ public fun interface MetaProvider : ValueProvider {
* TODO add documentation
* Same name siblings are supported via elements with the same [Name] but different indices.
*/
-@DfId(Meta.TYPE)
+@DfType(Meta.TYPE)
@Serializable(MetaSerializer::class)
public interface Meta : MetaRepr, MetaProvider {
public val value: Value?
@@ -151,6 +150,8 @@ public interface TypedMeta<out M : TypedMeta<M>> : Meta {
override val items: Map<NameToken, M>
+ public val self: M
+
override fun get(name: Name): M? {
tailrec fun M.find(name: Name): M? = if (name.isEmpty()) {
this
@@ -164,11 +165,6 @@ public interface TypedMeta<out M : TypedMeta<M>> : Meta {
override fun toMeta(): Meta = this
}
-/**
- * Access self as a recursive type instance
- */
-public inline val <M : TypedMeta<M>> TypedMeta<M>.self: M get() = unsafeCast()
-
//public typealias Meta = TypedMeta<*>
public operator fun <M : TypedMeta<M>> TypedMeta<M>?.get(token: NameToken): M? = this?.items?.get(token)
@@ -188,10 +184,12 @@ public operator fun <M : TypedMeta<M>> M?.get(key: String): M? = this?.get(key.p
/**
- * Get a sequence of [Name]-[Value] pairs using top-down traversal of the tree
+ * Get a sequence of [Name]-[Value] pairs using top-down traversal of the tree.
+ * The sequence includes root value with empty name
*/
public fun Meta.valueSequence(): Sequence<Pair<Name, Value>> = sequence {
items.forEach { (key, item) ->
+ value?.let { yield(Name.EMPTY to it) }
item.value?.let { itemValue ->
yield(key.asName() to itemValue)
}
@@ -248,7 +246,7 @@ public inline fun <reified E : Enum<E>> Meta?.enum(): E? = this?.value?.let {
}
}
-public val Meta.stringList: List<String>? get() = value?.list?.map { it.string }
+public val Meta?.stringList: List<String>? get() = this?.value?.list?.map { it.string }
/**
* Create a provider that uses given provider for default values if those are not found in this provider
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MetaConverter.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MetaConverter.kt
new file mode 100644
index 00000000..0f0e8efe
--- /dev/null
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MetaConverter.kt
@@ -0,0 +1,165 @@
+package space.kscience.dataforge.meta
+
+import kotlinx.serialization.KSerializer
+import kotlinx.serialization.json.Json
+import kotlinx.serialization.json.encodeToJsonElement
+import kotlinx.serialization.serializer
+import space.kscience.dataforge.meta.descriptors.MetaDescriptor
+import space.kscience.dataforge.misc.DFExperimental
+
+
+/**
+ * A converter of generic object to and from [Meta]
+ */
+public interface MetaConverter<T> : MetaReader<T> {
+
+ /**
+ * A descriptor for resulting meta
+ */
+ override val descriptor: MetaDescriptor? get() = null
+
+ /**
+ * Attempt conversion of [source] to an object or return null if conversion failed
+ */
+ override fun readOrNull(source: Meta): T?
+
+ override fun read(source: Meta): T =
+ readOrNull(source) ?: error("Meta $source could not be interpreted by $this")
+
+ public fun convert(obj: T): Meta
+
+ public companion object {
+
+ public val meta: MetaConverter<Meta> = object : MetaConverter<Meta> {
+ override fun readOrNull(source: Meta): Meta = source
+ override fun convert(obj: Meta): Meta = obj
+ }
+
+ public val value: MetaConverter<Value> = object : MetaConverter<Value> {
+ override fun readOrNull(source: Meta): Value? = source.value
+ override fun convert(obj: Value): Meta = Meta(obj)
+ }
+
+ public val string: MetaConverter<String> = object : MetaConverter<String> {
+ override val descriptor: MetaDescriptor = MetaDescriptor {
+ valueType(ValueType.STRING)
+ }
+
+
+ override fun readOrNull(source: Meta): String? = source.string
+ override fun convert(obj: String): Meta = Meta(obj.asValue())
+ }
+
+ public val boolean: MetaConverter<Boolean> = object : MetaConverter<Boolean> {
+ override val descriptor: MetaDescriptor = MetaDescriptor {
+ valueType(ValueType.BOOLEAN)
+ }
+
+ override fun readOrNull(source: Meta): Boolean? = source.boolean
+ override fun convert(obj: Boolean): Meta = Meta(obj.asValue())
+ }
+
+ public val number: MetaConverter<Number> = object : MetaConverter<Number> {
+ override val descriptor: MetaDescriptor = MetaDescriptor {
+ valueType(ValueType.NUMBER)
+ }
+
+ override fun readOrNull(source: Meta): Number? = source.number
+ override fun convert(obj: Number): Meta = Meta(obj.asValue())
+ }
+
+ public val double: MetaConverter<Double> = object : MetaConverter<Double> {
+ override val descriptor: MetaDescriptor = MetaDescriptor {
+ valueType(ValueType.NUMBER)
+ }
+
+ override fun readOrNull(source: Meta): Double? = source.double
+ override fun convert(obj: Double): Meta = Meta(obj.asValue())
+ }
+
+ public val float: MetaConverter<Float> = object : MetaConverter<Float> {
+ override val descriptor: MetaDescriptor = MetaDescriptor {
+ valueType(ValueType.NUMBER)
+ }
+
+ override fun readOrNull(source: Meta): Float? = source.float
+ override fun convert(obj: Float): Meta = Meta(obj.asValue())
+ }
+
+ public val int: MetaConverter<Int> = object : MetaConverter<Int> {
+ override val descriptor: MetaDescriptor = MetaDescriptor {
+ valueType(ValueType.NUMBER)
+ }
+
+ override fun readOrNull(source: Meta): Int? = source.int
+ override fun convert(obj: Int): Meta = Meta(obj.asValue())
+ }
+
+ public val long: MetaConverter<Long> = object : MetaConverter<Long> {
+ override val descriptor: MetaDescriptor = MetaDescriptor {
+ valueType(ValueType.NUMBER)
+ }
+
+ override fun readOrNull(source: Meta): Long? = source.long
+ override fun convert(obj: Long): Meta = Meta(obj.asValue())
+ }
+
+ public inline fun <reified E : Enum<E>> enum(): MetaConverter<E> = object : MetaConverter<E> {
+ override val descriptor: MetaDescriptor = MetaDescriptor {
+ valueType(ValueType.STRING)
+ allowedValues(enumValues<E>())
+ }
+
+ @Suppress("USELESS_CAST")
+ override fun readOrNull(source: Meta): E = source.enum<E>() as? E ?: error("The Item is not a Enum")
+
+ override fun convert(obj: E): Meta = Meta(obj.asValue())
+ }
+
+ public val stringList: MetaConverter<List<String>> = object : MetaConverter<List<String>> {
+ override fun convert(obj: List<String>): Meta = Meta(obj.map { it.asValue() }.asValue())
+
+ override fun readOrNull(source: Meta): List<String>? = source.stringList
+ }
+
+ public fun <T> valueList(
+ writer: (T) -> Value = { Value.of(it) },
+ reader: (Value) -> T,
+ ): MetaConverter<List<T>> = object : MetaConverter<List<T>> {
+ override val descriptor: MetaDescriptor = MetaDescriptor {
+ valueType(ValueType.LIST)
+ }
+
+ override fun readOrNull(source: Meta): List<T>? = source.value?.list?.map(reader)
+
+ override fun convert(obj: List<T>): Meta = Meta(obj.map(writer).asValue())
+ }
+
+ /**
+ * Automatically generate [MetaConverter] for a class using its serializer and optional [descriptor]
+ */
+ @DFExperimental
+ public inline fun <reified T> serializable(
+ descriptor: MetaDescriptor? = null,
+ jsonEncoder: Json = Json,
+ ): MetaConverter<T> = object : MetaConverter<T> {
+ private val serializer: KSerializer<T> = serializer()
+
+ override fun readOrNull(source: Meta): T? {
+ val json = source.toJson(descriptor)
+ return jsonEncoder.decodeFromJsonElement(serializer, json)
+ }
+
+ override fun convert(obj: T): Meta {
+ val json = jsonEncoder.encodeToJsonElement(obj)
+ return json.toMeta(descriptor)
+ }
+
+ }
+
+ }
+}
+
+public fun <T : Any> MetaConverter<T>.convertNullable(obj: T?): Meta? = obj?.let { convert(it) }
+
+
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MetaDelegate.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MetaDelegate.kt
index 73923d56..26044f18 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MetaDelegate.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MetaDelegate.kt
@@ -1,84 +1,190 @@
package space.kscience.dataforge.meta
-import space.kscience.dataforge.meta.transformations.MetaConverter
+import space.kscience.dataforge.meta.descriptors.Described
+import space.kscience.dataforge.meta.descriptors.MetaDescriptor
+import space.kscience.dataforge.misc.DFExperimental
import space.kscience.dataforge.names.Name
import space.kscience.dataforge.names.asName
import kotlin.properties.ReadOnlyProperty
+import kotlin.reflect.KProperty
/* Meta delegates */
-public fun MetaProvider.node(key: Name? = null): ReadOnlyProperty<Any?, Meta?> = ReadOnlyProperty { _, property ->
- get(key ?: property.name.asName())
+public interface MetaDelegate<T> : ReadOnlyProperty<Any?, T>, Described
+
+
+public fun MetaProvider.node(
+ key: Name? = null,
+ descriptor: MetaDescriptor? = null,
+): MetaDelegate<Meta?> = object : MetaDelegate<Meta?> {
+ override val descriptor: MetaDescriptor? = descriptor
+
+ override fun getValue(thisRef: Any?, property: KProperty<*>): Meta? {
+ return get(key ?: property.name.asName())
+ }
}
+
+/**
+ * Use [reader] to read the Meta node
+ */
+public fun <T> MetaProvider.readable(
+ reader: MetaReader<T>,
+ key: Name? = null,
+): MetaDelegate<T?> = object : MetaDelegate<T?> {
+ override val descriptor: MetaDescriptor? get() = reader.descriptor
+
+ override fun getValue(thisRef: Any?, property: KProperty<*>): T? {
+ return get(key ?: property.name.asName())?.let { reader.read(it) }
+ }
+}
+
+/**
+ * Use [reader] to read the Meta node or return [default] if node does not exist
+ */
+public fun <T> MetaProvider.readable(
+ reader: MetaReader<T>,
+ default: T,
+ key: Name? = null,
+): MetaDelegate<T> = object : MetaDelegate<T> {
+ override val descriptor: MetaDescriptor? get() = reader.descriptor
+
+ override fun getValue(thisRef: Any?, property: KProperty<*>): T {
+ return get(key ?: property.name.asName())?.let { reader.read(it) } ?: default
+ }
+}
+
+/**
+ * Use [reader] to read the Meta node
+ */
+@Deprecated("Replace with readable", ReplaceWith("readable(metaReader, key)"))
+public fun <T> MetaProvider.spec(
+ reader: MetaReader<T>,
+ key: Name? = null,
+): MetaDelegate<T?> = readable(reader, key)
+
+/**
+ * Use object serializer to transform it to Meta and back
+ */
+@DFExperimental
+public inline fun <reified T> MetaProvider.serializable(
+ key: Name? = null,
+ descriptor: MetaDescriptor? = null,
+): MetaDelegate<T?> = readable(MetaConverter.serializable(descriptor), key)
+
+@DFExperimental
+public inline fun <reified T> MetaProvider.serializable(
+ key: Name? = null,
+ default: T,
+ descriptor: MetaDescriptor? = null,
+): MetaDelegate<T> = readable(MetaConverter.serializable(descriptor), default, key)
+
+@Deprecated("Use convertable", ReplaceWith("convertable(converter, key)"))
public fun <T> MetaProvider.node(
key: Name? = null,
- converter: MetaConverter<T>
-): ReadOnlyProperty<Any?, T?> = ReadOnlyProperty { _, property ->
- get(key ?: property.name.asName())?.let { converter.metaToObject(it) }
+ converter: MetaReader<T>,
+): ReadOnlyProperty<Any?, T?> = readable(converter, key)
+
+/**
+ * Use [reader] to convert a list of same name siblings meta to object
+ */
+public fun <T> Meta.listOfReadable(
+ reader: MetaReader<T>,
+ key: Name? = null,
+): MetaDelegate<List<T>> = object : MetaDelegate<List<T>> {
+ override fun getValue(thisRef: Any?, property: KProperty<*>): List<T> {
+ val name = key ?: property.name.asName()
+ return getIndexed(name).values.map { reader.read(it) }
+ }
+
+ override val descriptor: MetaDescriptor? = reader.descriptor?.copy(multiple = true)
}
+
+/**
+ * Use [converter] to convert a list of same name siblings meta to object
+ */
+@Deprecated("Replace with readingList", ReplaceWith("readingList(converter, key)"))
+public fun <T> Meta.listOfSpec(
+ converter: MetaReader<T>,
+ key: Name? = null,
+): MetaDelegate<List<T>> = listOfReadable(converter, key)
+
+@DFExperimental
+public inline fun <reified T> Meta.listOfSerializable(
+ key: Name? = null,
+ descriptor: MetaDescriptor? = null,
+): MetaDelegate<List<T>> = listOfReadable(MetaConverter.serializable(descriptor), key)
+
/**
* A property delegate that uses custom key
*/
-public fun MetaProvider.value(key: Name? = null): ReadOnlyProperty<Any?, Value?> = ReadOnlyProperty { _, property ->
- get(key ?: property.name.asName())?.value
+public fun MetaProvider.value(
+ key: Name? = null,
+ descriptor: MetaDescriptor? = null,
+): MetaDelegate<Value?> = object : MetaDelegate<Value?> {
+ override fun getValue(thisRef: Any?, property: KProperty<*>): Value? = get(key ?: property.name.asName())?.value
+
+ override val descriptor: MetaDescriptor? = descriptor
}
public fun <R> MetaProvider.value(
key: Name? = null,
- reader: (Value?) -> R
-): ReadOnlyProperty<Any?, R> = ReadOnlyProperty { _, property ->
- reader(get(key ?: property.name.asName())?.value)
+ descriptor: MetaDescriptor? = null,
+ reader: (Value?) -> R,
+): MetaDelegate<R> = object : MetaDelegate<R> {
+ override fun getValue(thisRef: Any?, property: KProperty<*>): R = reader(get(key ?: property.name.asName())?.value)
+
+ override val descriptor: MetaDescriptor? = descriptor
}
//TODO add caching for sealed nodes
/* Read-only delegates for [Meta] */
-public fun MetaProvider.string(key: Name? = null): ReadOnlyProperty<Any?, String?> = value(key) { it?.string }
+public fun MetaProvider.string(key: Name? = null): MetaDelegate<String?> = value(key = key) { it?.string }
-public fun MetaProvider.boolean(key: Name? = null): ReadOnlyProperty<Any?, Boolean?> = value(key) { it?.boolean }
+public fun MetaProvider.boolean(key: Name? = null): MetaDelegate<Boolean?> = value(key = key) { it?.boolean }
-public fun MetaProvider.number(key: Name? = null): ReadOnlyProperty<Any?, Number?> = value(key) { it?.numberOrNull }
+public fun MetaProvider.number(key: Name? = null): MetaDelegate<Number?> = value(key = key) { it?.numberOrNull }
-public fun MetaProvider.double(key: Name? = null): ReadOnlyProperty<Any?, Double?> = value(key) { it?.double }
+public fun MetaProvider.double(key: Name? = null): MetaDelegate<Double?> = value(key = key) { it?.double }
-public fun MetaProvider.float(key: Name? = null): ReadOnlyProperty<Any?, Float?> = value(key) { it?.float }
+public fun MetaProvider.float(key: Name? = null): MetaDelegate<Float?> = value(key = key) { it?.float }
-public fun MetaProvider.int(key: Name? = null): ReadOnlyProperty<Any?, Int?> = value(key) { it?.int }
+public fun MetaProvider.int(key: Name? = null): MetaDelegate<Int?> = value(key = key) { it?.int }
-public fun MetaProvider.long(key: Name? = null): ReadOnlyProperty<Any?, Long?> = value(key) { it?.long }
+public fun MetaProvider.long(key: Name? = null): MetaDelegate<Long?> = value(key = key) { it?.long }
-public fun MetaProvider.string(default: String, key: Name? = null): ReadOnlyProperty<Any?, String> =
- value(key) { it?.string ?: default }
+public fun MetaProvider.string(default: String, key: Name? = null): MetaDelegate<String> =
+ value(key = key) { it?.string ?: default }
-public fun MetaProvider.boolean(default: Boolean, key: Name? = null): ReadOnlyProperty<Any?, Boolean> =
- value(key) { it?.boolean ?: default }
+public fun MetaProvider.boolean(default: Boolean, key: Name? = null): MetaDelegate<Boolean> =
+ value(key = key) { it?.boolean ?: default }
-public fun MetaProvider.number(default: Number, key: Name? = null): ReadOnlyProperty<Any?, Number> =
- value(key) { it?.numberOrNull ?: default }
+public fun MetaProvider.number(default: Number, key: Name? = null): MetaDelegate<Number> =
+ value(key = key) { it?.numberOrNull ?: default }
-public fun MetaProvider.double(default: Double, key: Name? = null): ReadOnlyProperty<Any?, Double> =
- value(key) { it?.double ?: default }
+public fun MetaProvider.double(default: Double, key: Name? = null): MetaDelegate<Double> =
+ value(key = key) { it?.double ?: default }
-public fun MetaProvider.float(default: Float, key: Name? = null): ReadOnlyProperty<Any?, Float> =
- value(key) { it?.float ?: default }
+public fun MetaProvider.float(default: Float, key: Name? = null): MetaDelegate<Float> =
+ value(key = key) { it?.float ?: default }
-public fun MetaProvider.int(default: Int, key: Name? = null): ReadOnlyProperty<Any?, Int> =
- value(key) { it?.int ?: default }
+public fun MetaProvider.int(default: Int, key: Name? = null): MetaDelegate<Int> =
+ value(key = key) { it?.int ?: default }
-public fun MetaProvider.long(default: Long, key: Name? = null): ReadOnlyProperty<Any?, Long> =
- value(key) { it?.long ?: default }
+public fun MetaProvider.long(default: Long, key: Name? = null): MetaDelegate<Long> =
+ value(key = key) { it?.long ?: default }
-public inline fun <reified E : Enum<E>> MetaProvider.enum(default: E, key: Name? = null): ReadOnlyProperty<Any?, E> =
- value<E>(key) { it?.enum<E>() ?: default }
+public inline fun <reified E : Enum<E>> MetaProvider.enum(default: E, key: Name? = null): MetaDelegate<E> =
+ value<E>(key = key) { it?.enum<E>() ?: default }
-public fun MetaProvider.string(key: Name? = null, default: () -> String): ReadOnlyProperty<Any?, String> =
- value(key) { it?.string ?: default() }
+public fun MetaProvider.string(key: Name? = null, default: () -> String): MetaDelegate<String> =
+ value(key = key) { it?.string ?: default() }
-public fun MetaProvider.boolean(key: Name? = null, default: () -> Boolean): ReadOnlyProperty<Any?, Boolean> =
- value(key) { it?.boolean ?: default() }
+public fun MetaProvider.boolean(key: Name? = null, default: () -> Boolean): MetaDelegate<Boolean> =
+ value(key = key) { it?.boolean ?: default() }
-public fun MetaProvider.number(key: Name? = null, default: () -> Number): ReadOnlyProperty<Any?, Number> =
- value(key) { it?.numberOrNull ?: default() }
+public fun MetaProvider.number(key: Name? = null, default: () -> Number): MetaDelegate<Number> =
+ value(key = key) { it?.numberOrNull ?: default() }
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MetaReader.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MetaReader.kt
new file mode 100644
index 00000000..a8514d63
--- /dev/null
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MetaReader.kt
@@ -0,0 +1,21 @@
+package space.kscience.dataforge.meta
+
+import space.kscience.dataforge.meta.descriptors.Described
+
+public interface MetaReader<out T> : Described {
+
+ /**
+ * Read the source meta into an object and return null if Meta could not be interpreted as a target type
+ */
+ public fun readOrNull(source: Meta): T?
+
+ /**
+ * Read generic read-only meta with this [MetaReader] producing instance of the desired type.
+ * Throws an error if conversion could not be done.
+ */
+ public fun read(source: Meta): T = readOrNull(source) ?: error("Meta $source could not be interpreted by $this")
+}
+
+
+public fun <T : Any> MetaReader<T>.readNullable(item: Meta?): T? = item?.let { read(it) }
+public fun <T> MetaReader<T>.readValue(value: Value): T? = read(Meta(value))
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MetaRef.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MetaRef.kt
new file mode 100644
index 00000000..2e6f3452
--- /dev/null
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MetaRef.kt
@@ -0,0 +1,221 @@
+package space.kscience.dataforge.meta
+
+import kotlinx.serialization.json.Json
+import space.kscience.dataforge.meta.descriptors.Described
+import space.kscience.dataforge.meta.descriptors.MetaDescriptor
+import space.kscience.dataforge.meta.descriptors.MetaDescriptorBuilder
+import space.kscience.dataforge.misc.DFExperimental
+import space.kscience.dataforge.names.Name
+import space.kscience.dataforge.names.asName
+import space.kscience.dataforge.names.startsWith
+import kotlin.properties.PropertyDelegateProvider
+import kotlin.properties.ReadOnlyProperty
+
+
+/**
+ * A reference to a read-only value of type [T] inside [MetaProvider] or writable value in [MutableMetaProvider]
+ */
+@DFExperimental
+public data class MetaRef<T>(
+ public val name: Name,
+ public val converter: MetaConverter<T>,
+ override val descriptor: MetaDescriptor? = converter.descriptor,
+) : Described
+
+/**
+ * Get a value from provider by [ref] or return null if node with given name is missing
+ */
+@DFExperimental
+public operator fun <T> MetaProvider.get(ref: MetaRef<T>): T? = get(ref.name)?.let { ref.converter.readOrNull(it) }
+
+/**
+ * Set a value in a mutable provider by [ref]
+ */
+@DFExperimental
+public operator fun <T> MutableMetaProvider.set(ref: MetaRef<T>, value: T) {
+ set(ref.name, ref.converter.convert(value))
+}
+
+/**
+ * Observe changes to specific property via given [ref].
+ *
+ * This listener should be removed in a same way as [ObservableMeta.onChange].
+ *
+ * @param callback an action to be performed on each change of item. Null means that the item is not present or malformed.
+ */
+@DFExperimental
+public fun <T: Any> ObservableMeta.onValueChange(owner: Any?, ref: MetaRef<T>, callback: (T?) -> Unit) {
+ onChange(owner) { name ->
+ if (name.startsWith(ref.name)) {
+ get(name)?.let { value ->
+ callback(ref.converter.readOrNull(value))
+ }
+ }
+ }
+}
+
+/**
+ * Remove a node corresponding to [ref] from a mutable provider if it exists
+ */
+@DFExperimental
+public fun MutableMetaProvider.remove(ref: MetaRef<*>) {
+ remove(ref.name)
+}
+
+/**
+ * Base storage of [MetaRef]
+ */
+@OptIn(DFExperimental::class)
+public interface MetaRefStore : Described {
+ public val refs: List<MetaRef<*>>
+}
+
+/**
+ * A base class for [Meta] specification that stores references to meta nodes.
+ */
+@DFExperimental
+public abstract class MetaSpec : MetaRefStore {
+ private val _refs: MutableList<MetaRef<*>> = mutableListOf()
+ override val refs: List<MetaRef<*>> get() = _refs
+
+ /**
+ * Register a ref in this specification
+ */
+ protected fun registerRef(ref: MetaRef<*>) {
+ _refs.add(ref)
+ }
+
+ /**
+ * Create and register a ref by property name and provided converter.
+ * By default, uses descriptor from the converter
+ */
+ public fun <T> item(
+ converter: MetaConverter<T>,
+ key: Name? = null,
+ descriptor: MetaDescriptor? = converter.descriptor,
+ ): PropertyDelegateProvider<MetaSpec, ReadOnlyProperty<MetaSpec, MetaRef<T>>> =
+ PropertyDelegateProvider { _, property ->
+ val ref = MetaRef(key ?: property.name.asName(), converter, descriptor)
+ registerRef(ref)
+ ReadOnlyProperty { _, _ ->
+ ref
+ }
+ }
+
+ /**
+ * Override to provide custom [MetaDescriptor]
+ */
+ protected open fun MetaDescriptorBuilder.buildDescriptor(): Unit = Unit
+
+ override val descriptor: MetaDescriptor by lazy {
+ MetaDescriptor {
+ refs.forEach { ref ->
+ ref.descriptor?.let {
+ node(ref.name, ref.descriptor)
+ }
+ }
+ buildDescriptor()
+ }
+ }
+}
+
+/**
+ * Register an item using a [descriptorBuilder] to customize descriptor
+ */
+@DFExperimental
+public fun <T> MetaSpec.item(
+ converter: MetaConverter<T>,
+ key: Name? = null,
+ descriptorBuilder: MetaDescriptorBuilder.() -> Unit = {},
+): PropertyDelegateProvider<MetaSpec, ReadOnlyProperty<MetaSpec, MetaRef<T>>> = item(converter, key, MetaDescriptor {
+ converter.descriptor?.let { from(it) }
+ descriptorBuilder()
+})
+
+//utility methods to add different nodes
+
+@DFExperimental
+public fun MetaSpec.metaItem(
+ key: Name? = null,
+ descriptorBuilder: MetaDescriptorBuilder.() -> Unit = {},
+): PropertyDelegateProvider<MetaSpec, ReadOnlyProperty<MetaSpec, MetaRef<Meta>>> =
+ item(MetaConverter.meta, key, descriptorBuilder)
+
+@DFExperimental
+public fun MetaSpec.string(
+ key: Name? = null,
+ descriptorBuilder: MetaDescriptorBuilder.() -> Unit = {},
+): PropertyDelegateProvider<MetaSpec, ReadOnlyProperty<MetaSpec, MetaRef<String>>> =
+ item(MetaConverter.string, key, descriptorBuilder)
+
+@DFExperimental
+public fun MetaSpec.boolean(
+ key: Name? = null,
+ descriptorBuilder: MetaDescriptorBuilder.() -> Unit = {},
+): PropertyDelegateProvider<MetaSpec, ReadOnlyProperty<MetaSpec, MetaRef<Boolean>>> =
+ item(MetaConverter.boolean, key, descriptorBuilder)
+
+@DFExperimental
+public fun MetaSpec.stringList(
+ key: Name? = null,
+ descriptorBuilder: MetaDescriptorBuilder.() -> Unit = {},
+): PropertyDelegateProvider<MetaSpec, ReadOnlyProperty<MetaSpec, MetaRef<List<String>>>> =
+ item(MetaConverter.stringList, key, descriptorBuilder)
+
+@DFExperimental
+public fun MetaSpec.float(
+ key: Name? = null,
+ descriptorBuilder: MetaDescriptorBuilder.() -> Unit = {},
+): PropertyDelegateProvider<MetaSpec, ReadOnlyProperty<MetaSpec, MetaRef<Float>>> =
+ item(MetaConverter.float, key, descriptorBuilder)
+
+@DFExperimental
+public fun MetaSpec.double(
+ key: Name? = null,
+ descriptorBuilder: MetaDescriptorBuilder.() -> Unit = {},
+): PropertyDelegateProvider<MetaSpec, ReadOnlyProperty<MetaSpec, MetaRef<Double>>> =
+ item(MetaConverter.double, key, descriptorBuilder)
+
+@DFExperimental
+public fun MetaSpec.int(
+ key: Name? = null,
+ descriptorBuilder: MetaDescriptorBuilder.() -> Unit = {},
+): PropertyDelegateProvider<MetaSpec, ReadOnlyProperty<MetaSpec, MetaRef<Int>>> =
+ item(MetaConverter.int, key, descriptorBuilder)
+
+@DFExperimental
+public fun MetaSpec.long(
+ key: Name? = null,
+ descriptorBuilder: MetaDescriptorBuilder.() -> Unit = {},
+): PropertyDelegateProvider<MetaSpec, ReadOnlyProperty<MetaSpec, MetaRef<Long>>> =
+ item(MetaConverter.long, key, descriptorBuilder)
+
+
+@DFExperimental
+public fun MetaSpec.doubleArray(
+ key: Name? = null,
+ descriptorBuilder: MetaDescriptorBuilder.() -> Unit = {},
+): PropertyDelegateProvider<MetaSpec, ReadOnlyProperty<MetaSpec, MetaRef<DoubleArray>>> =
+ item(MetaConverter.doubleArray, key, descriptorBuilder)
+
+@DFExperimental
+public fun MetaSpec.byteArray(
+ key: Name? = null,
+ descriptorBuilder: MetaDescriptorBuilder.() -> Unit = {},
+): PropertyDelegateProvider<MetaSpec, ReadOnlyProperty<MetaSpec, MetaRef<ByteArray>>> =
+ item(MetaConverter.byteArray, key, descriptorBuilder)
+
+@DFExperimental
+public inline fun <reified E : Enum<E>> MetaSpec.enum(
+ key: Name? = null,
+ noinline descriptorBuilder: MetaDescriptorBuilder.() -> Unit = {},
+): PropertyDelegateProvider<MetaSpec, ReadOnlyProperty<MetaSpec, MetaRef<E>>> =
+ item(MetaConverter.enum(), key, descriptorBuilder)
+
+@DFExperimental
+public inline fun <reified T> MetaSpec.serializable(
+ key: Name? = null,
+ jsonEncoder: Json = Json,
+ noinline descriptorBuilder: MetaDescriptorBuilder.() -> Unit = {},
+): PropertyDelegateProvider<MetaSpec, ReadOnlyProperty<MetaSpec, MetaRef<T>>> =
+ item(MetaConverter.serializable(jsonEncoder = jsonEncoder), key, descriptorBuilder)
\ No newline at end of file
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MetaSerializer.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MetaSerializer.kt
index 557cd5ef..2ccfd520 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MetaSerializer.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MetaSerializer.kt
@@ -45,4 +45,21 @@ public object MutableMetaSerializer : KSerializer<MutableMeta> {
override fun serialize(encoder: Encoder, value: MutableMeta) {
encoder.encodeSerializableValue(MetaSerializer, value)
}
+}
+
+/**
+ * A serializer for [ObservableMutableMeta]
+ */
+public object ObservableMutableMetaSerializer : KSerializer<ObservableMutableMeta> {
+
+ override val descriptor: SerialDescriptor = MetaSerializer.descriptor
+
+ override fun deserialize(decoder: Decoder): ObservableMutableMeta {
+ val meta = decoder.decodeSerializableValue(MetaSerializer)
+ return ((meta as? MutableMeta) ?: meta.toMutableMeta()).asObservable()
+ }
+
+ override fun serialize(encoder: Encoder, value: ObservableMutableMeta) {
+ encoder.encodeSerializableValue(MetaSerializer, value)
+ }
}
\ No newline at end of file
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/transformations/MetaTransformation.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MetaTransformation.kt
similarity index 98%
rename from dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/transformations/MetaTransformation.kt
rename to dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MetaTransformation.kt
index d41365a6..dbf94f0f 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/transformations/MetaTransformation.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MetaTransformation.kt
@@ -1,6 +1,5 @@
-package space.kscience.dataforge.meta.transformations
+package space.kscience.dataforge.meta
-import space.kscience.dataforge.meta.*
import space.kscience.dataforge.misc.DFExperimental
import space.kscience.dataforge.names.Name
import kotlin.jvm.JvmInline
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MutableMeta.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MutableMeta.kt
index 7e05d215..274671bf 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MutableMeta.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MutableMeta.kt
@@ -19,6 +19,10 @@ public annotation class MetaBuilderMarker
public interface MutableMetaProvider : MetaProvider, MutableValueProvider {
override fun get(name: Name): MutableMeta?
public operator fun set(name: Name, node: Meta?)
+
+ /**
+ * Set value with the given name. Does nothing if value is not changed.
+ */
override fun setValue(name: Name, value: Value?)
}
@@ -48,11 +52,13 @@ public interface MutableMeta : Meta, MutableMetaProvider {
}
override fun setValue(name: Name, value: Value?) {
- getOrCreate(name).value = value
+ if (value != getValue(name)) {
+ getOrCreate(name).value = value
+ }
}
/**
- * Get existing node or create a new one
+ * Get an existing node or create a new one
*/
public fun getOrCreate(name: Name): MutableMeta
@@ -122,6 +128,10 @@ public interface MutableMeta : Meta, MutableMetaProvider {
setValue(Name.parse(this), array.asValue())
}
+ public infix fun String.put(array: ByteArray) {
+ setValue(Name.parse(this), array.asValue())
+ }
+
public infix fun String.put(repr: MetaRepr) {
set(Name.parse(this), repr.toMeta())
}
@@ -149,7 +159,17 @@ public interface MutableTypedMeta<M : MutableTypedMeta<M>> : TypedMeta<M>, Mutab
*/
@DFExperimental
public fun attach(name: Name, node: M)
- override fun get(name: Name): M?
+
+ override fun get(name: Name): M? {
+ tailrec fun M.find(name: Name): M? = if (name.isEmpty()) {
+ self
+ } else {
+ items[name.firstOrNull()!!]?.find(name.cutFirst())
+ }
+
+ return self.find(name)
+ }
+
override fun getOrCreate(name: Name): M
}
@@ -165,7 +185,7 @@ public fun MutableMetaProvider.remove(key: String) {
// node setters
-public operator fun MutableMetaProvider.set(Key: NameToken, value: Meta): Unit = set(Key.asName(), value)
+public operator fun MutableMetaProvider.set(key: NameToken, value: Meta): Unit = set(key.asName(), value)
public operator fun MutableMetaProvider.set(key: String, value: Meta): Unit = set(Name.parse(key), value)
@@ -198,10 +218,8 @@ public operator fun MutableMetaProvider.set(key: String, metas: Iterable<Meta>):
/**
- * Update existing mutable node with another node. The rules are following:
- * * value replaces anything
- * * node updates node and replaces anything but node
- * * node list updates node list if number of nodes in the list is the same and replaces anything otherwise
+ * Update the existing mutable node with another node.
+ * Values that are present in the current provider and are missing in [meta] are kept.
*/
public fun MutableMetaProvider.update(meta: Meta) {
meta.valueSequence().forEach { (name, value) ->
@@ -222,7 +240,7 @@ public fun <M : MutableTypedMeta<M>> MutableTypedMeta<M>.edit(name: Name, builde
getOrCreate(name).apply(builder)
/**
- * Set a value at a given [name]. If node does not exist, create it.
+ * Set a value at a given [name]. If a node does not exist, create it.
*/
public operator fun <M : MutableTypedMeta<M>> MutableTypedMeta<M>.set(name: Name, value: Value?) {
edit(name) {
@@ -245,6 +263,9 @@ private class MutableMetaImpl(
value: Value?,
children: Map<NameToken, Meta> = emptyMap(),
) : AbstractObservableMeta(), ObservableMutableMeta {
+
+ override val self get() = this
+
override var value = value
@ThreadSafe set(value) {
val oldValue = field
@@ -324,8 +345,6 @@ private class MutableMetaImpl(
//remove child and invalidate if argument is null
if (node == null) {
children.remove(token)?.removeListener(this)
- // old item is not null otherwise we can't be here
- invalidate(name)
} else {
val newNode = wrapItem(node)
newNode.adoptBy(this, token)
@@ -335,7 +354,7 @@ private class MutableMetaImpl(
else -> {
val token = name.firstOrNull()!!
- //get existing or create new node.
+ //get an existing node or create a new node.
if (items[token] == null) {
val newNode = MutableMetaImpl(null)
newNode.adoptBy(this, token)
@@ -369,16 +388,37 @@ public fun MutableMeta.append(name: Name, value: Value): Unit = append(name, Met
public fun MutableMeta.append(key: String, value: Value): Unit = append(Name.parse(key), value)
+/**
+ * Update all items that exist in the [newMeta] and remove existing items that are missing in [newMeta].
+ * This produces the same result as clearing all items and updating blank meta with a [newMeta], but does not
+ * produce unnecessary invalidation events (if they are supported).
+ */
+public fun MutableMeta.reset(newMeta: Meta) {
+ //remove old items
+ (items.keys - newMeta.items.keys).forEach {
+ remove(it.asName())
+ }
+ newMeta.items.forEach { (token, item) ->
+ set(token, item)
+ }
+}
+
/**
* Create a mutable copy of this meta. The copy is created even if the Meta is already mutable
*/
-public fun Meta.toMutableMeta(): ObservableMutableMeta = MutableMetaImpl(value, items)
+public fun Meta.toMutableMeta(): MutableMeta =
+ MutableMeta { update(this@toMutableMeta) } //MutableMetaImpl(value, items)
public fun Meta.asMutableMeta(): MutableMeta = (this as? MutableMeta) ?: toMutableMeta()
@JsName("newObservableMutableMeta")
public fun ObservableMutableMeta(): ObservableMutableMeta = MutableMetaImpl(null)
+/**
+ * Create a pre-filled [ObservableMutableMeta]
+ */
+public fun ObservableMutableMeta(content: Meta): ObservableMutableMeta = ObservableMutableMeta { update(content) }
+
/**
* Build a [MutableMeta] using given transformation
*/
@@ -387,12 +427,14 @@ public inline fun ObservableMutableMeta(builder: MutableMeta.() -> Unit = {}): O
/**
- * Create a copy of this [Meta], optionally applying the given [block].
- * The listeners of the original Config are not retained.
+ * Create a read-only copy of this [Meta]. [modification] is an optional modification applied to [Meta] on copy.
+ *
+ * The copy does not reflect changes of the initial Meta.
*/
-public inline fun Meta.copy(block: MutableMeta.() -> Unit = {}): Meta =
- toMutableMeta().apply(block)
-
+public inline fun Meta.copy(modification: MutableMeta.() -> Unit = {}): Meta = Meta {
+ update(this@copy)
+ modification()
+}
private class MutableMetaWithDefault(
val source: MutableMeta, val default: MetaProvider, val rootName: Name,
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MutableMetaDelegate.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MutableMetaDelegate.kt
index 0f28523c..28a550d8 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MutableMetaDelegate.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MutableMetaDelegate.kt
@@ -1,52 +1,146 @@
package space.kscience.dataforge.meta
-import space.kscience.dataforge.meta.transformations.MetaConverter
+import space.kscience.dataforge.meta.descriptors.Described
+import space.kscience.dataforge.meta.descriptors.MetaDescriptor
+import space.kscience.dataforge.misc.DFExperimental
import space.kscience.dataforge.names.Name
import space.kscience.dataforge.names.asName
+import space.kscience.dataforge.names.getIndexedList
import kotlin.properties.ReadWriteProperty
import kotlin.reflect.KProperty
+
/* Read-write delegates */
-public fun MutableMetaProvider.node(key: Name? = null): ReadWriteProperty<Any?, Meta?> =
- object : ReadWriteProperty<Any?, Meta?> {
- override fun getValue(thisRef: Any?, property: KProperty<*>): Meta? {
- return get(key ?: property.name.asName())
- }
+public interface MutableMetaDelegate<T> : ReadWriteProperty<Any?, T>, Described
- override fun setValue(thisRef: Any?, property: KProperty<*>, value: Meta?) {
- val name = key ?: property.name.asName()
- set(name, value)
- }
+public fun MutableMetaProvider.node(
+ key: Name? = null,
+ descriptor: MetaDescriptor? = null,
+): MutableMetaDelegate<Meta?> = object : MutableMetaDelegate<Meta?> {
+
+ override val descriptor: MetaDescriptor? = descriptor
+
+ override fun getValue(thisRef: Any?, property: KProperty<*>): Meta? {
+ return get(key ?: property.name.asName())
}
-public fun <T> MutableMetaProvider.node(key: Name? = null, converter: MetaConverter<T>): ReadWriteProperty<Any?, T?> =
- object : ReadWriteProperty<Any?, T?> {
- override fun getValue(thisRef: Any?, property: KProperty<*>): T? {
- return get(key ?: property.name.asName())?.let { converter.metaToObject(it) }
- }
+ override fun setValue(thisRef: Any?, property: KProperty<*>, value: Meta?) {
+ val name = key ?: property.name.asName()
+ set(name, value)
+ }
+}
- override fun setValue(thisRef: Any?, property: KProperty<*>, value: T?) {
- val name = key ?: property.name.asName()
- set(name, value?.let { converter.objectToMeta(it) })
- }
+/**
+ * Use [converter] to transform an object to Meta and back.
+ * Note that mutation of the object does not change Meta.
+ */
+public fun <T> MutableMetaProvider.convertable(
+ converter: MetaConverter<T>,
+ key: Name? = null,
+): MutableMetaDelegate<T?> = object : MutableMetaDelegate<T?> {
+
+ override val descriptor: MetaDescriptor? get() = converter.descriptor
+
+
+ override fun getValue(thisRef: Any?, property: KProperty<*>): T? {
+ val name = key ?: property.name.asName()
+ return get(name)?.let { converter.read(it) }
}
-public fun MutableMetaProvider.value(key: Name? = null): ReadWriteProperty<Any?, Value?> =
- object : ReadWriteProperty<Any?, Value?> {
- override fun getValue(thisRef: Any?, property: KProperty<*>): Value? =
- get(key ?: property.name.asName())?.value
-
- override fun setValue(thisRef: Any?, property: KProperty<*>, value: Value?) {
- setValue(key ?: property.name.asName(), value)
- }
+ override fun setValue(thisRef: Any?, property: KProperty<*>, value: T?) {
+ val name = key ?: property.name.asName()
+ set(name, value?.let { converter.convert(it) })
}
+}
+
+public fun <T> MutableMetaProvider.convertable(
+ converter: MetaConverter<T>,
+ default: T,
+ key: Name? = null,
+): MutableMetaDelegate<T> = object : MutableMetaDelegate<T> {
+
+ override val descriptor: MetaDescriptor? get() = converter.descriptor
+
+
+ override fun getValue(thisRef: Any?, property: KProperty<*>): T {
+ val name = key ?: property.name.asName()
+ return get(name)?.let { converter.read(it) } ?: default
+ }
+
+ override fun setValue(thisRef: Any?, property: KProperty<*>, value: T) {
+ val name = key ?: property.name.asName()
+ set(name, value?.let { converter.convert(it) })
+ }
+}
+
+/**
+ * Use object serializer to transform it to Meta and back.
+ * Note that mutation of the object does not change Meta.
+ */
+@DFExperimental
+public inline fun <reified T> MutableMetaProvider.serializable(
+ descriptor: MetaDescriptor? = null,
+ key: Name? = null,
+): MutableMetaDelegate<T?> = convertable<T>(MetaConverter.serializable(descriptor), key)
+
+@DFExperimental
+public inline fun <reified T> MutableMetaProvider.serializable(
+ descriptor: MetaDescriptor? = null,
+ default: T,
+ key: Name? = null,
+): MutableMetaDelegate<T> = convertable(MetaConverter.serializable(descriptor), default, key)
+
+/**
+ * Use [converter] to convert a list of same name siblings meta to object and back.
+ * Note that mutation of the object does not change Meta.
+ */
+public fun <T> MutableMeta.listOfConvertable(
+ converter: MetaConverter<T>,
+ key: Name? = null,
+): MutableMetaDelegate<List<T>> = object : MutableMetaDelegate<List<T>> {
+ override val descriptor: MetaDescriptor? = converter.descriptor?.copy(multiple = true)
+
+ override fun getValue(thisRef: Any?, property: KProperty<*>): List<T> {
+ val name = key ?: property.name.asName()
+ return getIndexedList(name).map { converter.read(it) }
+ }
+
+ override fun setValue(thisRef: Any?, property: KProperty<*>, value: List<T>) {
+ val name = key ?: property.name.asName()
+ setIndexed(name, value.map { converter.convert(it) })
+ }
+}
+
+@DFExperimental
+public inline fun <reified T> MutableMeta.listOfSerializable(
+ key: Name? = null,
+ descriptor: MetaDescriptor? = null,
+): MutableMetaDelegate<List<T>> = listOfConvertable(MetaConverter.serializable(descriptor), key)
+
+
+public fun MutableMetaProvider.value(
+ key: Name? = null,
+ descriptor: MetaDescriptor? = null,
+): MutableMetaDelegate<Value?> = object : MutableMetaDelegate<Value?> {
+ override val descriptor: MetaDescriptor? = descriptor
+
+ override fun getValue(thisRef: Any?, property: KProperty<*>): Value? =
+ get(key ?: property.name.asName())?.value
+
+ override fun setValue(thisRef: Any?, property: KProperty<*>, value: Value?) {
+ setValue(key ?: property.name.asName(), value)
+ }
+}
public fun <T> MutableMetaProvider.value(
key: Name? = null,
writer: (T) -> Value? = { Value.of(it) },
- reader: (Value?) -> T
-): ReadWriteProperty<Any?, T> = object : ReadWriteProperty<Any?, T> {
+ descriptor: MetaDescriptor? = null,
+ reader: (Value?) -> T,
+): MutableMetaDelegate<T> = object : MutableMetaDelegate<T> {
+ override val descriptor: MetaDescriptor? = descriptor
+
override fun getValue(thisRef: Any?, property: KProperty<*>): T =
reader(get(key ?: property.name.asName())?.value)
@@ -57,65 +151,65 @@ public fun <T> MutableMetaProvider.value(
/* Read-write delegates for [MutableItemProvider] */
-public fun MutableMetaProvider.string(key: Name? = null): ReadWriteProperty<Any?, String?> =
+public fun MutableMetaProvider.string(key: Name? = null): MutableMetaDelegate<String?> =
value(key) { it?.string }
-public fun MutableMetaProvider.boolean(key: Name? = null): ReadWriteProperty<Any?, Boolean?> =
+public fun MutableMetaProvider.boolean(key: Name? = null): MutableMetaDelegate<Boolean?> =
value(key) { it?.boolean }
-public fun MutableMetaProvider.number(key: Name? = null): ReadWriteProperty<Any?, Number?> =
+public fun MutableMetaProvider.number(key: Name? = null): MutableMetaDelegate<Number?> =
value(key) { it?.number }
-public fun MutableMetaProvider.string(default: String, key: Name? = null): ReadWriteProperty<Any?, String> =
+public fun MutableMetaProvider.string(default: String, key: Name? = null): MutableMetaDelegate<String> =
value(key) { it?.string ?: default }
-public fun MutableMetaProvider.boolean(default: Boolean, key: Name? = null): ReadWriteProperty<Any?, Boolean> =
+public fun MutableMetaProvider.boolean(default: Boolean, key: Name? = null): MutableMetaDelegate<Boolean> =
value(key) { it?.boolean ?: default }
-public fun MutableMetaProvider.number(default: Number, key: Name? = null): ReadWriteProperty<Any?, Number> =
+public fun MutableMetaProvider.number(default: Number, key: Name? = null): MutableMetaDelegate<Number> =
value(key) { it?.number ?: default }
-public fun MutableMetaProvider.string(key: Name? = null, default: () -> String): ReadWriteProperty<Any?, String> =
+public fun MutableMetaProvider.string(key: Name? = null, default: () -> String): MutableMetaDelegate<String> =
value(key) { it?.string ?: default() }
-public fun MutableMetaProvider.boolean(key: Name? = null, default: () -> Boolean): ReadWriteProperty<Any?, Boolean> =
+public fun MutableMetaProvider.boolean(key: Name? = null, default: () -> Boolean): MutableMetaDelegate<Boolean> =
value(key) { it?.boolean ?: default() }
-public fun MutableMetaProvider.number(key: Name? = null, default: () -> Number): ReadWriteProperty<Any?, Number> =
+public fun MutableMetaProvider.number(key: Name? = null, default: () -> Number): MutableMetaDelegate<Number> =
value(key) { it?.number ?: default() }
public inline fun <reified E : Enum<E>> MutableMetaProvider.enum(
default: E,
key: Name? = null,
-): ReadWriteProperty<Any?, E> = value(key) { value -> value?.string?.let { enumValueOf<E>(it) } ?: default }
+): MutableMetaDelegate<E> = value(key) { value -> value?.string?.let { enumValueOf<E>(it) } ?: default }
/* Number delegates */
-public fun MutableMetaProvider.int(key: Name? = null): ReadWriteProperty<Any?, Int?> =
+public fun MutableMetaProvider.int(key: Name? = null): MutableMetaDelegate<Int?> =
value(key) { it?.int }
-public fun MutableMetaProvider.double(key: Name? = null): ReadWriteProperty<Any?, Double?> =
+public fun MutableMetaProvider.double(key: Name? = null): MutableMetaDelegate<Double?> =
value(key) { it?.double }
-public fun MutableMetaProvider.long(key: Name? = null): ReadWriteProperty<Any?, Long?> =
+public fun MutableMetaProvider.long(key: Name? = null): MutableMetaDelegate<Long?> =
value(key) { it?.long }
-public fun MutableMetaProvider.float(key: Name? = null): ReadWriteProperty<Any?, Float?> =
+public fun MutableMetaProvider.float(key: Name? = null): MutableMetaDelegate<Float?> =
value(key) { it?.float }
/* Safe number delegates*/
-public fun MutableMetaProvider.int(default: Int, key: Name? = null): ReadWriteProperty<Any?, Int> =
+public fun MutableMetaProvider.int(default: Int, key: Name? = null): MutableMetaDelegate<Int> =
value(key) { it?.int ?: default }
-public fun MutableMetaProvider.double(default: Double, key: Name? = null): ReadWriteProperty<Any?, Double> =
+public fun MutableMetaProvider.double(default: Double, key: Name? = null): MutableMetaDelegate<Double> =
value(key) { it?.double ?: default }
-public fun MutableMetaProvider.long(default: Long, key: Name? = null): ReadWriteProperty<Any?, Long> =
+public fun MutableMetaProvider.long(default: Long, key: Name? = null): MutableMetaDelegate<Long> =
value(key) { it?.long ?: default }
-public fun MutableMetaProvider.float(default: Float, key: Name? = null): ReadWriteProperty<Any?, Float> =
+public fun MutableMetaProvider.float(default: Float, key: Name? = null): MutableMetaDelegate<Float> =
value(key) { it?.float ?: default }
@@ -124,7 +218,7 @@ public fun MutableMetaProvider.float(default: Float, key: Name? = null): ReadWri
public fun MutableMetaProvider.stringList(
vararg default: String,
key: Name? = null,
-): ReadWriteProperty<Any?, List<String>> = value(
+): MutableMetaDelegate<List<String>> = value(
key,
writer = { list -> list.map { str -> str.asValue() }.asValue() },
reader = { it?.stringList ?: listOf(*default) },
@@ -132,7 +226,7 @@ public fun MutableMetaProvider.stringList(
public fun MutableMetaProvider.stringList(
key: Name? = null,
-): ReadWriteProperty<Any?, List<String>?> = value(
+): MutableMetaDelegate<List<String>?> = value(
key,
writer = { it -> it?.map { str -> str.asValue() }?.asValue() },
reader = { it?.stringList },
@@ -141,29 +235,18 @@ public fun MutableMetaProvider.stringList(
public fun MutableMetaProvider.numberList(
vararg default: Number,
key: Name? = null,
-): ReadWriteProperty<Any?, List<Number>> = value(
+): MutableMetaDelegate<List<Number>> = value(
key,
writer = { it.map { num -> num.asValue() }.asValue() },
reader = { it?.list?.map { value -> value.numberOrNull ?: Double.NaN } ?: listOf(*default) },
)
-/* A special delegate for double arrays */
-
-
-public fun MutableMetaProvider.doubleArray(
- vararg default: Double,
- key: Name? = null,
-): ReadWriteProperty<Any?, DoubleArray> = value(
- key,
- writer = { DoubleArrayValue(it) },
- reader = { it?.doubleArray ?: doubleArrayOf(*default) },
-)
public fun <T> MutableMetaProvider.listValue(
key: Name? = null,
writer: (T) -> Value = { Value.of(it) },
reader: (Value) -> T,
-): ReadWriteProperty<Any?, List<T>?> = value(
+): MutableMetaDelegate<List<T>?> = value(
key,
writer = { it?.map(writer)?.asValue() },
reader = { it?.list?.map(reader) }
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MutableMetaView.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MutableMetaView.kt
new file mode 100644
index 00000000..2bc3f9aa
--- /dev/null
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/MutableMetaView.kt
@@ -0,0 +1,47 @@
+package space.kscience.dataforge.meta
+
+import space.kscience.dataforge.names.Name
+import space.kscience.dataforge.names.NameToken
+import space.kscience.dataforge.names.parseAsName
+import space.kscience.dataforge.names.plus
+
+/**
+ * A [Meta] child proxy that creates required nodes on value write
+ */
+private class MutableMetaView(
+ val origin: MutableMeta,
+ val path: Name
+) : MutableMeta {
+
+ override val items: Map<NameToken, MutableMeta>
+ get() = origin[path]?.items ?: emptyMap()
+
+ override var value: Value?
+ get() = origin[path]?.value
+ set(value) {
+ origin[path] = value
+ }
+
+ override fun getOrCreate(name: Name): MutableMeta = MutableMetaView(origin, path + name)
+
+ override fun set(name: Name, node: Meta?) {
+ if (origin[path + name] == null && node?.isEmpty() == true) return
+ origin[path + name] = node
+ }
+
+ override fun equals(other: Any?): Boolean = Meta.equals(this, other as? Meta)
+
+ override fun hashCode(): Int = Meta.hashCode(this)
+
+ override fun toString(): String = Meta.toString(this)
+}
+
+/**
+ * Create a view of this [MutableMeta] node that creates child items only when their values are written.
+ *
+ * The difference between this method and regular [getOrCreate] is that [getOrCreate] always creates and attaches node
+ * even if it is empty.
+ */
+public fun MutableMeta.view(name: Name): MutableMeta = MutableMetaView(this, name)
+
+public fun MutableMeta.view(name: String): MutableMeta = view(name.parseAsName())
\ No newline at end of file
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/ObservableMeta.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/ObservableMeta.kt
index 5a2b05f7..3e53891b 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/ObservableMeta.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/ObservableMeta.kt
@@ -1,8 +1,8 @@
package space.kscience.dataforge.meta
+import kotlinx.serialization.Serializable
import space.kscience.dataforge.misc.ThreadSafe
-import space.kscience.dataforge.names.*
-import kotlin.reflect.KProperty1
+import space.kscience.dataforge.names.Name
internal data class MetaListener(
@@ -15,12 +15,15 @@ internal data class MetaListener(
*/
public interface ObservableMeta : Meta {
/**
- * Add change listener to this meta. Owner is declared to be able to remove listeners later. Listener without owner could not be removed
+ * Add change listener to this meta. The Owner is declared to be able to remove listeners later.
+ * Listeners without an owner could be only removed all together.
+ *
+ * `this` object in the listener represents the current state of this meta. The name points to a changed node
*/
public fun onChange(owner: Any?, callback: Meta.(name: Name) -> Unit)
/**
- * Remove all listeners belonging to given owner
+ * Remove all listeners belonging to the given [owner]. Passing null removes all listeners.
*/
public fun removeListener(owner: Any?)
@@ -33,18 +36,10 @@ public interface ObservableMeta : Meta {
/**
* A [Meta] which is both observable and mutable
*/
+@Serializable(ObservableMutableMetaSerializer::class)
+@MetaBuilderMarker
public interface ObservableMutableMeta : ObservableMeta, MutableMeta, MutableTypedMeta<ObservableMutableMeta> {
- override fun getOrCreate(name: Name): ObservableMutableMeta
-
- override fun get(name: Name): ObservableMutableMeta? {
- tailrec fun ObservableMutableMeta.find(name: Name): ObservableMutableMeta? = if (name.isEmpty()) {
- this
- } else {
- items[name.firstOrNull()!!]?.find(name.cutFirst())
- }
-
- return find(name)
- }
+ override val self: ObservableMutableMeta get() = this
}
internal abstract class AbstractObservableMeta : ObservableMeta {
@@ -67,24 +62,4 @@ internal abstract class AbstractObservableMeta : ObservableMeta {
override fun toString(): String = Meta.toString(this)
override fun equals(other: Any?): Boolean = Meta.equals(this, other as? Meta)
override fun hashCode(): Int = Meta.hashCode(this)
-}
-
-/**
- * Use the value of the property in a [callBack].
- * The callback is called once immediately after subscription to pass the initial value.
- *
- * Optional [owner] property is used for
- */
-public fun <S : Scheme, T> S.useProperty(
- property: KProperty1<S, T>,
- owner: Any? = null,
- callBack: S.(T) -> Unit,
-) {
- //Pass initial value.
- callBack(property.get(this))
- meta.onChange(owner) { name ->
- if (name.startsWith(property.name.asName())) {
- callBack(property.get(this@useProperty))
- }
- }
}
\ No newline at end of file
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/ObservableMetaWrapper.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/ObservableMetaWrapper.kt
index 76645d83..c8780417 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/ObservableMetaWrapper.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/ObservableMetaWrapper.kt
@@ -6,60 +6,74 @@ import space.kscience.dataforge.names.*
/**
* A class that takes [MutableMeta] provider and adds obsevability on top of that
+ *
+ * TODO rewrite to properly work with detached nodes
*/
private class ObservableMetaWrapper(
val root: MutableMeta,
- val absoluteName: Name,
+ val nodeName: Name,
val listeners: MutableSet<MetaListener>,
) : ObservableMutableMeta {
- override val items: Map<NameToken, ObservableMutableMeta>
- get() = root.items.keys.associateWith {
- ObservableMetaWrapper(root, absoluteName + it, listeners)
- }
- override fun get(name: Name): ObservableMutableMeta? =
- root.get(name)?.let { ObservableMetaWrapper(root, this.absoluteName + name, listeners) }
+ override val self get() = this
+
+ override val items: Map<NameToken, ObservableMutableMeta>
+ get() = root[nodeName]?.items?.keys?.associateWith {
+ ObservableMetaWrapper(root, nodeName + it, listeners)
+ } ?: emptyMap()
+
+ override fun get(name: Name): ObservableMutableMeta? = if (root[nodeName + name] == null) {
+ null
+ } else {
+ ObservableMetaWrapper(root, nodeName + name, listeners)
+ }
@ThreadSafe
override fun onChange(owner: Any?, callback: Meta.(name: Name) -> Unit) {
listeners.add(
- MetaListener(Pair(owner, absoluteName)) { name ->
- if (name.startsWith(absoluteName)) {
- (this[absoluteName] ?: Meta.EMPTY).callback(name.removeFirstOrNull(absoluteName)!!)
+ MetaListener(Pair(owner, nodeName)) { fullName ->
+ if (fullName.startsWith(nodeName)) {
+ root[nodeName]?.callback(fullName.removeFirstOrNull(nodeName)!!)
}
}
)
}
override fun removeListener(owner: Any?) {
- listeners.removeAll { it.owner === Pair(owner, absoluteName) }
+ listeners.removeAll { it.owner === Pair(owner, nodeName) }
}
override fun invalidate(name: Name) {
- listeners.forEach { it.callback(this, name) }
+ listeners.forEach { it.callback(this, nodeName + name) }
}
override var value: Value?
- get() = root.value
+ get() = root[nodeName]?.value
set(value) {
- root.value = value
+ root.getOrCreate(nodeName).value = value
invalidate(Name.EMPTY)
}
override fun getOrCreate(name: Name): ObservableMutableMeta =
- ObservableMetaWrapper(root, this.absoluteName + name, listeners)
+ ObservableMetaWrapper(root, nodeName + name, listeners)
+
+ fun removeNode(name: Name): Meta? {
+ val oldMeta = get(name)
+ //remember to remove listener
+ oldMeta?.removeListener(this)
+
+ return oldMeta
+ }
override fun set(name: Name, node: Meta?) {
- val oldMeta = get(name)
- //don't forget to remove listener
- oldMeta?.removeListener(this)
- root.set(absoluteName + name, node)
+ val oldMeta = removeNode(name)
+ root[nodeName + name] = node
if (oldMeta != node) {
invalidate(name)
}
}
- override fun toMeta(): Meta = root[absoluteName]?.toMeta() ?: Meta.EMPTY
+ override fun toMeta(): Meta = root[nodeName]?.toMeta() ?: Meta.EMPTY
override fun toString(): String = Meta.toString(this)
override fun equals(other: Any?): Boolean = Meta.equals(this, other as? Meta)
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Scheme.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Scheme.kt
index 90473286..12eb4c68 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Scheme.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Scheme.kt
@@ -7,36 +7,45 @@ import space.kscience.dataforge.meta.descriptors.validate
import space.kscience.dataforge.misc.DFExperimental
import space.kscience.dataforge.misc.ThreadSafe
import space.kscience.dataforge.names.*
+import kotlin.properties.ReadWriteProperty
+import kotlin.reflect.KProperty
+import kotlin.reflect.KProperty1
/**
- * A base for delegate-based or descriptor-based scheme. [Scheme] has an empty constructor to simplify usage from [Specification].
- * Default item provider and [MetaDescriptor] are optional
+ * A base for delegate-based or descriptor-based scheme. [Scheme] has an empty constructor to simplify usage from [MetaReader].
+ *
+ * @param prototype default values provided by this scheme
*/
-public open class Scheme : Described, MetaRepr, MutableMetaProvider, Configurable {
+public open class Scheme(
+ private var prototype: Meta? = null,
+ descriptor: MetaDescriptor? = null,
+) : Described, MetaRepr, MutableMetaProvider, Configurable {
/**
- * Meta to be mutated by this schme
+ * Meta to be mutated by this scheme
*/
- private var targetMeta: MutableMeta = MutableMeta()
+ internal var target: MutableMeta = MutableMeta()
/**
- * Default values provided by this scheme
+ * A descriptor of this scheme
*/
- private var defaultMeta: Meta? = null
+ final override var descriptor: MetaDescriptor? = descriptor
+ private set
+
final override val meta: ObservableMutableMeta = SchemeMeta(Name.EMPTY)
- final override var descriptor: MetaDescriptor? = null
- internal set
-
- internal fun wrap(
- newMeta: MutableMeta,
- preserveDefault: Boolean = false,
+ /**
+ * This method must be called before the scheme could be used
+ */
+ internal fun initialize(
+ target: MutableMeta,
+ prototype: Meta,
+ descriptor: MetaDescriptor?,
) {
- if (preserveDefault) {
- defaultMeta = targetMeta.seal()
- }
- targetMeta = newMeta
+ this.target = target
+ this.prototype = prototype
+ this.descriptor = descriptor
}
/**
@@ -47,11 +56,11 @@ public open class Scheme : Described, MetaRepr, MutableMetaProvider, Configurabl
return descriptor?.validate(meta) ?: true
}
- override fun get(name: Name): MutableMeta? = meta.get(name)
+ override fun get(name: Name): MutableMeta? = meta[name]
override fun set(name: Name, node: Meta?) {
if (validate(name, meta)) {
- meta.set(name, node)
+ meta[name] = node
} else {
error("Validation failed for node $node at $name")
}
@@ -68,14 +77,19 @@ public open class Scheme : Described, MetaRepr, MutableMetaProvider, Configurabl
private val listeners: MutableList<MetaListener> = mutableListOf()
+ override fun toString(): String = meta.toString()
+
private inner class SchemeMeta(val pathName: Name) : ObservableMutableMeta {
+
+ override val self get() = this
+
override var value: Value?
- get() = targetMeta[pathName]?.value
- ?: defaultMeta?.get(pathName)?.value
+ get() = target[pathName]?.value
+ ?: prototype?.get(pathName)?.value
?: descriptor?.get(pathName)?.defaultValue
set(value) {
- val oldValue = targetMeta[pathName]?.value
- targetMeta[pathName] = value
+ val oldValue = target[pathName]?.value
+ target[pathName] = value
if (oldValue != value) {
invalidate(Name.EMPTY)
}
@@ -83,8 +97,8 @@ public open class Scheme : Described, MetaRepr, MutableMetaProvider, Configurabl
override val items: Map<NameToken, ObservableMutableMeta>
get() {
- val targetKeys = targetMeta[pathName]?.items?.keys ?: emptySet()
- val defaultKeys = defaultMeta?.get(pathName)?.items?.keys ?: emptySet()
+ val targetKeys = target[pathName]?.items?.keys ?: emptySet()
+ val defaultKeys = prototype?.get(pathName)?.items?.keys ?: emptySet()
return (targetKeys + defaultKeys).associateWith { SchemeMeta(pathName + it) }
}
@@ -111,7 +125,7 @@ public open class Scheme : Described, MetaRepr, MutableMetaProvider, Configurabl
override fun hashCode(): Int = Meta.hashCode(this)
override fun set(name: Name, node: Meta?) {
- targetMeta.set(name, node)
+ target[pathName + name] = node
invalidate(name)
}
@@ -119,7 +133,6 @@ public open class Scheme : Described, MetaRepr, MutableMetaProvider, Configurabl
@DFExperimental
override fun attach(name: Name, node: ObservableMutableMeta) {
- //TODO implement zero-copy attachment
set(name, node)
node.onChange(this) { changeName ->
set(name + changeName, this[changeName])
@@ -131,10 +144,11 @@ public open class Scheme : Described, MetaRepr, MutableMetaProvider, Configurabl
/**
* Relocate scheme target onto given [MutableMeta]. Old provider does not get updates anymore.
- * Current state of the scheme used as a default.
+ * The Current state of the scheme that os used as a default.
*/
+@DFExperimental
public fun <T : Scheme> T.retarget(provider: MutableMeta): T = apply {
- wrap(provider, true)
+ initialize(provider, meta.seal(), descriptor)
}
/**
@@ -151,26 +165,151 @@ public inline fun <T : Scheme> T.copy(spec: SchemeSpec<T>, block: T.() -> Unit =
/**
* A specification for simplified generation of wrappers
*/
-public open class SchemeSpec<out T : Scheme>(
+public open class SchemeSpec<T : Scheme>(
private val builder: () -> T,
-) : Specification<T> {
+) : MetaConverter<T> {
- override fun read(source: Meta): T = builder().also {
- it.wrap(MutableMeta().withDefault(source))
- }
-
- override fun write(target: MutableMeta): T = empty().also {
- it.wrap(target)
- }
-
- //TODO Generate descriptor from Scheme class
override val descriptor: MetaDescriptor? get() = null
- override fun empty(): T = builder().also {
- it.descriptor = descriptor
+ override fun readOrNull(source: Meta): T = builder().also {
+ it.initialize(MutableMeta(), source, descriptor)
}
- @Suppress("OVERRIDE_BY_INLINE")
- final override inline operator fun invoke(action: T.() -> Unit): T = empty().apply(action)
+ /**
+ * Write changes made to the [Scheme] to target [MutableMeta]. If the empty [Scheme] contains any data it is copied to the target.
+ */
+ public fun write(target: MutableMeta): T = empty().also {
+ target.update(it.meta)
+ it.initialize(target, Meta.EMPTY, descriptor)
+ }
+ /**
+ * Generate a blank object. The object could contain some elements if they are defined in a constructor
+ */
+ public fun empty(): T = builder().also {
+ it.initialize(MutableMeta(), it.target, descriptor)
+ }
+
+ override fun convert(obj: T): Meta = obj.meta
+
+ /**
+ * A convenience method to use specifications in builders
+ */
+ public inline operator fun invoke(action: T.() -> Unit): T = empty().apply(action)
+
+}
+
+
+/**
+ * Update a [MutableMeta] using given specification
+ */
+public fun <T : Scheme> MutableMeta.updateWith(
+ spec: SchemeSpec<T>,
+ action: T.() -> Unit,
+): T = spec.write(this).apply(action)
+
+
+/**
+ * Update configuration using given specification
+ */
+public fun <T : Scheme> Configurable.updateWith(
+ spec: SchemeSpec<T>,
+ action: T.() -> Unit,
+): T = spec.write(meta).apply(action)
+
+
+/**
+ * A delegate that uses a [MetaReader] to wrap a child of this provider
+ */
+public fun <T : Scheme> MutableMetaProvider.scheme(
+ spec: SchemeSpec<T>,
+ key: Name? = null,
+): ReadWriteProperty<Any?, T> = object : ReadWriteProperty<Any?, T> {
+ override fun getValue(thisRef: Any?, property: KProperty<*>): T {
+ val name = key ?: property.name.asName()
+ val node = get(name) ?: MutableMeta().also { set(name, it) }
+ return spec.write(node)
+ }
+
+ override fun setValue(thisRef: Any?, property: KProperty<*>, value: T) {
+ val name = key ?: property.name.asName()
+ set(name, value.toMeta())
+ }
+}
+
+public fun <T : Scheme> Scheme.scheme(
+ spec: SchemeSpec<T>,
+ key: Name? = null,
+): ReadWriteProperty<Any?, T> = meta.scheme(spec, key)
+
+/**
+ * A delegate that uses a [MetaReader] to wrap a child of this provider.
+ * Returns null if meta with given name does not exist.
+ */
+public fun <T : Scheme> MutableMeta.schemeOrNull(
+ spec: SchemeSpec<T>,
+ key: Name? = null,
+): ReadWriteProperty<Any?, T?> = object : ReadWriteProperty<Any?, T?> {
+ override fun getValue(thisRef: Any?, property: KProperty<*>): T? {
+ val name = key ?: property.name.asName()
+ return if (get(name) == null) null else spec.write(getOrCreate(name))
+ }
+
+ override fun setValue(thisRef: Any?, property: KProperty<*>, value: T?) {
+ val name = key ?: property.name.asName()
+ if (value == null) remove(name)
+ else set(name, value.toMeta())
+ }
+}
+
+public fun <T : Scheme> Scheme.schemeOrNull(
+ spec: SchemeSpec<T>,
+ key: Name? = null,
+): ReadWriteProperty<Any?, T?> = meta.schemeOrNull(spec, key)
+
+/**
+ * A delegate that uses a [MetaReader] to wrap a list of child providers.
+ * If children are mutable, the changes in list elements are reflected on them.
+ * The list is a snapshot of children state, so change in structure is not reflected on its composition.
+ */
+public fun <T : Scheme> MutableMeta.listOfScheme(
+ spec: SchemeSpec<T>,
+ key: Name? = null,
+): ReadWriteProperty<Any?, List<T>> = object : ReadWriteProperty<Any?, List<T>> {
+ override fun getValue(thisRef: Any?, property: KProperty<*>): List<T> {
+ val name = key ?: property.name.asName()
+ return getIndexedList(name).map { spec.write(it as MutableMeta) }
+ }
+
+ override fun setValue(thisRef: Any?, property: KProperty<*>, value: List<T>) {
+ val name = key ?: property.name.asName()
+ setIndexed(name, value.map { it.toMeta() })
+ }
+}
+
+
+public fun <T : Scheme> Scheme.listOfScheme(
+ spec: SchemeSpec<T>,
+ key: Name? = null,
+): ReadWriteProperty<Any?, List<T>> = meta.listOfScheme(spec, key)
+
+
+/**
+ * Use the value of the property in a [callBack].
+ * The callback is called once immediately after subscription to pass the initial value.
+ *
+ * Optional [owner] property is used for
+ */
+public fun <S : Scheme, T> S.useProperty(
+ property: KProperty1<S, T>,
+ owner: Any? = null,
+ callBack: S.(T) -> Unit,
+) {
+ //Pass initial value.
+ callBack(property.get(this))
+ meta.onChange(owner) { name ->
+ if (name.startsWith(property.name.asName())) {
+ callBack(property.get(this@useProperty))
+ }
+ }
}
\ No newline at end of file
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/SealedMeta.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/SealedMeta.kt
index 217a6a04..b218fad6 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/SealedMeta.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/SealedMeta.kt
@@ -13,6 +13,9 @@ public class SealedMeta(
override val value: Value?,
override val items: Map<NameToken, SealedMeta>,
) : TypedMeta<SealedMeta> {
+
+ override val self: SealedMeta get() = this
+
override fun toString(): String = Meta.toString(this)
override fun equals(other: Any?): Boolean = Meta.equals(this, other as? Meta)
@@ -76,6 +79,8 @@ internal class MetaBuilder(
override fun set(name: Name, node: Meta?) {
+ //skip setting if value has not changed
+ if(node == get(name)) return
when (name.length) {
0 -> error("Can't set a meta with empty name")
1 -> {
@@ -89,7 +94,7 @@ internal class MetaBuilder(
}
else -> {
- getOrCreate(name.first().asName()).set(name.cutFirst(), node)
+ getOrCreate(name.first().asName())[name.cutFirst()] = node
}
}
}
@@ -101,11 +106,6 @@ internal class MetaBuilder(
override fun hashCode(): Int = Meta.hashCode(this)
}
-/**
- * Create a read-only meta.
- */
-public inline fun Meta(builder: MutableMeta.() -> Unit): Meta =
- MetaBuilder().apply(builder).seal()
/**
* Create an immutable meta.
@@ -113,6 +113,11 @@ public inline fun Meta(builder: MutableMeta.() -> Unit): Meta =
public inline fun SealedMeta(builder: MutableMeta.() -> Unit): SealedMeta =
MetaBuilder().apply(builder).seal()
+/**
+ * Create a read-only meta.
+ */
+public inline fun Meta(builder: MutableMeta.() -> Unit): Meta = SealedMeta(builder)
+
/**
* Create an empty meta mutable meta.
*/
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Specification.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Specification.kt
deleted file mode 100644
index 6d3afbea..00000000
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Specification.kt
+++ /dev/null
@@ -1,130 +0,0 @@
-package space.kscience.dataforge.meta
-
-import space.kscience.dataforge.meta.descriptors.Described
-import space.kscience.dataforge.misc.DFExperimental
-import space.kscience.dataforge.names.Name
-import space.kscience.dataforge.names.asName
-import kotlin.properties.ReadWriteProperty
-import kotlin.reflect.KProperty
-
-public interface ReadOnlySpecification<out T : Any>: Described {
-
- /**
- * Read generic read-only meta with this [Specification] producing instance of desired type.
- * The source is not mutated even if it is in theory mutable
- */
- public fun read(source: Meta): T
-
- /**
- * Generate an empty object
- */
- public fun empty(): T
-
- /**
- * A convenience method to use specifications in builders
- */
- public operator fun invoke(action: T.() -> Unit): T = empty().apply(action)
-}
-
-
-/**
- * Allows to apply custom configuration in a type safe way to simple untyped configuration.
- * By convention [Scheme] companion should inherit this class
- *
- */
-public interface Specification<out T : Any> : ReadOnlySpecification<T> {
- /**
- * Wrap [MutableMeta], using it as inner storage (changes to [Specification] are reflected on [MutableMeta]
- */
- public fun write(target: MutableMeta): T
-}
-
-/**
- * Update a [MutableMeta] using given specification
- */
-public fun <T : Any> MutableMeta.updateWith(
- spec: Specification<T>,
- action: T.() -> Unit,
-): T = spec.write(this).apply(action)
-
-
-/**
- * Update configuration using given specification
- */
-public fun <T : Any> Configurable.updateWith(
- spec: Specification<T>,
- action: T.() -> Unit,
-): T = spec.write(meta).apply(action)
-
-//
-//public fun <M : MutableTypedMeta<M>> MutableMeta.withSpec(spec: Specification<M>): M? =
-// spec.write(it)
-
-/**
- * A delegate that uses a [Specification] to wrap a child of this provider
- */
-public fun <T : Scheme> MutableMeta.spec(
- spec: Specification<T>,
- key: Name? = null,
-): ReadWriteProperty<Any?, T> = object : ReadWriteProperty<Any?, T> {
- override fun getValue(thisRef: Any?, property: KProperty<*>): T {
- val name = key ?: property.name.asName()
- return spec.write(getOrCreate(name))
- }
-
- override fun setValue(thisRef: Any?, property: KProperty<*>, value: T) {
- val name = key ?: property.name.asName()
- set(name, value.toMeta())
- }
-}
-
-public fun <T : Scheme> Scheme.spec(
- spec: Specification<T>,
- key: Name? = null,
-): ReadWriteProperty<Any?, T> = meta.spec(spec, key)
-
-/**
- * A delegate that uses a [Specification] to wrap a child of this provider.
- * Returns null if meta with given name does not exist.
- */
-public fun <T : Scheme> MutableMeta.specOrNull(
- spec: Specification<T>,
- key: Name? = null,
-): ReadWriteProperty<Any?, T?> = object : ReadWriteProperty<Any?, T?> {
- override fun getValue(thisRef: Any?, property: KProperty<*>): T? {
- val name = key ?: property.name.asName()
- return if (get(name) == null) null else spec.write(getOrCreate(name))
- }
-
- override fun setValue(thisRef: Any?, property: KProperty<*>, value: T?) {
- val name = key ?: property.name.asName()
- if (value == null) remove(name)
- else set(name, value.toMeta())
- }
-}
-
-public fun <T : Scheme> Scheme.specOrNull(
- spec: Specification<T>,
- key: Name? = null,
-): ReadWriteProperty<Any?, T?> = meta.specOrNull(spec, key)
-
-/**
- * A delegate that uses a [Specification] to wrap a list of child providers.
- * If children are mutable, the changes in list elements are reflected on them.
- * The list is a snapshot of children state, so change in structure is not reflected on its composition.
- */
-@DFExperimental
-public fun <T : Scheme> MutableMeta.listOfSpec(
- spec: Specification<T>,
- key: Name? = null,
-): ReadWriteProperty<Any?, List<T>> = object : ReadWriteProperty<Any?, List<T>> {
- override fun getValue(thisRef: Any?, property: KProperty<*>): List<T> {
- val name = key ?: property.name.asName()
- return getIndexed(name).values.map { spec.write(it as MutableMeta) }
- }
-
- override fun setValue(thisRef: Any?, property: KProperty<*>, value: List<T>) {
- val name = key ?: property.name.asName()
- setIndexed(name, value.map { it.toMeta() })
- }
-}
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Value.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Value.kt
index 66e14c86..2ab7b9ee 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Value.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/Value.kt
@@ -256,8 +256,6 @@ public fun ShortArray.asValue(): Value = if (isEmpty()) Null else ListValue(map
public fun FloatArray.asValue(): Value = if (isEmpty()) Null else ListValue(map { NumberValue(it) })
-public fun ByteArray.asValue(): Value = if (isEmpty()) Null else ListValue(map { NumberValue(it) })
-
public fun <E : Enum<E>> E.asValue(): Value = EnumValue(this)
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/ValueSerializer.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/ValueSerializer.kt
index dc13ef4c..0379187f 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/ValueSerializer.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/ValueSerializer.kt
@@ -8,6 +8,9 @@ import kotlinx.serialization.descriptors.element
import kotlinx.serialization.encoding.Decoder
import kotlinx.serialization.encoding.Encoder
+/**
+ * A serializer for [Value]
+ */
public object ValueSerializer : KSerializer<Value> {
private val listSerializer by lazy { ListSerializer(ValueSerializer) }
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/descriptors/MetaDescriptor.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/descriptors/MetaDescriptor.kt
index 742b89ed..45954985 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/descriptors/MetaDescriptor.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/descriptors/MetaDescriptor.kt
@@ -7,6 +7,7 @@ import space.kscience.dataforge.names.*
/**
* Restrictions on value in the node
*/
+@Serializable
public enum class ValueRestriction {
/**
* No restrictions
@@ -27,7 +28,7 @@ public enum class ValueRestriction {
/**
* The descriptor for a meta
* @param description description text
- * @param children child descriptors for this node
+ * @param nodes child descriptors for this node
* @param multiple True if same name siblings with this name are allowed
* @param valueRestriction The requirements for node content
* @param valueTypes list of allowed types for [Meta.value], null if all values are allowed.
@@ -39,7 +40,7 @@ public enum class ValueRestriction {
@Serializable
public data class MetaDescriptor(
public val description: String? = null,
- public val children: Map<String, MetaDescriptor> = emptyMap(),
+ public val nodes: Map<String, MetaDescriptor> = emptyMap(),
public val multiple: Boolean = false,
public val valueRestriction: ValueRestriction = ValueRestriction.NONE,
public val valueTypes: List<ValueType>? = null,
@@ -47,6 +48,9 @@ public data class MetaDescriptor(
public val defaultValue: Value? = null,
public val attributes: Meta = Meta.EMPTY,
) {
+ @Deprecated("Replace by nodes", ReplaceWith("nodes"))
+ public val children: Map<String, MetaDescriptor> get() = nodes
+
/**
* A node constructed of default values for this descriptor and its children
*/
@@ -55,7 +59,7 @@ public data class MetaDescriptor(
defaultValue?.let { defaultValue ->
this.value = defaultValue
}
- children.forEach { (key, descriptor) ->
+ nodes.forEach { (key, descriptor) ->
set(key, descriptor.defaultNode)
}
}
@@ -67,13 +71,13 @@ public data class MetaDescriptor(
}
}
-public val MetaDescriptor.required: Boolean get() = valueRestriction == ValueRestriction.REQUIRED || children.values.any { required }
+public val MetaDescriptor.required: Boolean get() = valueRestriction == ValueRestriction.REQUIRED || nodes.values.any { required }
public val MetaDescriptor.allowedValues: List<Value>? get() = attributes[MetaDescriptor.ALLOWED_VALUES_KEY]?.value?.list
public operator fun MetaDescriptor.get(name: Name): MetaDescriptor? = when (name.length) {
0 -> this
- 1 -> children[name.firstOrNull()!!.toString()]
+ 1 -> nodes[name.firstOrNull()!!.toString()]
else -> get(name.firstOrNull()!!.asName())?.get(name.cutFirst())
}
@@ -95,7 +99,7 @@ public fun MetaDescriptor.validate(item: Meta?): Boolean {
if (item == null) return !required
if (!validate(item.value)) return false
- children.forEach { (key, childDescriptor) ->
+ nodes.forEach { (key, childDescriptor) ->
if (!childDescriptor.validate(item[key])) return false
}
return true
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder.kt
index ae6c171a..2590273e 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/descriptors/MetaDescriptorBuilder.kt
@@ -9,7 +9,8 @@ import space.kscience.dataforge.names.length
import kotlin.collections.set
public class MetaDescriptorBuilder @PublishedApi internal constructor() {
- public var info: String? = null
+ public var description: String? = null
+
public var children: MutableMap<String, MetaDescriptorBuilder> = linkedMapOf()
public var multiple: Boolean = false
public var valueRestriction: ValueRestriction = ValueRestriction.NONE
@@ -40,38 +41,28 @@ public class MetaDescriptorBuilder @PublishedApi internal constructor() {
attributes.apply(block)
}
- public fun item(name: Name, block: MetaDescriptorBuilder.() -> Unit = {}): MetaDescriptorBuilder {
- return when (name.length) {
- 0 -> apply(block)
+ internal fun node(
+ name: Name,
+ descriptorBuilder: MetaDescriptorBuilder,
+ ): Unit {
+ when (name.length) {
+ 0 -> error("Can't set descriptor to root")
+
1 -> {
- val target = MetaDescriptorBuilder().apply(block)
- children[name.first().body] = target
- target
+ children[name.first().body] = descriptorBuilder
}
- else -> {
- children.getOrPut(name.first().body) { MetaDescriptorBuilder() }.item(name.cutFirst(), block)
- }
+ else -> children.getOrPut(name.first().body) {
+ MetaDescriptorBuilder()
+ }.node(name.cutFirst(), descriptorBuilder)
}
}
public fun node(
name: Name,
- descriptor: MetaDescriptor,
- block: MetaDescriptorBuilder.() -> Unit = {},
- ): MetaDescriptorBuilder = when (name.length) {
- 0 -> error("Can't set descriptor to root")
- 1 -> {
- val item = descriptor.toBuilder().apply {
- valueRestriction = ValueRestriction.ABSENT
- }.apply(block)
- children[name.first().body] = item
- item
- }
-
- else -> children.getOrPut(name.first().body) {
- MetaDescriptorBuilder()
- }.node(name.cutFirst(), descriptor, block)
+ descriptorBuilder: MetaDescriptor,
+ ): Unit {
+ node(name, descriptorBuilder.toBuilder())
}
public var allowedValues: List<Value>
@@ -85,10 +76,21 @@ public class MetaDescriptorBuilder @PublishedApi internal constructor() {
allowedValues = values.map { Value.of(it) }
}
+ public fun from(descriptor: MetaDescriptor) {
+ description = descriptor.description
+ children.putAll(descriptor.nodes.mapValues { it.value.toBuilder() })
+ multiple = descriptor.multiple
+ valueRestriction = descriptor.valueRestriction
+ valueTypes = descriptor.valueTypes
+ indexKey = descriptor.indexKey
+ default = descriptor.defaultValue
+ attributes.update(descriptor.attributes)
+ }
+
@PublishedApi
internal fun build(): MetaDescriptor = MetaDescriptor(
- description = info,
- children = children.mapValues { it.value.build() },
+ description = description,
+ nodes = children.mapValues { it.value.build() },
multiple = multiple,
valueRestriction = valueRestriction,
valueTypes = valueTypes,
@@ -98,12 +100,57 @@ public class MetaDescriptorBuilder @PublishedApi internal constructor() {
)
}
-public fun MetaDescriptorBuilder.item(name: String, block: MetaDescriptorBuilder.() -> Unit): MetaDescriptorBuilder =
- item(Name.parse(name), block)
+//public fun MetaDescriptorBuilder.item(name: String, block: MetaDescriptorBuilder.() -> Unit): MetaDescriptorBuilder =
+// item(Name.parse(name), block)
public inline fun MetaDescriptor(block: MetaDescriptorBuilder.() -> Unit): MetaDescriptor =
MetaDescriptorBuilder().apply(block).build()
+/**
+ * Create and configure child node descriptor
+ */
+public fun MetaDescriptorBuilder.node(
+ name: Name,
+ block: MetaDescriptorBuilder.() -> Unit,
+) {
+ node(
+ name,
+ MetaDescriptorBuilder().apply(block)
+ )
+}
+
+public fun MetaDescriptorBuilder.node(name: String, descriptor: MetaDescriptor) {
+ node(Name.parse(name), descriptor)
+}
+
+public fun MetaDescriptorBuilder.node(name: String, block: MetaDescriptorBuilder.() -> Unit) {
+ node(Name.parse(name), block)
+}
+
+public fun MetaDescriptorBuilder.node(
+ key: String,
+ base: Described,
+ block: MetaDescriptorBuilder.() -> Unit = {},
+) {
+ node(Name.parse(key), base.descriptor?.toBuilder()?.apply(block) ?: MetaDescriptorBuilder())
+}
+
+public fun MetaDescriptorBuilder.required() {
+ valueRestriction = ValueRestriction.REQUIRED
+}
+
+private fun MetaDescriptor.toBuilder(): MetaDescriptorBuilder = MetaDescriptorBuilder().apply {
+ description = this@toBuilder.description
+ children = this@toBuilder.nodes.mapValuesTo(LinkedHashMap()) { it.value.toBuilder() }
+ multiple = this@toBuilder.multiple
+ valueRestriction = this@toBuilder.valueRestriction
+ valueTypes = this@toBuilder.valueTypes
+ indexKey = this@toBuilder.indexKey
+ default = defaultValue
+ attributes = this@toBuilder.attributes.toMutableMeta()
+}
+
+
/**
* Create and configure child value descriptor
*/
@@ -112,7 +159,7 @@ public fun MetaDescriptorBuilder.value(
type: ValueType,
vararg additionalTypes: ValueType,
block: MetaDescriptorBuilder.() -> Unit = {},
-): MetaDescriptorBuilder = item(name) {
+): Unit = node(name) {
valueType(type, *additionalTypes)
block()
}
@@ -122,41 +169,14 @@ public fun MetaDescriptorBuilder.value(
type: ValueType,
vararg additionalTypes: ValueType,
block: MetaDescriptorBuilder.() -> Unit = {},
-): MetaDescriptorBuilder = value(Name.parse(name), type, additionalTypes = additionalTypes, block)
+): Unit = value(Name.parse(name), type, additionalTypes = additionalTypes, block)
-/**
- * Create and configure child value descriptor
- */
-public fun MetaDescriptorBuilder.node(
- name: Name, block: MetaDescriptorBuilder.() -> Unit,
-): MetaDescriptorBuilder = item(name) {
- valueRestriction = ValueRestriction.ABSENT
- block()
-}
-
-public fun MetaDescriptorBuilder.node(name: String, block: MetaDescriptorBuilder.() -> Unit) {
- node(Name.parse(name), block)
-}
-
-public fun MetaDescriptorBuilder.node(
- key: String,
- described: Described,
- block: MetaDescriptorBuilder.() -> Unit = {},
-) {
- described.descriptor?.let {
- node(Name.parse(key), it, block)
- }
-}
-
-public fun MetaDescriptorBuilder.required() {
- valueRestriction = ValueRestriction.REQUIRED
-}
public inline fun <reified E : Enum<E>> MetaDescriptorBuilder.enum(
key: Name,
default: E?,
crossinline modifier: MetaDescriptorBuilder.() -> Unit = {},
-): MetaDescriptorBuilder = value(key, ValueType.STRING) {
+): Unit = value(key, ValueType.STRING) {
default?.let {
this.default = default.asValue()
}
@@ -164,17 +184,6 @@ public inline fun <reified E : Enum<E>> MetaDescriptorBuilder.enum(
modifier()
}
-private fun MetaDescriptor.toBuilder(): MetaDescriptorBuilder = MetaDescriptorBuilder().apply {
- info = this@toBuilder.description
- children = this@toBuilder.children.mapValuesTo(LinkedHashMap()) { it.value.toBuilder() }
- multiple = this@toBuilder.multiple
- valueRestriction = this@toBuilder.valueRestriction
- valueTypes = this@toBuilder.valueTypes
- indexKey = this@toBuilder.indexKey
- default = defaultValue
- attributes = this@toBuilder.attributes.toMutableMeta()
-}
-
/**
* Make a deep copy of this descriptor applying given transformation [block]
*/
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/descriptors/schemeDescriptor.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/descriptors/schemeDescriptor.kt
index 79fb6fdb..c7b73508 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/descriptors/schemeDescriptor.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/descriptors/schemeDescriptor.kt
@@ -6,10 +6,13 @@ import space.kscience.dataforge.meta.ValueType
import kotlin.reflect.KProperty1
import kotlin.reflect.typeOf
+/**
+ * Add a value item to a [MetaDescriptor] inferring some of its properties from the type
+ */
public inline fun <S : Scheme, reified T> MetaDescriptorBuilder.value(
property: KProperty1<S, T>,
noinline block: MetaDescriptorBuilder.() -> Unit = {},
-): MetaDescriptorBuilder = when (typeOf<T>()) {
+): Unit = when (typeOf<T>()) {
typeOf<Number>(), typeOf<Int>(), typeOf<Double>(), typeOf<Short>(), typeOf<Long>(), typeOf<Float>() ->
value(property.name, ValueType.NUMBER) {
block()
@@ -34,9 +37,12 @@ public inline fun <S : Scheme, reified T> MetaDescriptorBuilder.value(
multiple = true
block()
}
- else -> item(property.name, block)
+ else -> node(property.name, block)
}
+/**
+ * Add a schem-based branch to a [MetaDescriptor]
+ */
public inline fun <S : Scheme, reified T : Scheme> MetaDescriptorBuilder.scheme(
property: KProperty1<S, T>,
spec: SchemeSpec<T>,
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/exoticValues.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/exoticValues.kt
index 74952053..17436990 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/exoticValues.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/exoticValues.kt
@@ -1,5 +1,9 @@
package space.kscience.dataforge.meta
+import space.kscience.dataforge.names.Name
+import kotlin.properties.ReadOnlyProperty
+import kotlin.properties.ReadWriteProperty
+
/**
* A value built from string which content and type are parsed on-demand
@@ -17,6 +21,9 @@ public class LazyParsedValue(public val string: String) : Value {
override fun hashCode(): Int = string.hashCode()
}
+/**
+ * Read this string as lazily parsed value
+ */
public fun String.lazyParseValue(): LazyParsedValue = LazyParsedValue(this)
/**
@@ -43,4 +50,102 @@ public class DoubleArrayValue(override val value: DoubleArray) : Value, Iterable
override fun iterator(): Iterator<Double> = value.iterator()
}
+
+/**
+ * A zero-copy wrapping of this [DoubleArray] in a [Value]
+ */
public fun DoubleArray.asValue(): Value = if (isEmpty()) Null else DoubleArrayValue(this)
+
+public val Value.doubleArray: DoubleArray
+ get() = if (this is DoubleArrayValue) {
+ value
+ } else {
+ DoubleArray(list.size) { list[it].double }
+ }
+
+public val Meta?.doubleArray: DoubleArray? get() = this?.value?.doubleArray
+
+public fun MetaProvider.doubleArray(
+ vararg default: Double,
+ key: Name? = null,
+): ReadOnlyProperty<Any?, DoubleArray> = value(
+ key,
+ reader = { it?.doubleArray ?: doubleArrayOf(*default) },
+)
+
+public fun MutableMetaProvider.doubleArray(
+ vararg default: Double,
+ key: Name? = null,
+): ReadWriteProperty<Any?, DoubleArray> = value(
+ key,
+ writer = { DoubleArrayValue(it) },
+ reader = { it?.doubleArray ?: doubleArrayOf(*default) },
+)
+
+private object DoubleArrayMetaConverter : MetaConverter<DoubleArray> {
+ override fun readOrNull(source: Meta): DoubleArray? = source.doubleArray
+
+ override fun convert(obj: DoubleArray): Meta = Meta(obj.asValue())
+}
+
+public val MetaConverter.Companion.doubleArray: MetaConverter<DoubleArray> get() = DoubleArrayMetaConverter
+
+/**
+ * A [Value] wrapping a [ByteArray]
+ */
+public class ByteArrayValue(override val value: ByteArray) : Value, Iterable<Byte> {
+ override val type: ValueType get() = ValueType.LIST
+ override val list: List<Value> get() = value.map { NumberValue(it) }
+
+ override fun equals(other: Any?): Boolean {
+ if (this === other) return true
+ if (other !is Value) return false
+
+ return when (other) {
+ is ByteArrayValue -> value.contentEquals(other.value)
+ else -> list == other.list
+ }
+ }
+
+ override fun hashCode(): Int = value.contentHashCode()
+
+ override fun toString(): String = list.joinToString(prefix = "[", postfix = "]")
+
+ override fun iterator(): Iterator<Byte> = value.iterator()
+}
+
+public fun ByteArray.asValue(): Value = ByteArrayValue(this)
+
+public val Value.byteArray: ByteArray
+ get() = if (this is ByteArrayValue) {
+ value
+ } else {
+ ByteArray(list.size) { list[it].number.toByte() }
+ }
+
+public val Meta?.byteArray: ByteArray? get() = this?.value?.byteArray
+
+public fun MetaProvider.byteArray(
+ vararg default: Byte,
+ key: Name? = null,
+): ReadOnlyProperty<Any?, ByteArray> = value(
+ key,
+ reader = { it?.byteArray ?: byteArrayOf(*default) },
+)
+
+public fun MutableMetaProvider.byteArray(
+ vararg default: Byte,
+ key: Name? = null,
+): ReadWriteProperty<Any?, ByteArray> = value(
+ key,
+ writer = { ByteArrayValue(it) },
+ reader = { it?.byteArray ?: byteArrayOf(*default) },
+)
+
+private object ByteArrayMetaConverter : MetaConverter<ByteArray> {
+ override fun readOrNull(source: Meta): ByteArray? = source.byteArray
+
+ override fun convert(obj: ByteArray): Meta = Meta(obj.asValue())
+}
+
+public val MetaConverter.Companion.byteArray: MetaConverter<ByteArray> get() = ByteArrayMetaConverter
\ No newline at end of file
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/transformations/MetaConverter.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/transformations/MetaConverter.kt
deleted file mode 100644
index 0dfb63d7..00000000
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/transformations/MetaConverter.kt
+++ /dev/null
@@ -1,163 +0,0 @@
-package space.kscience.dataforge.meta.transformations
-
-import space.kscience.dataforge.meta.*
-import space.kscience.dataforge.meta.descriptors.MetaDescriptor
-import kotlin.reflect.KType
-import kotlin.reflect.typeOf
-
-/**
- * A converter of generic object to and from [Meta]
- */
-public interface MetaConverter<T> {
-
- /**
- * Runtime type of [T]
- */
- public val type: KType
-
- /**
- * A descriptor for resulting meta
- */
- public val descriptor: MetaDescriptor get() = MetaDescriptor.EMPTY
-
- /**
- * Attempt conversion of [meta] to an object or return null if conversion failed
- */
- public fun metaToObjectOrNull(meta: Meta): T?
-
- public fun metaToObject(meta: Meta): T =
- metaToObjectOrNull(meta) ?: error("Meta $meta could not be interpreted by $this")
-
- public fun objectToMeta(obj: T): Meta
-
- public companion object {
-
- public val meta: MetaConverter<Meta> = object : MetaConverter<Meta> {
- override val type: KType = typeOf<Meta>()
-
- override fun metaToObjectOrNull(meta: Meta): Meta = meta
- override fun objectToMeta(obj: Meta): Meta = obj
- }
-
- public val value: MetaConverter<Value> = object : MetaConverter<Value> {
- override val type: KType = typeOf<Value>()
-
- override fun metaToObjectOrNull(meta: Meta): Value? = meta.value
- override fun objectToMeta(obj: Value): Meta = Meta(obj)
- }
-
- public val string: MetaConverter<String> = object : MetaConverter<String> {
- override val type: KType = typeOf<String>()
-
- override val descriptor: MetaDescriptor = MetaDescriptor {
- valueType(ValueType.STRING)
- }
-
-
- override fun metaToObjectOrNull(meta: Meta): String? = meta.string
- override fun objectToMeta(obj: String): Meta = Meta(obj.asValue())
- }
-
- public val boolean: MetaConverter<Boolean> = object : MetaConverter<Boolean> {
- override val type: KType = typeOf<Boolean>()
-
- override val descriptor: MetaDescriptor = MetaDescriptor {
- valueType(ValueType.BOOLEAN)
- }
-
- override fun metaToObjectOrNull(meta: Meta): Boolean? = meta.boolean
- override fun objectToMeta(obj: Boolean): Meta = Meta(obj.asValue())
- }
-
- public val number: MetaConverter<Number> = object : MetaConverter<Number> {
- override val type: KType = typeOf<Number>()
-
- override val descriptor: MetaDescriptor = MetaDescriptor {
- valueType(ValueType.NUMBER)
- }
-
- override fun metaToObjectOrNull(meta: Meta): Number? = meta.number
- override fun objectToMeta(obj: Number): Meta = Meta(obj.asValue())
- }
-
- public val double: MetaConverter<Double> = object : MetaConverter<Double> {
- override val type: KType = typeOf<Double>()
-
- override val descriptor: MetaDescriptor = MetaDescriptor {
- valueType(ValueType.NUMBER)
- }
-
- override fun metaToObjectOrNull(meta: Meta): Double? = meta.double
- override fun objectToMeta(obj: Double): Meta = Meta(obj.asValue())
- }
-
- public val float: MetaConverter<Float> = object : MetaConverter<Float> {
- override val type: KType = typeOf<Float>()
-
- override val descriptor: MetaDescriptor = MetaDescriptor {
- valueType(ValueType.NUMBER)
- }
-
- override fun metaToObjectOrNull(meta: Meta): Float? = meta.float
- override fun objectToMeta(obj: Float): Meta = Meta(obj.asValue())
- }
-
- public val int: MetaConverter<Int> = object : MetaConverter<Int> {
- override val type: KType = typeOf<Int>()
-
- override val descriptor: MetaDescriptor = MetaDescriptor {
- valueType(ValueType.NUMBER)
- }
-
- override fun metaToObjectOrNull(meta: Meta): Int? = meta.int
- override fun objectToMeta(obj: Int): Meta = Meta(obj.asValue())
- }
-
- public val long: MetaConverter<Long> = object : MetaConverter<Long> {
- override val type: KType = typeOf<Long>()
-
- override val descriptor: MetaDescriptor = MetaDescriptor {
- valueType(ValueType.NUMBER)
- }
-
- override fun metaToObjectOrNull(meta: Meta): Long? = meta.long
- override fun objectToMeta(obj: Long): Meta = Meta(obj.asValue())
- }
-
- public inline fun <reified E : Enum<E>> enum(): MetaConverter<E> = object : MetaConverter<E> {
- override val type: KType = typeOf<E>()
-
- override val descriptor: MetaDescriptor = MetaDescriptor {
- valueType(ValueType.STRING)
- allowedValues(enumValues<E>())
- }
-
- @Suppress("USELESS_CAST")
- override fun metaToObjectOrNull(meta: Meta): E = meta.enum<E>() as? E ?: error("The Item is not a Enum")
-
- override fun objectToMeta(obj: E): Meta = Meta(obj.asValue())
- }
-
- public fun <T> valueList(
- writer: (T) -> Value = { Value.of(it) },
- reader: (Value) -> T,
- ): MetaConverter<List<T>> =
- object : MetaConverter<List<T>> {
- override val type: KType = typeOf<List<T>>()
-
- override val descriptor: MetaDescriptor = MetaDescriptor {
- valueType(ValueType.LIST)
- }
-
- override fun metaToObjectOrNull(meta: Meta): List<T>? = meta.value?.list?.map(reader)
-
- override fun objectToMeta(obj: List<T>): Meta = Meta(obj.map(writer).asValue())
- }
-
- }
-}
-
-public fun <T : Any> MetaConverter<T>.nullableMetaToObject(item: Meta?): T? = item?.let { metaToObject(it) }
-public fun <T : Any> MetaConverter<T>.nullableObjectToMeta(obj: T?): Meta? = obj?.let { objectToMeta(it) }
-
-public fun <T> MetaConverter<T>.valueToObject(value: Value): T? = metaToObject(Meta(value))
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/valueExtensions.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/valueExtensions.kt
index e6b622ff..0c87bcc2 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/valueExtensions.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/meta/valueExtensions.kt
@@ -11,9 +11,18 @@ public fun Value.isNull(): Boolean = this == Null
public fun Value.isList(): Boolean = this.type == ValueType.LIST
public val Value.boolean: Boolean
- get() = this == True
- || this.list.firstOrNull() == True
- || (type == ValueType.STRING && string.toBoolean())
+ get() = when (type) {
+ ValueType.NUMBER -> int > 0
+ ValueType.STRING -> string.toBoolean()
+ ValueType.BOOLEAN -> this === True
+ ValueType.LIST -> list.singleOrNull()?.boolean == true
+ ValueType.NULL -> false
+ }
+
+// this == True
+// || this.list.firstOrNull() == True
+// || (type == ValueType.STRING && string.toBoolean())
+// || (type == ValueType.)
public val Value.int: Int get() = number.toInt()
@@ -31,12 +40,5 @@ public inline fun <reified E : Enum<E>> Value.enum(): E = if (this is EnumValue<
public val Value.stringList: List<String> get() = list.map { it.string }
-public val Value.doubleArray: DoubleArray
- get() = if (this is DoubleArrayValue) {
- value
- } else {
- DoubleArray(list.size) { list[it].double }
- }
-
public fun Value.toMeta(): Meta = Meta(this)
\ No newline at end of file
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/misc/DfId.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/misc/DfType.kt
similarity index 80%
rename from dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/misc/DfId.kt
rename to dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/misc/DfType.kt
index 5d485e23..936e793a 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/misc/DfId.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/misc/DfType.kt
@@ -5,4 +5,4 @@ package space.kscience.dataforge.misc
*/
@MustBeDocumented
@Target(AnnotationTarget.CLASS)
-public annotation class DfId(val id: String)
+public annotation class DfType(val id: String)
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/misc/annotations.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/misc/annotations.kt
index 3c9d6ac3..29568403 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/misc/annotations.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/misc/annotations.kt
@@ -18,4 +18,11 @@ public annotation class DFExperimental
*/
@RequiresOptIn(level = RequiresOptIn.Level.WARNING)
@Retention(AnnotationRetention.BINARY)
-public annotation class DFInternal
\ No newline at end of file
+public annotation class DFInternal
+
+/**
+ * Annotation marks methods that explicitly use KType without checking that it corresponds to the type parameter
+ */
+@RequiresOptIn(level = RequiresOptIn.Level.WARNING)
+@Retention(AnnotationRetention.BINARY)
+public annotation class UnsafeKType
\ No newline at end of file
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/misc/cast.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/misc/cast.kt
deleted file mode 100644
index e714d596..00000000
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/misc/cast.kt
+++ /dev/null
@@ -1,3 +0,0 @@
-package space.kscience.dataforge.misc
-
-public expect inline fun <T> Any?.unsafeCast(): T
\ No newline at end of file
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/names/Name.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/names/Name.kt
index 160ea3a1..7867330c 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/names/Name.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/names/Name.kt
@@ -16,12 +16,10 @@ public class Name(public val tokens: List<NameToken>) {
override fun toString(): String = tokens.joinToString(separator = NAME_SEPARATOR) { it.toString() }
- override fun equals(other: Any?): Boolean {
- return when (other) {
- is Name -> this.tokens == other.tokens
- is NameToken -> this.length == 1 && this.tokens.first() == other
- else -> false
- }
+ override fun equals(other: Any?): Boolean = when (other) {
+ is Name -> this.tokens == other.tokens
+ is NameToken -> this.length == 1 && this.tokens.first() == other
+ else -> false
}
private val cachedHashCode = if (tokens.size == 1) {
@@ -60,7 +58,7 @@ public class Name(public val tokens: List<NameToken>) {
*/
public fun parse(string: String): Name {
if (string.isBlank()) return EMPTY
- val tokens = sequence {
+ val tokens = buildList<NameToken> {
var bodyBuilder = StringBuilder()
var queryBuilder = StringBuilder()
var bracketCount = 0
@@ -93,7 +91,7 @@ public class Name(public val tokens: List<NameToken>) {
else -> when (it) {
'.' -> {
val query = if (queryBuilder.isEmpty()) null else queryBuilder.toString()
- yield(NameToken(bodyBuilder.toString(), query))
+ add(NameToken(bodyBuilder.toString(), query))
bodyBuilder = StringBuilder()
queryBuilder = StringBuilder()
}
@@ -108,13 +106,20 @@ public class Name(public val tokens: List<NameToken>) {
}
}
val query = if (queryBuilder.isEmpty()) null else queryBuilder.toString()
- yield(NameToken(bodyBuilder.toString(), query))
+ add(NameToken(bodyBuilder.toString(), query))
}
- return Name(tokens.toList())
+ return Name(tokens)
}
}
}
+/**
+ * Transform this [Name] to a string without escaping special characters in tokens.
+ *
+ * Parsing it back will produce a valid, but different name
+ */
+public fun Name.toStringUnescaped(): String = tokens.joinToString(separator = Name.NAME_SEPARATOR) { it.toStringUnescaped() }
+
public operator fun Name.get(i: Int): NameToken = tokens[i]
/**
@@ -216,9 +221,13 @@ public fun Name.endsWith(token: NameToken): Boolean = lastOrNull() == token
public fun Name.startsWith(name: Name): Boolean =
this.length >= name.length && (this == name || tokens.subList(0, name.length) == name.tokens)
+public fun Name.startsWith(name: String): Boolean = startsWith(name.parseAsName())
+
public fun Name.endsWith(name: Name): Boolean =
this.length >= name.length && (this == name || tokens.subList(length - name.length, length) == name.tokens)
+public fun Name.endsWith(name: String): Boolean = endsWith(name.parseAsName())
+
/**
* if [this] starts with given [head] name, returns the reminder of the name (could be empty). Otherwise, returns null
*/
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/names/NameIndexComparator.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/names/NameIndexComparator.kt
new file mode 100644
index 00000000..bb95cf65
--- /dev/null
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/names/NameIndexComparator.kt
@@ -0,0 +1,30 @@
+package space.kscience.dataforge.names
+
+import space.kscience.dataforge.meta.Meta
+import space.kscience.dataforge.meta.getIndexed
+
+
+/**
+ * A comparator for indices in a [Name]. If both indices are integers, compare them as integers.
+ * Null always stays "before" non-null index.
+ */
+public object NameIndexComparator : Comparator<String?> {
+ override fun compare(a: String?, b: String?): Int {
+ if (a == b) return 0
+ if (a == null) return 1
+ if (b == null) return -1
+ val aInt = a.toIntOrNull()
+ val bInt = b.toIntOrNull()
+ return if (aInt != null && bInt != null) {
+ aInt.compareTo(bInt)
+ } else {
+ a.compareTo(b)
+ }
+ }
+
+}
+
+public fun Meta.getIndexedList(name: Name): List<Meta> = getIndexed(name).entries.sortedWith(
+ //sort by index
+ compareBy(space.kscience.dataforge.names.NameIndexComparator) { it.key }
+).map { it.value }
\ No newline at end of file
diff --git a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/names/NameToken.kt b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/names/NameToken.kt
index 0dc83c57..d6a760f1 100644
--- a/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/names/NameToken.kt
+++ b/dataforge-meta/src/commonMain/kotlin/space/kscience/dataforge/names/NameToken.kt
@@ -67,10 +67,29 @@ public class NameToken(public val body: String, public val index: String? = null
* Parse name token from a string
*/
public fun parse(string: String): NameToken {
- val body = string.substringBefore('[')
- val index = string.substringAfter('[', "")
- if (index.isNotEmpty() && index.endsWith(']')) error("NameToken with index must end with ']'")
- return NameToken(body, index.removeSuffix("]"))
+ var indexStart = -1
+ var indexEnd = -1
+ string.forEachIndexed { index, c ->
+ when (c) {
+ '[' -> when {
+ indexStart >= 0 -> error("Second opening bracket not allowed in NameToken: $string")
+ else -> indexStart = index
+ }
+
+ ']' -> when {
+ indexStart < 0 -> error("Closing index bracket could not be used before opening bracket in NameToken: $string")
+ indexEnd >= 0 -> error("Second closing bracket not allowed in NameToken: $string")
+ else -> indexEnd = index
+ }
+
+ else -> if (indexEnd >= 0) error("Symbols not allowed after index in NameToken: $string")
+ }
+ }
+ if (indexStart >= 0 && indexEnd < 0) error("Opening bracket without closing bracket not allowed in NameToken: $string")
+ return NameToken(
+ if (indexStart >= 0) string.substring(0, indexStart) else string,
+ if (indexStart >= 0) string.substring(indexStart + 1, indexEnd) else null
+ )
}
}
}
diff --git a/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/ConvertersTest.kt b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/ConvertersTest.kt
new file mode 100644
index 00000000..fda978e4
--- /dev/null
+++ b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/ConvertersTest.kt
@@ -0,0 +1,17 @@
+package space.kscience.dataforge.meta
+
+import kotlin.test.Test
+import kotlin.test.assertEquals
+
+class ConvertersTest {
+
+ @Test
+ fun stringListConversion() {
+ val list = listOf("A", "B", "C")
+ val meta = MetaConverter.stringList.convert(list)
+ val json = meta.toJson()
+ val reconstructedMeta = json.toMeta()
+ val reconstructed = MetaConverter.stringList.read(reconstructedMeta)
+ assertEquals(list,reconstructed)
+ }
+}
\ No newline at end of file
diff --git a/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/JsonMetaTest.kt b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/JsonMetaTest.kt
index fba9b596..2ada3ade 100644
--- a/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/JsonMetaTest.kt
+++ b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/JsonMetaTest.kt
@@ -2,7 +2,7 @@ package space.kscience.dataforge.meta
import kotlinx.serialization.json.*
import space.kscience.dataforge.meta.descriptors.MetaDescriptor
-import space.kscience.dataforge.meta.descriptors.item
+import space.kscience.dataforge.meta.descriptors.node
import kotlin.test.Test
import kotlin.test.assertEquals
@@ -32,7 +32,7 @@ class JsonMetaTest {
}
val descriptor = MetaDescriptor {
- item("nodeArray") {
+ node("nodeArray") {
indexKey = "index"
}
}
diff --git a/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/MetaDelegateTest.kt b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/MetaDelegateTest.kt
index 7a2dbc22..4b99cc3b 100644
--- a/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/MetaDelegateTest.kt
+++ b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/MetaDelegateTest.kt
@@ -20,7 +20,7 @@ class MetaDelegateTest {
var myValue by string()
var safeValue by double(2.2)
var enumValue by enum(TestEnum.YES)
- var inner by spec(InnerScheme)
+ var inner by scheme(InnerScheme)
companion object : SchemeSpec<TestScheme>(::TestScheme)
}
diff --git a/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/MetaRefTest.kt b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/MetaRefTest.kt
new file mode 100644
index 00000000..a93841d9
--- /dev/null
+++ b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/MetaRefTest.kt
@@ -0,0 +1,35 @@
+package space.kscience.dataforge.meta
+
+import kotlinx.serialization.Serializable
+import space.kscience.dataforge.misc.DFExperimental
+import kotlin.test.Test
+import kotlin.test.assertEquals
+
+@DFExperimental
+internal class MetaRefTest {
+
+ @Serializable
+ data class XY(val x: Double, val y: Double)
+
+ object TestMetaSpec : MetaSpec() {
+ val integer by int { description = "Integer value" }
+ val string by string { description = "String value" }
+ val custom by item(MetaConverter.serializable<XY>()) { description = "custom value" }
+ }
+
+ @Test
+ fun specWriteRead() = with(TestMetaSpec){
+ val meta = MutableMeta()
+
+ meta[integer] = 22
+ meta[string] = "33"
+ val xy = XY(33.0, -33.0)
+ meta[custom] = xy
+
+ val sealed = meta.seal()
+
+ assertEquals(22, sealed[integer])
+ assertEquals("33", sealed[string])
+ assertEquals(xy, sealed[custom])
+ }
+}
\ No newline at end of file
diff --git a/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/MetaTest.kt b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/MetaTest.kt
index 85db7bd6..78b5cdb4 100644
--- a/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/MetaTest.kt
+++ b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/MetaTest.kt
@@ -61,4 +61,33 @@ class MetaTest {
assertEquals(null, indexed["8"])
assertEquals(12, indexed["12"].int)
}
+
+ @Test
+ fun reset() {
+ val oldMeta = MutableMeta {
+ "a" put {
+ "value" put "aValue"
+ }
+ "b" put {
+ "value" put "bValue"
+ }
+ "c" put {
+ "value" put "cValue"
+ }
+ }
+ val newMeta = Meta {
+ "a" put {
+ "value" put "aValue"
+ }
+ "b" put {
+ "value" put "bValue"
+ }
+ "d" put {
+ "value" put "dValue"
+ }
+ }
+ oldMeta.reset(newMeta)
+ println(oldMeta)
+ assertEquals(setOf("a", "b", "d"), oldMeta.items.keys.map { it.toString() }.toSet())
+ }
}
\ No newline at end of file
diff --git a/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/MutableMetaViewTest.kt b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/MutableMetaViewTest.kt
new file mode 100644
index 00000000..61af1d04
--- /dev/null
+++ b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/MutableMetaViewTest.kt
@@ -0,0 +1,25 @@
+package space.kscience.dataforge.meta
+
+import space.kscience.dataforge.names.asName
+import kotlin.test.Test
+import kotlin.test.assertEquals
+import kotlin.test.assertTrue
+
+class MutableMetaViewTest {
+ @Test
+ fun metaView() {
+ val meta = MutableMeta()
+ val view = meta.view("a".asName())
+
+ view["b"] = Meta.EMPTY
+
+ assertTrue { meta.items.isEmpty() }
+
+ view["c"] = Meta {
+ "d" put 22
+ }
+
+ assertEquals(22, meta["a.c.d"].int)
+ }
+
+}
\ No newline at end of file
diff --git a/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/ObservableMetaTest.kt b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/ObservableMetaTest.kt
new file mode 100644
index 00000000..4681ec12
--- /dev/null
+++ b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/ObservableMetaTest.kt
@@ -0,0 +1,49 @@
+package space.kscience.dataforge.meta
+
+import space.kscience.dataforge.names.startsWith
+import kotlin.test.Ignore
+import kotlin.test.Test
+import kotlin.test.assertEquals
+
+class ObservableMetaTest {
+
+ @Test
+ fun asObservable() {
+ val meta = MutableMeta {
+ "data" put {
+ "x" put ListValue(1, 2, 3)
+ "y" put ListValue(5, 6, 7)
+ "type" put "scatter"
+ }
+ }.asObservable()
+
+ assertEquals("scatter", meta["data.type"].string)
+ }
+
+ @Test
+ @Ignore
+ fun detachNode() {
+ val meta = MutableMeta {
+ "data" put {
+ "x" put ListValue(1, 2, 3)
+ "y" put ListValue(5, 6, 7)
+ "type" put "scatter"
+ }
+ }.asObservable()
+
+ var collector: Value? = null
+
+ meta.onChange(null) { name ->
+ if (name.startsWith("data")) {
+ collector = get("data.z")?.value
+ }
+ }
+
+ val data = meta["data"]!!
+
+ meta.remove("data")
+
+ data["z"] = ListValue(2, 5, 7)
+ assertEquals(null, collector)
+ }
+}
\ No newline at end of file
diff --git a/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/SchemeTest.kt b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/SchemeTest.kt
deleted file mode 100644
index bb2736ce..00000000
--- a/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/SchemeTest.kt
+++ /dev/null
@@ -1,49 +0,0 @@
-package space.kscience.dataforge.meta
-
-import space.kscience.dataforge.misc.DFExperimental
-import kotlin.test.Test
-import kotlin.test.assertEquals
-import kotlin.test.assertNotNull
-
-@DFExperimental
-class SchemeTest {
- @Test
- fun testSchemeWrappingBeforeEdit() {
- val config = MutableMeta()
- val scheme = TestScheme.write(config)
- scheme.a = 29
- assertEquals(29, config["a"].int)
- }
-
- @Test
- fun testSchemeWrappingAfterEdit() {
- val scheme = TestScheme.empty()
- scheme.a = 29
- val config = MutableMeta()
- scheme.retarget(config)
- assertEquals(29, scheme.a)
- }
-
- @Test
- fun testSchemeSubscription() {
- val scheme = TestScheme.empty()
- var flag: Int? = null
- scheme.useProperty(TestScheme::a) { a ->
- flag = a
- }
- scheme.a = 2
- assertEquals(2, flag)
- }
-
- @Test
- fun testListSubscription(){
- val scheme = TestScheme.empty()
- var value: Value? = null
- scheme.v = ListValue(0.0,0.0,0.0)
- scheme.useProperty(TestScheme::v){
- value = it
- }
- scheme.v = ListValue(1.0, 2.0, 3.0)
- assertNotNull(value)
- }
-}
\ No newline at end of file
diff --git a/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/SpecificationTest.kt b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/SpecificationTest.kt
index 8d4d3537..dc9b9d64 100644
--- a/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/SpecificationTest.kt
+++ b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/SpecificationTest.kt
@@ -1,7 +1,17 @@
package space.kscience.dataforge.meta
+import space.kscience.dataforge.misc.DFExperimental
import kotlin.test.Test
import kotlin.test.assertEquals
+import kotlin.test.assertNotNull
+
+
+internal class SubScheme : Scheme() {
+
+ var subValue by string()
+
+ companion object : SchemeSpec<SubScheme>(::SubScheme)
+}
internal class TestScheme : Scheme() {
var list by numberList(1, 2, 3)
@@ -11,9 +21,23 @@ internal class TestScheme : Scheme() {
var v by value()
+ var sub by scheme(SubScheme)
+
companion object : SchemeSpec<TestScheme>(::TestScheme)
}
+private class SchemeWithInit: Scheme(){
+ init {
+ set("initial", "initialValue")
+ }
+
+ var initial by string()
+
+ companion object: SchemeSpec<SchemeWithInit>(::SchemeWithInit)
+}
+
+
+
class SpecificationTest {
// @Test
@@ -71,4 +95,64 @@ class SpecificationTest {
assertEquals(22, config["child.a"].int)
assertEquals("test", config["child.b"].string)
}
+
+ @Test
+ fun testSchemeWrappingBeforeEdit() {
+ val config = MutableMeta()
+ val scheme = TestScheme.write(config)
+ scheme.a = 29
+ assertEquals(29, config["a"].int)
+ }
+
+ @OptIn(DFExperimental::class)
+ @Test
+ fun testSchemeWrappingAfterEdit() {
+ val scheme = TestScheme.empty()
+ scheme.a = 29
+ val config = MutableMeta()
+ scheme.retarget(config)
+ assertEquals(29, scheme.a)
+ }
+
+ @Test
+ fun testSchemeSubscription() {
+ val scheme = TestScheme.empty()
+ var flag: Int? = null
+ scheme.useProperty(TestScheme::a) { a ->
+ flag = a
+ }
+ scheme.a = 2
+ assertEquals(2, flag)
+ }
+
+ @Test
+ fun testListSubscription(){
+ val scheme = TestScheme.empty()
+ var value: Value? = null
+ scheme.v = ListValue(0.0,0.0,0.0)
+ scheme.useProperty(TestScheme::v){
+ value = it
+ }
+ scheme.v = ListValue(1.0, 2.0, 3.0)
+ assertNotNull(value)
+ }
+
+ @Test
+ fun testSubScheme(){
+ val scheme = TestScheme.empty()
+
+ scheme.sub.subValue = "aaa"
+
+ assertEquals("aaa",scheme.sub.subValue)
+ }
+
+
+ @Test
+ fun testSchemeWithInit(){
+ val scheme = SchemeWithInit()
+ assertEquals("initialValue", scheme.initial)
+ scheme.initial = "none"
+ assertEquals("none", scheme.initial)
+ }
+
}
\ No newline at end of file
diff --git a/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/descriptors/DescriptorTest.kt b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/descriptors/DescriptorTest.kt
index e8c321fc..1a08ce34 100644
--- a/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/descriptors/DescriptorTest.kt
+++ b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/meta/descriptors/DescriptorTest.kt
@@ -11,14 +11,14 @@ class DescriptorTest {
val descriptor = MetaDescriptor {
node("aNode") {
- info = "A root demo node"
+ description = "A root demo node"
value("b", ValueType.NUMBER) {
- info = "b number value"
+ description = "b number value"
}
node("otherNode") {
value("otherValue", ValueType.BOOLEAN) {
default(false)
- info = "default value"
+ description = "default value"
}
}
}
diff --git a/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/names/NameTest.kt b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/names/NameTest.kt
index a5bdf3fc..25725333 100644
--- a/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/names/NameTest.kt
+++ b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/names/NameTest.kt
@@ -1,9 +1,6 @@
package space.kscience.dataforge.names
-import kotlin.test.Test
-import kotlin.test.assertEquals
-import kotlin.test.assertFalse
-import kotlin.test.assertTrue
+import kotlin.test.*
class NameTest {
@Test
@@ -50,4 +47,31 @@ class NameTest {
val name = Name.parse("a.b.c")
assertEquals("a.b".parseAsName(), name.cutLast())
}
+
+ @Test
+ fun tokenParseTest(){
+ val token1 = NameToken.parse("token[index]")
+ assertEquals("token", token1.body)
+ assertEquals("index", token1.index)
+
+ val token2 = NameToken.parse("token-body")
+ assertEquals("token-body", token2.body)
+ assertEquals(null, token2.index)
+
+// val token3 = NameToken.parse("[token-index]")
+// assertEquals("", token3.body)
+// assertEquals("token-index", token3.index)
+
+ assertFails{
+ NameToken.parse("[token-index]")
+ }
+
+ assertFails {
+ NameToken.parse("token[22")
+ }
+
+ assertFails {
+ NameToken.parse("token[22]ddd")
+ }
+ }
}
\ No newline at end of file
diff --git a/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/values/DoubleArrayValue.kt b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/values/DoubleArrayValue.kt
new file mode 100644
index 00000000..30bc2adc
--- /dev/null
+++ b/dataforge-meta/src/commonTest/kotlin/space/kscience/dataforge/values/DoubleArrayValue.kt
@@ -0,0 +1,24 @@
+package space.kscience.dataforge.values
+
+import space.kscience.dataforge.meta.DoubleArrayValue
+import space.kscience.dataforge.meta.Meta
+import space.kscience.dataforge.meta.doubleArray
+import space.kscience.dataforge.meta.get
+import kotlin.test.Test
+import kotlin.test.assertEquals
+import kotlin.test.assertTrue
+
+class DoubleArrayValue {
+ @Test
+ fun doubleArrayWriteRead(){
+ val meta = Meta{
+ "doubleArray" put doubleArrayOf(1.0,2.0,3.0)
+ }
+
+ assertTrue {
+ meta["doubleArray"]?.value is DoubleArrayValue
+ }
+
+ assertEquals(2.0, meta["doubleArray"].doubleArray?.get(1))
+ }
+}
\ No newline at end of file
diff --git a/dataforge-meta/src/jsMain/kotlin/space/kscience/dataforge/meta/DynamicMeta.kt b/dataforge-meta/src/jsMain/kotlin/space/kscience/dataforge/meta/DynamicMeta.kt
index b38d5891..57d324c7 100644
--- a/dataforge-meta/src/jsMain/kotlin/space/kscience/dataforge/meta/DynamicMeta.kt
+++ b/dataforge-meta/src/jsMain/kotlin/space/kscience/dataforge/meta/DynamicMeta.kt
@@ -31,6 +31,9 @@ public fun Meta.toDynamic(): dynamic {
}
public class DynamicMeta(internal val obj: dynamic) : TypedMeta<DynamicMeta> {
+
+ override val self: DynamicMeta get() = this
+
private fun keys(): Array<String> = js("Object").keys(obj) as Array<String>
private fun isArray(obj: dynamic): Boolean =
diff --git a/dataforge-meta/src/jsMain/kotlin/space/kscience/dataforge/misc/castJs.kt b/dataforge-meta/src/jsMain/kotlin/space/kscience/dataforge/misc/castJs.kt
deleted file mode 100644
index b404ebb4..00000000
--- a/dataforge-meta/src/jsMain/kotlin/space/kscience/dataforge/misc/castJs.kt
+++ /dev/null
@@ -1,5 +0,0 @@
-package space.kscience.dataforge.misc
-import kotlin.js.unsafeCast as unsafeCastJs
-
-@Suppress("UNCHECKED_CAST", "NOTHING_TO_INLINE")
-public actual inline fun <T> Any?.unsafeCast(): T = this.unsafeCastJs<T>()
\ No newline at end of file
diff --git a/dataforge-meta/src/jvmMain/kotlin/space/kscience/dataforge/misc/castJvm.kt b/dataforge-meta/src/jvmMain/kotlin/space/kscience/dataforge/misc/castJvm.kt
deleted file mode 100644
index 27d399fe..00000000
--- a/dataforge-meta/src/jvmMain/kotlin/space/kscience/dataforge/misc/castJvm.kt
+++ /dev/null
@@ -1,4 +0,0 @@
-package space.kscience.dataforge.misc
-
-@Suppress("UNCHECKED_CAST", "NOTHING_TO_INLINE")
-public actual inline fun <T> Any?.unsafeCast(): T = this as T
\ No newline at end of file
diff --git a/dataforge-meta/src/nativeMain/kotlin/space/kscience/dataforge/misc/castNative.kt b/dataforge-meta/src/nativeMain/kotlin/space/kscience/dataforge/misc/castNative.kt
deleted file mode 100644
index 27d399fe..00000000
--- a/dataforge-meta/src/nativeMain/kotlin/space/kscience/dataforge/misc/castNative.kt
+++ /dev/null
@@ -1,4 +0,0 @@
-package space.kscience.dataforge.misc
-
-@Suppress("UNCHECKED_CAST", "NOTHING_TO_INLINE")
-public actual inline fun <T> Any?.unsafeCast(): T = this as T
\ No newline at end of file
diff --git a/dataforge-output/api/dataforge-output.api b/dataforge-output/api/dataforge-output.api
deleted file mode 100644
index f991b754..00000000
--- a/dataforge-output/api/dataforge-output.api
+++ /dev/null
@@ -1,68 +0,0 @@
-public final class hep/dataforge/output/ConsoleOutputManager : hep/dataforge/context/AbstractPlugin, hep/dataforge/output/OutputManager {
- public static final field Companion Lhep/dataforge/output/ConsoleOutputManager$Companion;
- public fun <init> ()V
- public fun get (Lkotlin/reflect/KClass;Lhep/dataforge/names/Name;Lhep/dataforge/names/Name;Lhep/dataforge/meta/Meta;)Lhep/dataforge/output/Renderer;
- public fun getTag ()Lhep/dataforge/context/PluginTag;
-}
-
-public final class hep/dataforge/output/ConsoleOutputManager$Companion : hep/dataforge/context/PluginFactory {
- public fun getTag ()Lhep/dataforge/context/PluginTag;
- public fun getType ()Lkotlin/reflect/KClass;
- public fun invoke (Lhep/dataforge/meta/Meta;Lhep/dataforge/context/Context;)Lhep/dataforge/output/ConsoleOutputManager;
- public synthetic fun invoke (Lhep/dataforge/meta/Meta;Lhep/dataforge/context/Context;)Ljava/lang/Object;
-}
-
-public final class hep/dataforge/output/DefaultTextFormat : hep/dataforge/output/TextFormat {
- public static final field INSTANCE Lhep/dataforge/output/DefaultTextFormat;
- public fun getPriority ()I
- public fun getType ()Lkotlin/reflect/KClass;
- public fun render (Ljava/lang/Appendable;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object;
-}
-
-public final class hep/dataforge/output/OutputJVMKt {
- public static final fun getOutput (Lkotlinx/coroutines/Dispatchers;)Lkotlinx/coroutines/CoroutineDispatcher;
-}
-
-public abstract interface class hep/dataforge/output/OutputManager {
- public abstract fun get (Lkotlin/reflect/KClass;Lhep/dataforge/names/Name;Lhep/dataforge/names/Name;Lhep/dataforge/meta/Meta;)Lhep/dataforge/output/Renderer;
-}
-
-public final class hep/dataforge/output/OutputManager$DefaultImpls {
- public static synthetic fun get$default (Lhep/dataforge/output/OutputManager;Lkotlin/reflect/KClass;Lhep/dataforge/names/Name;Lhep/dataforge/names/Name;Lhep/dataforge/meta/Meta;ILjava/lang/Object;)Lhep/dataforge/output/Renderer;
-}
-
-public final class hep/dataforge/output/OutputManagerKt {
- public static final fun getCONSOLE_RENDERER ()Lhep/dataforge/output/Renderer;
- public static final fun getOutput (Lhep/dataforge/context/Context;)Lhep/dataforge/output/OutputManager;
- public static final fun render (Lhep/dataforge/output/OutputManager;Ljava/lang/Object;Lhep/dataforge/names/Name;Lhep/dataforge/names/Name;Lhep/dataforge/meta/Meta;)V
- public static synthetic fun render$default (Lhep/dataforge/output/OutputManager;Ljava/lang/Object;Lhep/dataforge/names/Name;Lhep/dataforge/names/Name;Lhep/dataforge/meta/Meta;ILjava/lang/Object;)V
-}
-
-public abstract interface class hep/dataforge/output/Renderer : hep/dataforge/context/ContextAware {
- public abstract fun render (Ljava/lang/Object;Lhep/dataforge/meta/Meta;)V
-}
-
-public final class hep/dataforge/output/Renderer$DefaultImpls {
- public static fun getLogger (Lhep/dataforge/output/Renderer;)Lmu/KLogger;
- public static synthetic fun render$default (Lhep/dataforge/output/Renderer;Ljava/lang/Object;Lhep/dataforge/meta/Meta;ILjava/lang/Object;)V
-}
-
-public abstract interface class hep/dataforge/output/TextFormat {
- public static final field Companion Lhep/dataforge/output/TextFormat$Companion;
- public static final field TEXT_RENDERER_TYPE Ljava/lang/String;
- public abstract fun getPriority ()I
- public abstract fun getType ()Lkotlin/reflect/KClass;
- public abstract fun render (Ljava/lang/Appendable;Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object;
-}
-
-public final class hep/dataforge/output/TextFormat$Companion {
- public static final field TEXT_RENDERER_TYPE Ljava/lang/String;
-}
-
-public final class hep/dataforge/output/TextRenderer : hep/dataforge/output/Renderer {
- public fun <init> (Lhep/dataforge/context/Context;Ljava/lang/Appendable;)V
- public fun getContext ()Lhep/dataforge/context/Context;
- public fun getLogger ()Lmu/KLogger;
- public fun render (Ljava/lang/Object;Lhep/dataforge/meta/Meta;)V
-}
-
diff --git a/dataforge-output/build.gradle.kts b/dataforge-output/build.gradle.kts
deleted file mode 100644
index a858ff93..00000000
--- a/dataforge-output/build.gradle.kts
+++ /dev/null
@@ -1,15 +0,0 @@
-plugins {
- id("space.kscience.gradle.mpp")
- id("space.kscience.gradle.native")
-}
-
-kotlin {
- sourceSets {
- val commonMain by getting{
- dependencies {
- api(project(":dataforge-context"))
- //api(project(":dataforge-io"))
- }
- }
- }
-}
\ No newline at end of file
diff --git a/dataforge-output/src/commonMain/kotlin/hep/dataforge/output/OutputManager.kt b/dataforge-output/src/commonMain/kotlin/hep/dataforge/output/OutputManager.kt
deleted file mode 100644
index c8580403..00000000
--- a/dataforge-output/src/commonMain/kotlin/hep/dataforge/output/OutputManager.kt
+++ /dev/null
@@ -1,75 +0,0 @@
-package space.kscience.dataforge.output
-
-import space.kscience.dataforge.context.*
-import space.kscience.dataforge.context.PluginTag.Companion.DATAFORGE_GROUP
-import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.names.Name
-import kotlinx.coroutines.CoroutineDispatcher
-import kotlinx.coroutines.Dispatchers
-import kotlin.reflect.KClass
-
-/**
- * A manager for outputs
- */
-public interface OutputManager {
-
- /**
- * Get an output specialized for given type, name and stage.
- * @param stage represents the node or directory for the output. Empty means root node.
- * @param name represents the name inside the node.
- * @param meta configuration for [Renderer] (not for rendered object)
- */
- public fun <T : Any> getOutputContainer(
- type: KClass<out T>,
- name: Name,
- stage: Name = Name.EMPTY,
- meta: Meta = Meta.EMPTY
- ): Renderer<T>
-}
-
-/**
- * Get an output manager for a context
- */
-public val Context.output: OutputManager get() = plugins.get() ?: ConsoleOutputManager()
-
-/**
- * Get an output with given [name], [stage] and reified content type
- */
-public inline fun <reified T : Any> OutputManager.getOutputContainer(
- name: Name,
- stage: Name = Name.EMPTY,
- meta: Meta = Meta.EMPTY
-): Renderer<T> {
- return getOutputContainer(T::class, name, stage, meta)
-}
-
-/**
- * Directly render an object using the most suitable renderer
- */
-public fun OutputManager.render(obj: Any, name: Name, stage: Name = Name.EMPTY, meta: Meta = Meta.EMPTY): Unit =
- getOutputContainer(obj::class, name, stage).render(obj, meta)
-
-/**
- * System console output.
- * The [CONSOLE_RENDERER] is used when no other [OutputManager] is provided.
- */
-public val CONSOLE_RENDERER: Renderer<Any> = Renderer { obj, meta -> println(obj) }
-
-public class ConsoleOutputManager : AbstractPlugin(), OutputManager {
- override val tag: PluginTag get() = ConsoleOutputManager.tag
-
- override fun <T : Any> getOutputContainer(type: KClass<out T>, name: Name, stage: Name, meta: Meta): Renderer<T> = CONSOLE_RENDERER
-
- public companion object : PluginFactory<ConsoleOutputManager> {
- override val tag: PluginTag = PluginTag("output.console", group = DATAFORGE_GROUP)
-
- override val type: KClass<ConsoleOutputManager> = ConsoleOutputManager::class
-
- override fun invoke(meta: Meta, context: Context): ConsoleOutputManager = ConsoleOutputManager()
- }
-}
-
-/**
- * A dispatcher for output tasks.
- */
-public expect val Dispatchers.Output: CoroutineDispatcher
\ No newline at end of file
diff --git a/dataforge-output/src/commonMain/kotlin/hep/dataforge/output/Renderer.kt b/dataforge-output/src/commonMain/kotlin/hep/dataforge/output/Renderer.kt
deleted file mode 100644
index f6caaeb3..00000000
--- a/dataforge-output/src/commonMain/kotlin/hep/dataforge/output/Renderer.kt
+++ /dev/null
@@ -1,21 +0,0 @@
-package space.kscience.dataforge.output
-
-import space.kscience.dataforge.context.ContextAware
-import space.kscience.dataforge.meta.Meta
-
-/**
- * A generic way to render any object in the output.
- *
- * An object could be rendered either in append or overlay mode. The mode is decided by the [Renderer]
- * based on its configuration and provided meta
- *
- */
-public fun interface Renderer<in T : Any> {
- /**
- * Render specific object with configuration.
- *
- * By convention actual render is called in asynchronous mode, so this method should never
- * block execution
- */
- public fun render(obj: T, meta: Meta)
-}
diff --git a/dataforge-output/src/commonMain/kotlin/hep/dataforge/output/TextRenderer.kt b/dataforge-output/src/commonMain/kotlin/hep/dataforge/output/TextRenderer.kt
deleted file mode 100644
index 8b33241b..00000000
--- a/dataforge-output/src/commonMain/kotlin/hep/dataforge/output/TextRenderer.kt
+++ /dev/null
@@ -1,78 +0,0 @@
-package space.kscience.dataforge.output
-
-import space.kscience.dataforge.context.Context
-import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.output.TextFormat.Companion.TEXT_RENDERER_TYPE
-import space.kscience.dataforge.provider.Type
-import space.kscience.dataforge.provider.top
-import kotlinx.coroutines.Dispatchers
-import kotlinx.coroutines.launch
-import kotlin.reflect.KClass
-import kotlin.reflect.KType
-
-
-/**
- * A text or binary renderer based on [Output]
- */
-@Type(TEXT_RENDERER_TYPE)
-@Deprecated("Bad design")
-public interface TextFormat {
- /**
- * The priority of this renderer compared to other renderers
- */
- public val priority: Int
- /**
- * The type of the content served by this renderer
- */
- public val type: KClass<*>
-
- public suspend fun Appendable.render(obj: Any)
-
- public companion object {
- public const val TEXT_RENDERER_TYPE: String = "dataforge.textRenderer"
- }
-}
-
-@Deprecated("Bad design")
-public object DefaultTextFormat : TextFormat {
- override val priority: Int = Int.MAX_VALUE
- override val type: KClass<*> = Any::class
-
- override suspend fun Appendable.render(obj: Any) {
- append(obj.toString() + "\n")
- }
-}
-
-/**
- * A text-based renderer
- */
-@Deprecated("Bad design")
-public class TextRenderer(override val context: Context, private val output: Appendable) : Renderer<Any> {
- private val cache = HashMap<KClass<*>, TextFormat>()
-
- /**
- * Find the first [TextFormat] matching the given object type.
- */
- override fun render(obj: Any, meta: Meta) {
- val format: TextFormat = if (obj is CharSequence) {
- DefaultTextFormat
- } else {
- val value = cache[obj::class]
- if (value == null) {
- val answer =
- context.top<TextFormat>(TEXT_RENDERER_TYPE).values.firstOrNull { it.type.isInstance(obj) }
- if (answer != null) {
- cache[obj::class] = answer
- answer
- } else {
- DefaultTextFormat
- }
- } else {
- value
- }
- }
- context.launch(Dispatchers.Output) {
- format.run { output.render(obj) }
- }
- }
-}
\ No newline at end of file
diff --git a/dataforge-output/src/jsMain/kotlin/hep/dataforge/output/outputJS.kt b/dataforge-output/src/jsMain/kotlin/hep/dataforge/output/outputJS.kt
deleted file mode 100644
index 453d7351..00000000
--- a/dataforge-output/src/jsMain/kotlin/hep/dataforge/output/outputJS.kt
+++ /dev/null
@@ -1,7 +0,0 @@
-package space.kscience.dataforge.output
-
-import kotlinx.coroutines.CoroutineDispatcher
-import kotlinx.coroutines.Dispatchers
-
-
-public actual val Dispatchers.Output: CoroutineDispatcher get() = Default
\ No newline at end of file
diff --git a/dataforge-output/src/jvmMain/kotlin/hep/dataforge/output/outputJVM.kt b/dataforge-output/src/jvmMain/kotlin/hep/dataforge/output/outputJVM.kt
deleted file mode 100644
index d9ba0b2f..00000000
--- a/dataforge-output/src/jvmMain/kotlin/hep/dataforge/output/outputJVM.kt
+++ /dev/null
@@ -1,6 +0,0 @@
-package space.kscience.dataforge.output
-
-import kotlinx.coroutines.CoroutineDispatcher
-import kotlinx.coroutines.Dispatchers
-
-public actual val Dispatchers.Output: CoroutineDispatcher get() = IO
\ No newline at end of file
diff --git a/dataforge-output/src/nativeMain/kotlin/hep/dataforge/output/outputNative.kt b/dataforge-output/src/nativeMain/kotlin/hep/dataforge/output/outputNative.kt
deleted file mode 100644
index 2d59ae4e..00000000
--- a/dataforge-output/src/nativeMain/kotlin/hep/dataforge/output/outputNative.kt
+++ /dev/null
@@ -1,6 +0,0 @@
-package space.kscience.dataforge.output
-
-import kotlinx.coroutines.CoroutineDispatcher
-import kotlinx.coroutines.Dispatchers
-
-public actual val Dispatchers.Output: CoroutineDispatcher get() = Dispatchers.Default
\ No newline at end of file
diff --git a/dataforge-scripting/README.md b/dataforge-scripting/README.md
index af79cc8f..76b36b5b 100644
--- a/dataforge-scripting/README.md
+++ b/dataforge-scripting/README.md
@@ -1,23 +1,21 @@
# Module dataforge-scripting
-
+Scripting definition fow workspace generation
## Usage
## Artifact:
-The Maven coordinates of this project are `space.kscience:dataforge-scripting:0.7.0`.
+The Maven coordinates of this project are `space.kscience:dataforge-scripting:0.10.0`.
**Gradle Kotlin DSL:**
```kotlin
repositories {
maven("https://repo.kotlin.link")
- //uncomment to access development builds
- //maven("https://maven.pkg.jetbrains.space/spc/p/sci/dev")
mavenCentral()
}
dependencies {
- implementation("space.kscience:dataforge-scripting:0.7.0")
+ implementation("space.kscience:dataforge-scripting:0.10.0")
}
```
diff --git a/dataforge-scripting/build.gradle.kts b/dataforge-scripting/build.gradle.kts
index be81fe70..37bc2e11 100644
--- a/dataforge-scripting/build.gradle.kts
+++ b/dataforge-scripting/build.gradle.kts
@@ -2,22 +2,24 @@ plugins {
id("space.kscience.gradle.mpp")
}
-kscience{
+description = "Scripting definition fow workspace generation"
+
+kscience {
jvm()
- dependencies {
+ commonMain {
api(projects.dataforgeWorkspace)
implementation(kotlin("scripting-common"))
}
- dependencies(jvmMain){
+ jvmMain {
implementation(kotlin("scripting-jvm-host"))
implementation(kotlin("scripting-jvm"))
}
- dependencies(jvmTest){
+ jvmTest {
implementation(spclibs.logback.classic)
}
}
-readme{
+readme {
maturity = space.kscience.gradle.Maturity.PROTOTYPE
}
\ No newline at end of file
diff --git a/dataforge-workspace/README.md b/dataforge-workspace/README.md
index c096699f..8bb476a3 100644
--- a/dataforge-workspace/README.md
+++ b/dataforge-workspace/README.md
@@ -6,18 +6,16 @@
## Artifact:
-The Maven coordinates of this project are `space.kscience:dataforge-workspace:0.7.0`.
+The Maven coordinates of this project are `space.kscience:dataforge-workspace:0.10.0`.
**Gradle Kotlin DSL:**
```kotlin
repositories {
maven("https://repo.kotlin.link")
- //uncomment to access development builds
- //maven("https://maven.pkg.jetbrains.space/spc/p/sci/dev")
mavenCentral()
}
dependencies {
- implementation("space.kscience:dataforge-workspace:0.7.0")
+ implementation("space.kscience:dataforge-workspace:0.10.0")
}
```
diff --git a/dataforge-workspace/build.gradle.kts b/dataforge-workspace/build.gradle.kts
index ec117865..8254ef7d 100644
--- a/dataforge-workspace/build.gradle.kts
+++ b/dataforge-workspace/build.gradle.kts
@@ -2,29 +2,29 @@ plugins {
id("space.kscience.gradle.mpp")
}
-kscience{
+description = "A framework for pull-based data processing"
+
+kscience {
jvm()
js()
native()
+ wasm()
useCoroutines()
- useSerialization{
+ useSerialization {
protobuf()
}
- commonMain{
- dependencies {
- api(projects.dataforgeContext)
- api(projects.dataforgeData)
- api(projects.dataforgeIo)
- }
+ commonMain {
+ api(projects.dataforgeContext)
+ api(projects.dataforgeData)
+ api(projects.dataforgeIo)
+
}
- jvmTest{
- dependencies {
- implementation(spclibs.logback.classic)
- implementation(projects.dataforgeIo.dataforgeIoYaml)
- }
+ jvmTest {
+ implementation(spclibs.logback.classic)
+ implementation(projects.dataforgeIo.dataforgeIoYaml)
}
}
-readme{
+readme {
maturity = space.kscience.gradle.Maturity.EXPERIMENTAL
}
\ No newline at end of file
diff --git a/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/EnvelopeTask.kt b/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/EnvelopeTask.kt
deleted file mode 100644
index a1588a54..00000000
--- a/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/EnvelopeTask.kt
+++ /dev/null
@@ -1,46 +0,0 @@
-package space.kscience.dataforge.workspace
-
-import space.kscience.dataforge.data.DataTree.Companion.META_ITEM_NAME_TOKEN
-import space.kscience.dataforge.io.Envelope
-import space.kscience.dataforge.io.IOReader
-import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.meta.descriptors.MetaDescriptor
-import space.kscience.dataforge.names.Name
-import space.kscience.dataforge.names.asName
-import kotlin.reflect.KType
-
-public abstract class EnvelopeTask<T : Any>(
- override val descriptor: MetaDescriptor?,
- private val reader: IOReader<T>,
-) : Task<T> {
-
- public abstract suspend fun produceEnvelopes(
- workspace: Workspace,
- taskName: Name,
- taskMeta: Meta,
- ): Map<Name, Envelope>
-
- override suspend fun execute(workspace: Workspace, taskName: Name, taskMeta: Meta): TaskResult<T> =
- Result(workspace, taskName, taskMeta, reader, produceEnvelopes(workspace, taskName, taskMeta))
-
- private class Result<T : Any>(
- override val workspace: Workspace,
- override val taskName: Name,
- override val taskMeta: Meta,
- val reader: IOReader<T>,
- envelopes: Map<Name, Envelope>,
- ) : TaskResult<T> {
-
- private val dataMap = envelopes.mapValues {
- workspace.wrapData(it.value.toData(reader), it.key, taskName, taskMeta)
- }
- override val meta: Meta get() = dataMap[META_ITEM_NAME_TOKEN.asName()]?.meta ?: Meta.EMPTY
-
- override val dataType: KType get() = reader.type
-
- override fun iterator(): Iterator<TaskData<T>> = dataMap.values.iterator()
-
- override fun get(name: Name): TaskData<T>? = dataMap[name]
- }
-}
-
diff --git a/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/Task.kt b/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/Task.kt
index a1ef7be2..5e0ff572 100644
--- a/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/Task.kt
+++ b/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/Task.kt
@@ -1,15 +1,16 @@
package space.kscience.dataforge.workspace
import kotlinx.coroutines.withContext
-import space.kscience.dataforge.data.DataSetBuilder
+import space.kscience.dataforge.data.DataBuilderScope
import space.kscience.dataforge.data.DataTree
import space.kscience.dataforge.data.GoalExecutionRestriction
import space.kscience.dataforge.meta.Meta
+import space.kscience.dataforge.meta.MetaReader
import space.kscience.dataforge.meta.MetaRepr
-import space.kscience.dataforge.meta.Specification
import space.kscience.dataforge.meta.descriptors.Described
import space.kscience.dataforge.meta.descriptors.MetaDescriptor
-import space.kscience.dataforge.misc.DfId
+import space.kscience.dataforge.misc.DfType
+import space.kscience.dataforge.misc.UnsafeKType
import space.kscience.dataforge.names.Name
import space.kscience.dataforge.workspace.Task.Companion.TYPE
import kotlin.reflect.KType
@@ -19,8 +20,8 @@ import kotlin.reflect.typeOf
* A configurable task that could be executed on a workspace. The [TaskResult] represents a lazy result of the task.
* In general no computations should be made until the result is called.
*/
-@DfId(TYPE)
-public interface Task<out T : Any> : Described {
+@DfType(TYPE)
+public interface Task<T> : Described {
/**
* A task identification string used to compare tasks and check task body for change
@@ -28,10 +29,10 @@ public interface Task<out T : Any> : Described {
public val fingerprint: String get() = hashCode().toString(radix = 16)
/**
- * Compute a [TaskResult] using given meta. In general, the result is lazy and represents both computation model
- * and a handler for actual result
+ * Compute a [TaskResult] using given meta. In general, the result is lazy and represents both the computation model
+ * and a handler for the actual result
*
- * @param workspace a workspace to run task in
+ * @param workspace a workspace to run the task in
* @param taskName the name of the task in this workspace
* @param taskMeta configuration for current stage computation
*/
@@ -43,10 +44,10 @@ public interface Task<out T : Any> : Described {
}
/**
- * A [Task] with [Specification] for wrapping and unwrapping task configuration
+ * A [Task] with [MetaReader] for wrapping and unwrapping task configuration
*/
-public interface TaskWithSpec<out T : Any, C : Any> : Task<T> {
- public val spec: Specification<C>
+public interface TaskWithSpec<T, C : Any> : Task<T> {
+ public val spec: MetaReader<C>
override val descriptor: MetaDescriptor? get() = spec.descriptor
public suspend fun execute(workspace: Workspace, taskName: Name, configuration: C): TaskResult<T>
@@ -55,18 +56,18 @@ public interface TaskWithSpec<out T : Any, C : Any> : Task<T> {
execute(workspace, taskName, spec.read(taskMeta))
}
-public suspend fun <T : Any, C : Any> TaskWithSpec<T, C>.execute(
- workspace: Workspace,
- taskName: Name,
- block: C.() -> Unit = {},
-): TaskResult<T> = execute(workspace, taskName, spec(block))
+//public suspend fun <T : Any, C : Scheme> TaskWithSpec<T, C>.execute(
+// workspace: Workspace,
+// taskName: Name,
+// block: C.() -> Unit = {},
+//): TaskResult<T> = execute(workspace, taskName, spec(block))
-public class TaskResultBuilder<in T : Any>(
+public class TaskResultScope<in T>(
+ public val resultType: KType,
public val workspace: Workspace,
public val taskName: Name,
public val taskMeta: Meta,
- private val dataDrop: DataSetBuilder<T>,
-) : DataSetBuilder<T> by dataDrop
+) : DataBuilderScope<T>
/**
* Create a [Task] that composes a result using [builder]. Only data from the workspace could be used.
@@ -76,11 +77,11 @@ public class TaskResultBuilder<in T : Any>(
* @param descriptor of meta accepted by this task
* @param builder for resulting data set
*/
-@Suppress("FunctionName")
+@UnsafeKType
public fun <T : Any> Task(
resultType: KType,
descriptor: MetaDescriptor? = null,
- builder: suspend TaskResultBuilder<T>.() -> Unit,
+ builder: suspend TaskResultScope<T>.() -> DataTree<T>,
): Task<T> = object : Task<T> {
override val descriptor: MetaDescriptor? = descriptor
@@ -91,17 +92,17 @@ public fun <T : Any> Task(
taskMeta: Meta,
): TaskResult<T> = withContext(GoalExecutionRestriction() + workspace.goalLogger) {
//TODO use safe builder and check for external data on add and detects cycles
- val dataset = DataTree<T>(resultType) {
- TaskResultBuilder(workspace, taskName, taskMeta, this).apply { builder() }
- }
+ val dataset = TaskResultScope<T>(resultType, workspace, taskName, taskMeta).builder()
+
+
workspace.wrapResult(dataset, taskName, taskMeta)
}
}
-@Suppress("FunctionName")
+@OptIn(UnsafeKType::class)
public inline fun <reified T : Any> Task(
descriptor: MetaDescriptor? = null,
- noinline builder: suspend TaskResultBuilder<T>.() -> Unit,
+ noinline builder: suspend TaskResultScope<T>.() -> DataTree<T>,
): Task<T> = Task(typeOf<T>(), descriptor, builder)
@@ -116,10 +117,10 @@ public inline fun <reified T : Any> Task(
@Suppress("FunctionName")
public fun <T : Any, C : MetaRepr> Task(
resultType: KType,
- specification: Specification<C>,
- builder: suspend TaskResultBuilder<T>.(C) -> Unit,
+ specification: MetaReader<C>,
+ builder: suspend TaskResultScope<T>.(C) -> DataTree<T>,
): TaskWithSpec<T, C> = object : TaskWithSpec<T, C> {
- override val spec: Specification<C> = specification
+ override val spec: MetaReader<C> = specification
override suspend fun execute(
workspace: Workspace,
@@ -128,15 +129,15 @@ public fun <T : Any, C : MetaRepr> Task(
): TaskResult<T> = withContext(GoalExecutionRestriction() + workspace.goalLogger) {
//TODO use safe builder and check for external data on add and detects cycles
val taskMeta = configuration.toMeta()
- val dataset = DataTree<T>(resultType) {
- TaskResultBuilder(workspace, taskName, taskMeta, this).apply { builder(configuration) }
- }
+
+ @OptIn(UnsafeKType::class)
+ val dataset = TaskResultScope<T>(resultType, workspace, taskName, taskMeta).builder(configuration)
+
workspace.wrapResult(dataset, taskName, taskMeta)
}
}
-@Suppress("FunctionName")
public inline fun <reified T : Any, C : MetaRepr> Task(
- specification: Specification<C>,
- noinline builder: suspend TaskResultBuilder<T>.(C) -> Unit,
+ specification: MetaReader<C>,
+ noinline builder: suspend TaskResultScope<T>.(C) -> DataTree<T>,
): Task<T> = Task(typeOf<T>(), specification, builder)
\ No newline at end of file
diff --git a/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/TaskData.kt b/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/TaskData.kt
deleted file mode 100644
index 080ffec3..00000000
--- a/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/TaskData.kt
+++ /dev/null
@@ -1,50 +0,0 @@
-package space.kscience.dataforge.workspace
-
-import space.kscience.dataforge.data.Data
-import space.kscience.dataforge.data.NamedData
-import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.names.Name
-
-/**
- * A [Workspace]-locked [NamedData], that serves as a computation model.
- */
-public interface TaskData<out T : Any> : NamedData<T> {
- /**
- * The [Workspace] this data belongs to
- */
- public val workspace: Workspace
-
- /**
- * The name of the stage that produced this data. [Name.EMPTY] if the workspace intrinsic data is used.
- */
- public val taskName: Name
-
- /**
- * Stage configuration used to produce this data.
- */
- public val taskMeta: Meta
-
- /**
- * Dependencies that allow to compute transitive dependencies as well.
- */
-// override val dependencies: Collection<TaskData<*>>
-}
-
-private class TaskDataImpl<out T : Any>(
- override val workspace: Workspace,
- override val data: Data<T>,
- override val name: Name,
- override val taskName: Name,
- override val taskMeta: Meta,
-) : TaskData<T>, Data<T> by data {
-// override val dependencies: Collection<TaskData<*>> = data.dependencies.map {
-// it as? TaskData<*> ?: error("TaskData can't depend on external data")
-// }
-}
-
-/**
- * Adopt data into this workspace
- */
-public fun <T : Any> Workspace.wrapData(data: Data<T>, name: Name, taskName: Name, taskMeta: Meta): TaskData<T> =
- TaskDataImpl(this, data, name, taskName, taskMeta)
-
diff --git a/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/TaskResult.kt b/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/TaskResult.kt
index d8db6417..aff438ca 100644
--- a/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/TaskResult.kt
+++ b/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/TaskResult.kt
@@ -1,54 +1,41 @@
package space.kscience.dataforge.workspace
-import space.kscience.dataforge.data.DataSet
-import space.kscience.dataforge.data.forEach
+import kotlinx.coroutines.CoroutineScope
+import kotlinx.coroutines.Job
+import kotlinx.coroutines.joinAll
+import kotlinx.coroutines.launch
+import space.kscience.dataforge.data.DataTree
+import space.kscience.dataforge.data.asSequence
+import space.kscience.dataforge.data.launchIn
import space.kscience.dataforge.meta.Meta
import space.kscience.dataforge.names.Name
/**
* A result of a [Task]
+ * @param workspace the [Workspace] that produced the result
+ * @param taskName the name of the task that produced the result
+ * @param taskMeta The configuration of the task that produced the result
*/
-public interface TaskResult<out T : Any> : DataSet<T> {
- /**
- * The [Workspace] this [DataSet] belongs to
- */
- public val workspace: Workspace
-
- /**
- * The [Name] of the stage that produced this [DataSet]
- */
- public val taskName: Name
-
- /**
- * The configuration of the stage that produced this [DataSet]
- */
- public val taskMeta: Meta
-
- override fun iterator(): Iterator<TaskData<T>>
-
- override fun get(name: Name): TaskData<T>?
-}
-
-private class TaskResultImpl<out T : Any>(
- override val workspace: Workspace,
- override val taskName: Name,
- override val taskMeta: Meta,
- val dataSet: DataSet<T>,
-) : TaskResult<T>, DataSet<T> by dataSet {
-
- override fun iterator(): Iterator<TaskData<T>> = iterator {
- dataSet.forEach {
- yield(workspace.wrapData(it, it.name, taskName, taskMeta))
- }
- }
-
- override fun get(name: Name): TaskData<T>? = dataSet[name]?.let {
- workspace.wrapData(it, name, taskName, taskMeta)
- }
-}
+public data class TaskResult<T>(
+ public val content: DataTree<T>,
+ public val workspace: Workspace,
+ public val taskName: Name,
+ public val taskMeta: Meta,
+) : DataTree<T> by content
/**
* Wrap data into [TaskResult]
*/
-public fun <T : Any> Workspace.wrapResult(dataSet: DataSet<T>, taskName: Name, taskMeta: Meta): TaskResult<T> =
- TaskResultImpl(this, taskName, taskMeta, dataSet)
\ No newline at end of file
+public fun <T> Workspace.wrapResult(data: DataTree<T>, taskName: Name, taskMeta: Meta): TaskResult<T> =
+ TaskResult(data, this, taskName, taskMeta)
+
+/**
+ * Start computation for all data elements of this node.
+ * The resulting [Job] is completed only when all of them are completed.
+ */
+public fun TaskResult<*>.launchIn(scope: CoroutineScope): Job {
+ val jobs = asSequence().map {
+ it.launchIn(scope)
+ }.toList()
+ return scope.launch { jobs.joinAll() }
+}
\ No newline at end of file
diff --git a/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/Workspace.kt b/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/Workspace.kt
index ee00f539..7247240b 100644
--- a/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/Workspace.kt
+++ b/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/Workspace.kt
@@ -1,29 +1,35 @@
package space.kscience.dataforge.workspace
+import kotlinx.coroutines.CoroutineScope
import space.kscience.dataforge.context.ContextAware
import space.kscience.dataforge.data.Data
-import space.kscience.dataforge.data.DataSet
+import space.kscience.dataforge.data.DataTree
import space.kscience.dataforge.data.asSequence
+import space.kscience.dataforge.data.get
import space.kscience.dataforge.meta.Meta
import space.kscience.dataforge.meta.MutableMeta
-import space.kscience.dataforge.misc.DfId
+import space.kscience.dataforge.misc.DfType
import space.kscience.dataforge.names.Name
import space.kscience.dataforge.provider.Provider
+import kotlin.coroutines.CoroutineContext
-public interface DataSelector<T: Any>{
- public suspend fun select(workspace: Workspace, meta: Meta): DataSet<T>
+public fun interface DataSelector<out T> {
+ public suspend fun select(workspace: Workspace, meta: Meta): DataTree<T>
}
/**
* An environment for pull-mode computation
*/
-@DfId(Workspace.TYPE)
-public interface Workspace : ContextAware, Provider {
+@DfType(Workspace.TYPE)
+public interface Workspace : ContextAware, Provider, CoroutineScope {
+
+ override val coroutineContext: CoroutineContext get() = context.coroutineContext
+
/**
* The whole data node for current workspace
*/
- public val data: TaskResult<*>
+ public val data: DataTree<*>
/**
* All targets associated with the workspace
@@ -37,7 +43,7 @@ public interface Workspace : ContextAware, Provider {
override fun content(target: String): Map<Name, Any> {
return when (target) {
- "target", Meta.TYPE -> targets.mapKeys { Name.parse(it.key)}
+ "target", Meta.TYPE -> targets.mapKeys { Name.parse(it.key) }
Task.TYPE -> tasks
Data.TYPE -> data.asSequence().associateBy { it.name }
else -> emptyMap()
@@ -49,7 +55,7 @@ public interface Workspace : ContextAware, Provider {
return task.execute(this, taskName, taskMeta)
}
- public suspend fun produceData(taskName: Name, taskMeta: Meta, name: Name): TaskData<*>? =
+ public suspend fun produceData(taskName: Name, taskMeta: Meta, name: Name): Data<*>? =
produce(taskName, taskMeta)[name]
public companion object {
diff --git a/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/WorkspaceBuilder.kt b/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/WorkspaceBuilder.kt
index 1538460f..38d90b31 100644
--- a/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/WorkspaceBuilder.kt
+++ b/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/WorkspaceBuilder.kt
@@ -1,30 +1,27 @@
package space.kscience.dataforge.workspace
-import kotlinx.coroutines.CoroutineScope
+import space.kscience.dataforge.actions.Action
import space.kscience.dataforge.context.Context
import space.kscience.dataforge.context.ContextBuilder
import space.kscience.dataforge.context.Global
-import space.kscience.dataforge.data.*
-import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.meta.MetaRepr
-import space.kscience.dataforge.meta.MutableMeta
-import space.kscience.dataforge.meta.Specification
+import space.kscience.dataforge.data.DataTree
+import space.kscience.dataforge.data.StaticDataBuilder
+import space.kscience.dataforge.data.static
+import space.kscience.dataforge.meta.*
import space.kscience.dataforge.meta.descriptors.MetaDescriptor
import space.kscience.dataforge.meta.descriptors.MetaDescriptorBuilder
import space.kscience.dataforge.misc.DFBuilder
-import space.kscience.dataforge.misc.DFExperimental
import space.kscience.dataforge.names.Name
import space.kscience.dataforge.names.asName
-import kotlin.collections.set
import kotlin.properties.PropertyDelegateProvider
import kotlin.properties.ReadOnlyProperty
-public data class TaskReference<T : Any>(public val taskName: Name, public val task: Task<T>) : DataSelector<T> {
+public data class TaskReference<T>(public val taskName: Name, public val task: Task<T>) : DataSelector<T> {
@Suppress("UNCHECKED_CAST")
- override suspend fun select(workspace: Workspace, meta: Meta): DataSet<T> {
+ override suspend fun select(workspace: Workspace, meta: Meta): DataTree<T> {
if (workspace.tasks[taskName] == task) {
- return workspace.produce(taskName, meta) as TaskResult<T>
+ return workspace.produce(taskName, meta) as DataTree<T>
} else {
error("Task $taskName does not belong to the workspace")
}
@@ -42,13 +39,16 @@ public interface TaskContainer {
public inline fun <reified T : Any> TaskContainer.registerTask(
name: String,
descriptorBuilder: MetaDescriptorBuilder.() -> Unit = {},
- noinline builder: suspend TaskResultBuilder<T>.() -> Unit,
+ noinline builder: suspend TaskResultScope<T>.() -> DataTree<T>,
): Unit = registerTask(Name.parse(name), Task(MetaDescriptor(descriptorBuilder), builder))
+/**
+ * Create and register a new task
+ */
public inline fun <reified T : Any> TaskContainer.buildTask(
name: String,
descriptorBuilder: MetaDescriptorBuilder.() -> Unit = {},
- noinline builder: suspend TaskResultBuilder<T>.() -> Unit,
+ noinline builder: suspend TaskResultScope<T>.() -> DataTree<T>,
): TaskReference<T> {
val theName = Name.parse(name)
val descriptor = MetaDescriptor(descriptorBuilder)
@@ -59,7 +59,7 @@ public inline fun <reified T : Any> TaskContainer.buildTask(
public inline fun <reified T : Any> TaskContainer.task(
descriptor: MetaDescriptor,
- noinline builder: suspend TaskResultBuilder<T>.() -> Unit,
+ noinline builder: suspend TaskResultScope<T>.() -> DataTree<T>,
): PropertyDelegateProvider<Any?, ReadOnlyProperty<Any?, TaskReference<T>>> = PropertyDelegateProvider { _, property ->
val taskName = Name.parse(property.name)
val task = Task(descriptor, builder)
@@ -67,9 +67,12 @@ public inline fun <reified T : Any> TaskContainer.task(
ReadOnlyProperty { _, _ -> TaskReference(taskName, task) }
}
+/**
+ * Create a task based on [MetaReader]
+ */
public inline fun <reified T : Any, C : MetaRepr> TaskContainer.task(
- specification: Specification<C>,
- noinline builder: suspend TaskResultBuilder<T>.(C) -> Unit,
+ specification: MetaReader<C>,
+ noinline builder: suspend TaskResultScope<T>.(C) -> DataTree<T>,
): PropertyDelegateProvider<Any?, ReadOnlyProperty<Any?, TaskReference<T>>> = PropertyDelegateProvider { _, property ->
val taskName = Name.parse(property.name)
val task = Task(specification, builder)
@@ -77,15 +80,34 @@ public inline fun <reified T : Any, C : MetaRepr> TaskContainer.task(
ReadOnlyProperty { _, _ -> TaskReference(taskName, task) }
}
+/**
+ * A delegate to create a custom task
+ */
public inline fun <reified T : Any> TaskContainer.task(
noinline descriptorBuilder: MetaDescriptorBuilder.() -> Unit = {},
- noinline builder: suspend TaskResultBuilder<T>.() -> Unit,
+ noinline builder: suspend TaskResultScope<T>.() -> DataTree<T>,
): PropertyDelegateProvider<Any?, ReadOnlyProperty<Any?, TaskReference<T>>> =
task(MetaDescriptor(descriptorBuilder), builder)
-public class WorkspaceBuilder(private val parentContext: Context = Global) : TaskContainer {
+/**
+ * A delegate for creating a task based on [action]
+ */
+public inline fun <T : Any, reified R : Any> TaskContainer.action(
+ selector: DataSelector<T>,
+ action: Action<T, R>,
+ noinline metaTransform: MutableMeta.() -> Unit = {},
+ noinline descriptorBuilder: MetaDescriptorBuilder.() -> Unit = {},
+): PropertyDelegateProvider<Any?, ReadOnlyProperty<Any?, TaskReference<R>>> =
+ task(MetaDescriptor(descriptorBuilder)) {
+ action.execute(from(selector), taskMeta.copy(metaTransform), workspace)
+ }
+
+public class WorkspaceBuilder(
+ private val parentContext: Context = Global,
+) : TaskContainer {
private var context: Context? = null
- private var data: DataSet<*>? = null
+
+ private var data: DataTree<Any?>? = null
private val targets: HashMap<String, Meta> = HashMap()
private val tasks = HashMap<Name, Task<*>>()
private var cache: WorkspaceCache? = null
@@ -100,13 +122,8 @@ public class WorkspaceBuilder(private val parentContext: Context = Global) : Tas
/**
* Define intrinsic data for the workspace
*/
- public fun data(builder: DataSetBuilder<Any>.() -> Unit) {
- data = DataTree(builder)
- }
-
- @DFExperimental
- public fun data(scope: CoroutineScope, builder: DataSourceBuilder<Any>.() -> Unit) {
- data = DataSource(scope, builder)
+ public fun data(builder: StaticDataBuilder<Any?>.() -> Unit) {
+ data = DataTree.static(builder)
}
/**
@@ -130,9 +147,9 @@ public class WorkspaceBuilder(private val parentContext: Context = Global) : Tas
public fun build(): Workspace {
val postProcess: suspend (TaskResult<*>) -> TaskResult<*> = { result ->
- cache?.evaluate(result) ?: result
+ cache?.cache(result) ?: result
}
- return WorkspaceImpl(context ?: parentContext, data ?: DataSet.EMPTY, targets, tasks, postProcess)
+ return WorkspaceImpl(context ?: parentContext, data ?: DataTree.EMPTY, targets, tasks, postProcess)
}
}
diff --git a/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/WorkspaceCache.kt b/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/WorkspaceCache.kt
index 62df6744..42cb7b4f 100644
--- a/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/WorkspaceCache.kt
+++ b/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/WorkspaceCache.kt
@@ -1,5 +1,5 @@
package space.kscience.dataforge.workspace
public interface WorkspaceCache {
- public suspend fun <T : Any> evaluate(result: TaskResult<T>): TaskResult<T>
+ public suspend fun <T> cache(result: TaskResult<T>): TaskResult<T>
}
diff --git a/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/WorkspaceImpl.kt b/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/WorkspaceImpl.kt
index dae9667a..94839d62 100644
--- a/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/WorkspaceImpl.kt
+++ b/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/WorkspaceImpl.kt
@@ -2,21 +2,19 @@ package space.kscience.dataforge.workspace
import space.kscience.dataforge.context.Context
import space.kscience.dataforge.context.gather
-import space.kscience.dataforge.data.DataSet
+import space.kscience.dataforge.data.DataTree
import space.kscience.dataforge.meta.Meta
import space.kscience.dataforge.names.Name
internal class WorkspaceImpl internal constructor(
override val context: Context,
- data: DataSet<*>,
+ override val data: DataTree<*>,
override val targets: Map<String, Meta>,
tasks: Map<Name, Task<*>>,
private val postProcess: suspend (TaskResult<*>) -> TaskResult<*>,
) : Workspace {
- override val data: TaskResult<*> = wrapResult(data, Name.EMPTY, Meta.EMPTY)
-
override val tasks: Map<Name, Task<*>> by lazy { context.gather<Task<*>>(Task.TYPE) + tasks }
override suspend fun produce(taskName: Name, taskMeta: Meta): TaskResult<*> {
diff --git a/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/envelopeData.kt b/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/envelopeData.kt
index 39bb0726..a74f8f05 100644
--- a/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/envelopeData.kt
+++ b/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/envelopeData.kt
@@ -3,14 +3,15 @@ package space.kscience.dataforge.workspace
import space.kscience.dataforge.data.Data
import space.kscience.dataforge.data.await
import space.kscience.dataforge.io.*
-import space.kscience.dataforge.misc.DFInternal
+import space.kscience.dataforge.misc.UnsafeKType
+import kotlin.reflect.typeOf
/**
* Convert an [Envelope] to a data via given format. The actual parsing is done lazily.
*/
-@OptIn(DFInternal::class)
-public fun <T : Any> Envelope.toData(format: IOReader<T>): Data<T> = Data(format.type, meta) {
+@OptIn(UnsafeKType::class)
+public inline fun <reified T : Any> Envelope.toData(format: IOReader<T>): Data<T> = Data(typeOf<T>(), meta) {
data?.readWith(format) ?: error("Can't convert envelope without data to Data")
}
diff --git a/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/taskBuilders.kt b/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/taskResultBuilders.kt
similarity index 57%
rename from dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/taskBuilders.kt
rename to dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/taskResultBuilders.kt
index bf3d5921..9df49aba 100644
--- a/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/taskBuilders.kt
+++ b/dataforge-workspace/src/commonMain/kotlin/space/kscience/dataforge/workspace/taskResultBuilders.kt
@@ -1,18 +1,20 @@
package space.kscience.dataforge.workspace
import space.kscience.dataforge.context.PluginFactory
-import space.kscience.dataforge.data.DataSet
-import space.kscience.dataforge.data.forEach
-import space.kscience.dataforge.data.map
-import space.kscience.dataforge.meta.*
+import space.kscience.dataforge.data.*
+import space.kscience.dataforge.meta.Meta
+import space.kscience.dataforge.meta.MutableMeta
+import space.kscience.dataforge.meta.copy
+import space.kscience.dataforge.meta.remove
import space.kscience.dataforge.misc.DFExperimental
+import space.kscience.dataforge.misc.UnsafeKType
import space.kscience.dataforge.names.Name
import space.kscience.dataforge.names.plus
/**
* A task meta without a node corresponding to the task itself (removing a node with name of the task).
*/
-public val TaskResultBuilder<*>.defaultDependencyMeta: Meta
+public val TaskResultScope<*>.defaultDependencyMeta: Meta
get() = taskMeta.copy {
remove(taskName)
}
@@ -23,22 +25,22 @@ public val TaskResultBuilder<*>.defaultDependencyMeta: Meta
* @param selector a workspace data selector. Could be either task selector or initial data selector.
* @param dependencyMeta meta used for selector. The same meta is used for caching. By default, uses [defaultDependencyMeta].
*/
-public suspend fun <T : Any> TaskResultBuilder<*>.from(
+public suspend fun <T> TaskResultScope<*>.from(
selector: DataSelector<T>,
dependencyMeta: Meta = defaultDependencyMeta,
-): DataSet<T> = selector.select(workspace, dependencyMeta)
+): DataTree<T> = selector.select(workspace, dependencyMeta)
-public suspend inline fun <T : Any, reified P : WorkspacePlugin> TaskResultBuilder<*>.from(
+public suspend inline fun <T, reified P : WorkspacePlugin> TaskResultScope<*>.from(
plugin: P,
dependencyMeta: Meta = defaultDependencyMeta,
selectorBuilder: P.() -> TaskReference<T>,
-): DataSet<T> {
+): TaskResult<T> {
require(workspace.context.plugins.contains(plugin)) { "Plugin $plugin is not loaded into $workspace" }
val taskReference: TaskReference<T> = plugin.selectorBuilder()
val res = workspace.produce(plugin.name + taskReference.taskName, dependencyMeta)
//TODO add explicit check after https://youtrack.jetbrains.com/issue/KT-32956
@Suppress("UNCHECKED_CAST")
- return res as TaskResult<T>
+ return res as TaskResult<T>
}
/**
@@ -48,11 +50,11 @@ public suspend inline fun <T : Any, reified P : WorkspacePlugin> TaskResultBuild
* @param dependencyMeta meta used for selector. The same meta is used for caching. By default, uses [defaultDependencyMeta].
* @param selectorBuilder a builder of task from the plugin.
*/
-public suspend inline fun <reified T : Any, reified P : WorkspacePlugin> TaskResultBuilder<*>.from(
+public suspend inline fun <reified T, reified P : WorkspacePlugin> TaskResultScope<*>.from(
pluginFactory: PluginFactory<P>,
dependencyMeta: Meta = defaultDependencyMeta,
selectorBuilder: P.() -> TaskReference<T>,
-): DataSet<T> {
+): TaskResult<T> {
val plugin = workspace.context.plugins[pluginFactory]
?: error("Plugin ${pluginFactory.tag} not loaded into workspace context")
val taskReference: TaskReference<T> = plugin.selectorBuilder()
@@ -62,13 +64,11 @@ public suspend inline fun <reified T : Any, reified P : WorkspacePlugin> TaskRes
return res as TaskResult<T>
}
-public val TaskResultBuilder<*>.allData: DataSelector<*>
- get() = object : DataSelector<Any> {
- override suspend fun select(workspace: Workspace, meta: Meta): DataSet<Any> = workspace.data
- }
+public val TaskResultScope<*>.allData: DataSelector<*>
+ get() = DataSelector { workspace, _ -> workspace.data }
/**
- * Perform a lazy mapping task using given [selector] and [action]. The meta of resulting
+ * Perform a lazy mapping task using given [selector] and one-to-one [action].
* TODO move selector to receiver with multi-receivers
*
* @param selector a workspace data selector. Could be either task selector or initial data selector.
@@ -76,25 +76,50 @@ public val TaskResultBuilder<*>.allData: DataSelector<*>
* @param dataMetaTransform additional transformation of individual data meta.
* @param action process individual data asynchronously.
*/
+@OptIn(UnsafeKType::class)
@DFExperimental
-public suspend inline fun <T : Any, reified R : Any> TaskResultBuilder<R>.pipeFrom(
+public suspend fun <T, R> TaskResultScope<R>.transformEach(
selector: DataSelector<T>,
dependencyMeta: Meta = defaultDependencyMeta,
dataMetaTransform: MutableMeta.(name: Name) -> Unit = {},
- crossinline action: suspend (arg: T, name: Name, meta: Meta) -> R,
-) {
- from(selector, dependencyMeta).forEach { data ->
- val meta = data.meta.toMutableMeta().apply {
- taskMeta[taskName]?.let { taskName.put(it) }
- dataMetaTransform(data.name)
- }
-
- val res = data.map(workspace.context.coroutineContext, meta) {
- action(it, data.name, meta)
- }
-
- data(data.name, res)
+ action: suspend NamedValueWithMeta<T>.() -> R,
+): DataTree<R> = from(selector, dependencyMeta).transformEach<T, R>(
+ resultType,
+ workspace.context,
+ metaTransform = { name ->
+ taskMeta[taskName]?.let { taskName put it }
+ dataMetaTransform(name)
}
+) {
+ action(it)
}
+@OptIn(UnsafeKType::class)
+public fun <R> TaskResultScope<R>.result(data: Data<R>): DataTree<R> = DataTree.static(resultType) {
+ data(Name.EMPTY, data)
+}
+
+@OptIn(UnsafeKType::class)
+public fun <R> TaskResultScope<R>.result(builder: StaticDataBuilder<R>.() -> Unit): DataTree<R> =
+ DataTree.static(resultType, builder)
+
+///**
+// * Set given [dataSet] as a task result.
+// */
+//public fun <T> TaskResultBuilder<T>.result(dataSet: DataTree<T>) {
+// putAll(dataSet)
+//}
+
+///**
+// * Use provided [action] to fill the result
+// */
+//@DFExperimental
+//public suspend inline fun <T, reified R> TaskResultScope<R>.actionFrom(
+// selector: DataSelector<T>,
+// action: Action<T, R>,
+// dependencyMeta: Meta = defaultDependencyMeta,
+//) {
+// putAll(action.execute(from(selector, dependencyMeta), dependencyMeta, workspace))
+//}
+
diff --git a/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/CachingAction.kt b/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/CachingAction.kt
new file mode 100644
index 00000000..57b0746e
--- /dev/null
+++ b/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/CachingAction.kt
@@ -0,0 +1,27 @@
+package space.kscience.dataforge.workspace
+
+import space.kscience.dataforge.actions.AbstractAction
+import space.kscience.dataforge.data.*
+import space.kscience.dataforge.meta.Meta
+import space.kscience.dataforge.names.Name
+import kotlin.reflect.KType
+
+internal class CachingAction<T>(
+ type: KType, private val caching: (NamedData<T>) -> NamedData<T>
+) : AbstractAction<T, T>(type) {
+
+ override fun DataBuilderScope<T>.generate(
+ source: DataTree<T>,
+ meta: Meta
+ ): Map<Name, Data<T>> = buildMap {
+ source.forEach {
+ val cached = caching(it)
+ put(cached.name, cached)
+ }
+ }
+
+ override suspend fun DataSink<T>.update(source: DataTree<T>, actionMeta: Meta, updateName: Name) {
+ val updatedData = source.read(updateName)
+ write(updateName, updatedData?.named(updateName)?.let(caching))
+ }
+}
\ No newline at end of file
diff --git a/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/FileDataTree.kt b/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/FileDataTree.kt
new file mode 100644
index 00000000..049ec2ec
--- /dev/null
+++ b/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/FileDataTree.kt
@@ -0,0 +1,196 @@
+package space.kscience.dataforge.workspace
+
+import kotlinx.coroutines.Dispatchers
+import kotlinx.coroutines.Job
+import kotlinx.coroutines.channels.awaitClose
+import kotlinx.coroutines.flow.*
+import kotlinx.coroutines.isActive
+import kotlinx.coroutines.launch
+import space.kscience.dataforge.data.Data
+import space.kscience.dataforge.data.DataTree
+import space.kscience.dataforge.data.StaticData
+import space.kscience.dataforge.io.*
+import space.kscience.dataforge.meta.Meta
+import space.kscience.dataforge.meta.copy
+import space.kscience.dataforge.names.Name
+import space.kscience.dataforge.names.NameToken
+import space.kscience.dataforge.names.asName
+import space.kscience.dataforge.names.plus
+import java.nio.file.*
+import java.nio.file.attribute.BasicFileAttributes
+import java.nio.file.spi.FileSystemProvider
+import kotlin.io.path.*
+import kotlin.reflect.KType
+import kotlin.reflect.typeOf
+
+
+public class FileDataTree(
+ public val io: IOPlugin,
+ public val path: Path,
+ private val monitor: Boolean = false
+) : DataTree<Binary> {
+ override val dataType: KType = typeOf<Binary>()
+
+ /**
+ * Read data with supported envelope format and binary format. If the envelope format is null, then read binary directly from file.
+ * The operation is blocking since it must read the meta header. The reading of envelope body is lazy
+ */
+ private fun readFileAsData(
+ path: Path,
+ ): Data<Binary> {
+ val envelope = io.readEnvelopeFile(path, true)
+ val updatedMeta = envelope.meta.copy {
+ FILE_PATH_KEY put path.toString()
+ FILE_EXTENSION_KEY put path.extension
+
+ val attributes = path.readAttributes<BasicFileAttributes>()
+ FILE_UPDATE_TIME_KEY put attributes.lastModifiedTime().toInstant().toString()
+ FILE_CREATE_TIME_KEY put attributes.creationTime().toInstant().toString()
+ }
+ return StaticData(
+ typeOf<Binary>(),
+ envelope.data ?: Binary.EMPTY,
+ updatedMeta
+ )
+ }
+
+ private fun readFilesFromDirectory(
+ path: Path
+ ): Map<NameToken, FileDataTree> = path.listDirectoryEntries().filterNot { it.name.startsWith("@") }.associate {
+ NameToken.parse(it.nameWithoutExtension) to FileDataTree(io, it)
+ }
+
+ override val data: Data<Binary>?
+ get() = when {
+ path.isRegularFile() -> {
+ //TODO process zip
+ readFileAsData(path)
+ }
+
+ path.isDirectory() -> {
+ //FIXME find data and meta in a single pass instead of two
+
+ val dataBinary: Binary? = path.listDirectoryEntries().find {
+ it.fileName.nameWithoutExtension == IOPlugin.DATA_FILE_NAME
+ }?.asBinary()
+
+ val meta: Meta? = path.listDirectoryEntries().find {
+ it.fileName.nameWithoutExtension == IOPlugin.META_FILE_NAME
+ }?.let {
+ io.readMetaFileOrNull(it)
+ }
+
+ if (dataBinary != null || meta != null) {
+ StaticData(
+ typeOf<Binary>(),
+ dataBinary ?: Binary.EMPTY,
+ meta ?: Meta.EMPTY
+ )
+ } else {
+ null
+ }
+ }
+
+ else -> {
+ null
+ }
+ }
+
+
+ override val items: Map<NameToken, DataTree<Binary>>
+ get() = when {
+ path.isDirectory() -> readFilesFromDirectory(path)
+ path.isRegularFile() && path.extension == "zip" -> {
+ //Using an explicit Zip file system to avoid bizarre compatibility bugs
+ val fsProvider = FileSystemProvider.installedProviders().find { it.scheme == "jar" }
+ ?: error("Zip file system provider not found")
+ val fs = fsProvider.newFileSystem(path, emptyMap<String, Any>())
+ readFilesFromDirectory(fs.rootDirectories.single())
+ }
+
+ else -> emptyMap()
+ }
+
+
+ override val updates: Flow<Name> = if (monitor) {
+ callbackFlow<Name> {
+ val watchService: WatchService = path.fileSystem.newWatchService()
+
+ fun Path.toName() = Name(map { NameToken.parse(it.nameWithoutExtension) })
+
+ fun monitor(childPath: Path): Job {
+ val key: WatchKey = childPath.register(
+ watchService, arrayOf(
+ StandardWatchEventKinds.ENTRY_DELETE,
+ StandardWatchEventKinds.ENTRY_MODIFY,
+ StandardWatchEventKinds.ENTRY_CREATE,
+ )
+ )
+
+ return launch {
+ while (isActive) {
+ for (event: WatchEvent<*> in key.pollEvents()) {
+ val eventPath = event.context() as Path
+ if (event.kind() === StandardWatchEventKinds.ENTRY_CREATE) {
+ monitor(eventPath)
+ } else {
+ send(eventPath.relativeTo(path).toName())
+ }
+ }
+ key.reset()
+ }
+ }
+ }
+
+ monitor(path)
+
+ awaitClose {
+ watchService.close()
+ }
+
+ }.flowOn(Dispatchers.IO).shareIn(io.context, SharingStarted.WhileSubscribed())
+ } else {
+ emptyFlow()
+ }
+
+ public companion object {
+ public val FILE_KEY: Name = "file".asName()
+ public val FILE_PATH_KEY: Name = FILE_KEY + "path"
+ public val FILE_EXTENSION_KEY: Name = FILE_KEY + "extension"
+ public val FILE_CREATE_TIME_KEY: Name = FILE_KEY + "created"
+ public val FILE_UPDATE_TIME_KEY: Name = FILE_KEY + "updated"
+ public const val DF_FILE_EXTENSION: String = "df"
+ public val DEFAULT_IGNORE_EXTENSIONS: Set<String> = setOf(DF_FILE_EXTENSION)
+ }
+}
+
+public fun IOPlugin.readDirectory(path: Path, monitor: Boolean = false): FileDataTree =
+ FileDataTree(this, path, monitor)
+
+
+///**
+// * @param resources The names of the resources to read.
+// * @param classLoader The class loader to use for loading the resources. By default, it uses the current thread's context class loader.
+// */
+//@DFExperimental
+//public fun DataSink<Binary>.resources(
+// io: IOPlugin,
+// resource: String,
+// vararg otherResources: String,
+// classLoader: ClassLoader = Thread.currentThread().contextClassLoader,
+//) {
+// //create a file system if necessary
+// val uri = Thread.currentThread().contextClassLoader.getResource("common")!!.toURI()
+// try {
+// uri.toPath()
+// } catch (e: FileSystemNotFoundException) {
+// FileSystems.newFileSystem(uri, mapOf("create" to "true"))
+// }
+//
+// listOf(resource, *otherResources).forEach { r ->
+// val path = classLoader.getResource(r)?.toURI()?.toPath() ?: error(
+// "Resource with name $r is not resolved"
+// )
+// io.readAsDataTree(r.asName(), path)
+// }
+//}
diff --git a/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/FileWorkspaceCache.kt b/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/FileWorkspaceCache.kt
index 279e61a2..ce32848a 100644
--- a/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/FileWorkspaceCache.kt
+++ b/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/FileWorkspaceCache.kt
@@ -6,15 +6,18 @@ import kotlinx.serialization.KSerializer
import kotlinx.serialization.json.Json
import kotlinx.serialization.protobuf.ProtoBuf
import kotlinx.serialization.serializer
+import space.kscience.dataforge.actions.Action
+import space.kscience.dataforge.actions.invoke
import space.kscience.dataforge.context.error
import space.kscience.dataforge.context.logger
import space.kscience.dataforge.context.request
import space.kscience.dataforge.data.Data
import space.kscience.dataforge.data.await
+import space.kscience.dataforge.data.named
import space.kscience.dataforge.io.*
+import space.kscience.dataforge.meta.Meta
import space.kscience.dataforge.misc.DFExperimental
-import space.kscience.dataforge.misc.DFInternal
-import space.kscience.dataforge.names.Name
+import space.kscience.dataforge.misc.UnsafeKType
import space.kscience.dataforge.names.withIndex
import java.nio.file.Path
import kotlin.io.path.deleteIfExists
@@ -22,11 +25,7 @@ import kotlin.io.path.div
import kotlin.io.path.exists
import kotlin.reflect.KType
-public class JsonIOFormat<T : Any>(override val type: KType) : IOFormat<T> {
-
- @Suppress("UNCHECKED_CAST")
- private val serializer: KSerializer<T> = serializer(type) as KSerializer<T>
-
+public class JsonIOFormat<T>(public val serializer: KSerializer<T>) : IOFormat<T> {
override fun readFrom(source: Source): T = Json.decodeFromString(serializer, source.readString())
override fun writeTo(sink: Sink, obj: T) {
@@ -34,12 +33,11 @@ public class JsonIOFormat<T : Any>(override val type: KType) : IOFormat<T> {
}
}
+/**
+ * An [IOFormat] based on Protobuf representation of the serializeable object.
+ */
@OptIn(ExperimentalSerializationApi::class)
-public class ProtobufIOFormat<T : Any>(override val type: KType) : IOFormat<T> {
-
- @Suppress("UNCHECKED_CAST")
- private val serializer: KSerializer<T> = serializer(type) as KSerializer<T>
-
+public class ProtobufIOFormat<T>(public val serializer: KSerializer<T>) : IOFormat<T> {
override fun readFrom(source: Source): T = ProtoBuf.decodeFromByteArray(serializer, source.readByteArray())
override fun writeTo(sink: Sink, obj: T) {
@@ -47,21 +45,42 @@ public class ProtobufIOFormat<T : Any>(override val type: KType) : IOFormat<T> {
}
}
+public interface IOFormatResolveStrategy {
+ public fun <T> resolve(type: KType, meta: Meta): IOFormat<T>
-public class FileWorkspaceCache(public val cacheDirectory: Path) : WorkspaceCache {
+ public companion object {
+ public val PROTOBUF: IOFormatResolveStrategy = object : IOFormatResolveStrategy {
+ @Suppress("UNCHECKED_CAST")
+ override fun <T> resolve(
+ type: KType,
+ meta: Meta
+ ): IOFormat<T> = ProtobufIOFormat(serializer(type) as KSerializer<T>)
+ }
- // private fun <T : Any> TaskData<*>.checkType(taskType: KType): TaskData<T> = this as TaskData<T>
+ public val JSON: IOFormatResolveStrategy = object : IOFormatResolveStrategy {
+ @Suppress("UNCHECKED_CAST")
+ override fun <T> resolve(
+ type: KType,
+ meta: Meta
+ ): IOFormat<T> = JsonIOFormat(serializer(type) as KSerializer<T>)
+ }
+ }
+}
- @OptIn(DFExperimental::class, DFInternal::class)
- override suspend fun <T : Any> evaluate(result: TaskResult<T>): TaskResult<T> {
+public class FileWorkspaceCache(
+ public val cacheDirectory: Path,
+ private val ioFormatResolveStrategy: IOFormatResolveStrategy,
+) : WorkspaceCache {
+
+
+ @OptIn(DFExperimental::class, UnsafeKType::class)
+ override suspend fun <T> cache(result: TaskResult<T>): TaskResult<T> {
val io = result.workspace.context.request(IOPlugin)
- val format: IOFormat<T> = io.resolveIOFormat(result.dataType, result.taskMeta)
- ?: ProtobufIOFormat(result.dataType)
- ?: error("Can't resolve IOFormat for ${result.dataType}")
+ val format: IOFormat<T> = ioFormatResolveStrategy.resolve<T>(result.dataType, result.taskMeta)
- fun evaluateDatum(data: TaskData<T>): TaskData<T> {
+ val cachingAction: Action<T, T> = CachingAction(result.dataType) { data ->
val path = cacheDirectory /
result.taskName.withIndex(result.taskMeta.hashCode().toString(16)).toString() /
data.name.toString()
@@ -80,7 +99,7 @@ public class FileWorkspaceCache(public val cacheDirectory: Path) : WorkspaceCach
}
}
- //waiting for data in current scope because Envelope is synchronous
+ //waiting for data in the current scope because Envelope is synchronous
return@Data data.await().also { result ->
val envelope = Envelope {
meta = data.meta
@@ -92,16 +111,16 @@ public class FileWorkspaceCache(public val cacheDirectory: Path) : WorkspaceCach
}
}
- return data.workspace.wrapData(datum, data.name, data.taskName, data.taskMeta)
+ datum.named(data.name)
}
- return object : TaskResult<T> by result {
- override fun iterator(): Iterator<TaskData<T>> =
- result.iterator().asSequence().map { evaluateDatum(it) }.iterator()
+ val cachedTree = cachingAction(result)
- override fun get(name: Name): TaskData<T>? = result[name]?.let { evaluateDatum(it) }
- }
+ return result.workspace.wrapResult(cachedTree, result.taskName, result.taskMeta)
}
}
-public fun WorkspaceBuilder.fileCache(cacheDir: Path): Unit = cache(FileWorkspaceCache(cacheDir))
\ No newline at end of file
+public fun WorkspaceBuilder.fileCache(
+ cacheDir: Path,
+ ioFormatResolveStrategy: IOFormatResolveStrategy = IOFormatResolveStrategy.PROTOBUF
+): Unit = cache(FileWorkspaceCache(cacheDir, ioFormatResolveStrategy))
\ No newline at end of file
diff --git a/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/InMemoryWorkspaceCache.kt b/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/InMemoryWorkspaceCache.kt
index bb8ea9a4..9e986ba9 100644
--- a/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/InMemoryWorkspaceCache.kt
+++ b/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/InMemoryWorkspaceCache.kt
@@ -1,39 +1,41 @@
package space.kscience.dataforge.workspace
+import space.kscience.dataforge.actions.Action
+import space.kscience.dataforge.actions.invoke
+import space.kscience.dataforge.data.Data
+import space.kscience.dataforge.data.named
import space.kscience.dataforge.meta.Meta
+import space.kscience.dataforge.misc.DFExperimental
import space.kscience.dataforge.names.Name
import kotlin.reflect.KType
import kotlin.reflect.full.isSubtypeOf
-private typealias TaskResultId = Pair<Name, Meta>
+private data class TaskResultId(val name: Name, val meta: Meta)
public class InMemoryWorkspaceCache : WorkspaceCache {
- // never do that at home!
- private val cache = HashMap<TaskResultId, HashMap<Name, TaskData<*>>>()
+ private val cache = HashMap<TaskResultId, HashMap<Name, Data<*>>>()
@Suppress("UNCHECKED_CAST")
- private fun <T : Any> TaskData<*>.checkType(taskType: KType): TaskData<T> =
- if (type.isSubtypeOf(taskType)) this as TaskData<T>
+ private fun <T> Data<*>.checkType(taskType: KType): Data<T> =
+ if (type.isSubtypeOf(taskType)) this as Data<T>
else error("Cached data type mismatch: expected $taskType but got $type")
- override suspend fun <T : Any> evaluate(result: TaskResult<T>): TaskResult<T> {
- for (d: TaskData<T> in result) {
- cache.getOrPut(result.taskName to result.taskMeta) { HashMap() }.getOrPut(d.name) { d }
- }
-
- return object : TaskResult<T> by result {
- override fun iterator(): Iterator<TaskData<T>> = (cache[result.taskName to result.taskMeta]
- ?.values?.map { it.checkType<T>(result.dataType) }
- ?: emptyList()).iterator()
-
- override fun get(name: Name): TaskData<T>? {
- val cached: TaskData<*> = cache[result.taskName to result.taskMeta]?.get(name) ?: return null
- //TODO check types
- return cached.checkType(result.dataType)
+ @OptIn(DFExperimental::class)
+ override suspend fun <T> cache(result: TaskResult<T>): TaskResult<T> {
+ val cachingAction: Action<T, T> = CachingAction(result.dataType) { data ->
+ val cachedData = cache.getOrPut(TaskResultId(result.taskName, result.taskMeta)){
+ HashMap()
+ }.getOrPut(data.name){
+ data
}
+ cachedData.checkType<T>(result.dataType).named(data.name)
}
+
+ val cachedTree = cachingAction(result)
+
+ return result.workspace.wrapResult(cachedTree, result.taskName, result.taskMeta)
}
}
diff --git a/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/dataFilterJvm.kt b/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/dataFilterJvm.kt
new file mode 100644
index 00000000..ba8e148c
--- /dev/null
+++ b/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/dataFilterJvm.kt
@@ -0,0 +1,61 @@
+package space.kscience.dataforge.workspace
+
+import space.kscience.dataforge.data.*
+import space.kscience.dataforge.misc.DFInternal
+import space.kscience.dataforge.names.Name
+import kotlin.reflect.KType
+import kotlin.reflect.full.isSubtypeOf
+import kotlin.reflect.typeOf
+
+
+/**
+ * Cast the node to given type if the cast is possible or return null
+ */
+@Suppress("UNCHECKED_CAST")
+private fun <R> Data<*>.castOrNull(type: KType): Data<R>? =
+ if (!this.type.isSubtypeOf(type)) {
+ null
+ } else {
+ object : Data<R> by (this as Data<R>) {
+ override val type: KType = type
+ }
+ }
+
+/**
+ * Select all data matching given type and filters. Does not modify paths
+ *
+ * @param filter additional filtering condition based on item name and meta. By default, accepts all
+ */
+@Suppress("UNCHECKED_CAST")
+@DFInternal
+public fun <R> DataTree<*>.filterByType(
+ type: KType,
+ branch: Name = Name.EMPTY,
+ filter: DataFilter = DataFilter.Companion.EMPTY,
+): DataTree<R> {
+ val filterWithType = DataFilter { name, meta, dataType ->
+ filter.accepts(name, meta, dataType) && dataType.isSubtypeOf(type)
+ }
+ return FilteredDataTree(this, filterWithType, branch, type) as DataTree<R>
+}
+
+/**
+ * Select a single datum of the appropriate type
+ */
+@OptIn(DFInternal::class)
+public inline fun <reified R : Any> DataTree<*>.filterByType(
+ branch: Name = Name.EMPTY,
+ filter: DataFilter = DataFilter.Companion.EMPTY,
+): DataTree<R> = filterByType(typeOf<R>(), branch, filter = filter)
+
+/**
+ * Select a single datum if it is present and of given [type]
+ */
+public fun <R> DataTree<*>.getByType(type: KType, name: Name): NamedData<R>? =
+ get(name)?.castOrNull<R>(type)?.named(name)
+
+public inline fun <reified R : Any> DataTree<*>.getByType(name: Name): NamedData<R>? =
+ this@getByType.getByType(typeOf<R>(), name)
+
+public inline fun <reified R : Any> DataTree<*>.getByType(name: String): NamedData<R>? =
+ this@getByType.getByType(typeOf<R>(), Name.parse(name))
diff --git a/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/fileData.kt b/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/fileData.kt
deleted file mode 100644
index d9f678b3..00000000
--- a/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/fileData.kt
+++ /dev/null
@@ -1,299 +0,0 @@
-package space.kscience.dataforge.workspace
-
-import kotlinx.coroutines.Dispatchers
-import kotlinx.coroutines.isActive
-import kotlinx.coroutines.launch
-import kotlinx.coroutines.withContext
-import space.kscience.dataforge.context.error
-import space.kscience.dataforge.context.logger
-import space.kscience.dataforge.context.warn
-import space.kscience.dataforge.data.*
-import space.kscience.dataforge.io.*
-import space.kscience.dataforge.meta.Meta
-import space.kscience.dataforge.meta.copy
-import space.kscience.dataforge.meta.get
-import space.kscience.dataforge.meta.string
-import space.kscience.dataforge.misc.DFExperimental
-import space.kscience.dataforge.misc.DFInternal
-import space.kscience.dataforge.names.Name
-import space.kscience.dataforge.names.NameToken
-import space.kscience.dataforge.names.asName
-import space.kscience.dataforge.names.plus
-import space.kscience.dataforge.workspace.FileData.Companion.DEFAULT_IGNORE_EXTENSIONS
-import java.nio.file.Files
-import java.nio.file.Path
-import java.nio.file.StandardWatchEventKinds
-import java.nio.file.WatchEvent
-import java.nio.file.attribute.BasicFileAttributes
-import java.nio.file.spi.FileSystemProvider
-import java.time.Instant
-import kotlin.io.path.extension
-import kotlin.io.path.name
-import kotlin.io.path.nameWithoutExtension
-import kotlin.io.path.readAttributes
-import kotlin.reflect.KType
-import kotlin.reflect.typeOf
-
-
-//public typealias FileFormatResolver<T> = (Path, Meta) -> IOFormat<T>
-
-public typealias FileFormatResolver<T> = (path: Path, meta: Meta) -> IOReader<T>?
-
-/**
- * A data based on a filesystem [Path]
- */
-public class FileData<T> internal constructor(private val data: Data<T>, public val path: Path) : Data<T> by data {
-
- // public val path: String? get() = meta[META_FILE_PATH_KEY].string
-// public val extension: String? get() = meta[META_FILE_EXTENSION_KEY].string
-//
- public val createdTime: Instant? get() = meta[FILE_CREATE_TIME_KEY].string?.let { Instant.parse(it) }
- public val updatedTime: Instant? get() = meta[FILE_UPDATE_TIME_KEY].string?.let { Instant.parse(it) }
-
- public companion object {
- public val FILE_KEY: Name = "file".asName()
- public val FILE_PATH_KEY: Name = FILE_KEY + "path"
- public val FILE_EXTENSION_KEY: Name = FILE_KEY + "extension"
- public val FILE_CREATE_TIME_KEY: Name = FILE_KEY + "created"
- public val FILE_UPDATE_TIME_KEY: Name = FILE_KEY + "updated"
- public const val DF_FILE_EXTENSION: String = "df"
- public val DEFAULT_IGNORE_EXTENSIONS: Set<String> = setOf(DF_FILE_EXTENSION)
- }
-}
-
-
-/**
- * Read data with supported envelope format and binary format. If envelope format is null, then read binary directly from file.
- * The operation is blocking since it must read meta header. The reading of envelope body is lazy
- */
-@OptIn(DFInternal::class)
-@DFExperimental
-public fun <T : Any> IOPlugin.readDataFile(
- path: Path,
- formatResolver: FileFormatResolver<T>,
-): FileData<T>? {
- val envelope = readEnvelopeFile(path, true)
- val format = formatResolver(path, envelope.meta) ?: return null
- val updatedMeta = envelope.meta.copy {
- FileData.FILE_PATH_KEY put path.toString()
- FileData.FILE_EXTENSION_KEY put path.extension
-
- val attributes = path.readAttributes<BasicFileAttributes>()
- FileData.FILE_UPDATE_TIME_KEY put attributes.lastModifiedTime().toInstant().toString()
- FileData.FILE_CREATE_TIME_KEY put attributes.creationTime().toInstant().toString()
- }
- return FileData(
- Data(format.type, updatedMeta) {
- (envelope.data ?: Binary.EMPTY).readWith(format)
- },
- path
- )
-}
-
-
-context(IOPlugin) @DFExperimental
-private fun <T : Any> DataSetBuilder<T>.directory(
- path: Path,
- ignoreExtensions: Set<String>,
- formatResolver: FileFormatResolver<T>,
-) {
- Files.list(path).forEach { childPath ->
- val fileName = childPath.fileName.toString()
- if (fileName.startsWith(IOPlugin.META_FILE_NAME)) {
- meta(readMetaFile(childPath))
- } else if (!fileName.startsWith("@")) {
- file(childPath, ignoreExtensions, formatResolver)
- }
- }
-}
-
-/**
- * Read the directory as a data node. If [path] is a zip archive, read it as directory
- */
-@DFExperimental
-@DFInternal
-public fun <T : Any> IOPlugin.readDataDirectory(
- type: KType,
- path: Path,
- ignoreExtensions: Set<String> = DEFAULT_IGNORE_EXTENSIONS,
- formatResolver: FileFormatResolver<T>,
-): DataTree<T> {
- //read zipped data node
- if (path.fileName != null && path.fileName.toString().endsWith(".zip")) {
- //Using explicit Zip file system to avoid bizarre compatibility bugs
- val fsProvider = FileSystemProvider.installedProviders().find { it.scheme == "jar" }
- ?: error("Zip file system provider not found")
- val fs = fsProvider.newFileSystem(path, mapOf("create" to "true"))
-
- return readDataDirectory(type, fs.rootDirectories.first(), ignoreExtensions, formatResolver)
- }
- if (!Files.isDirectory(path)) error("Provided path $path is not a directory")
- return DataTree(type) {
- meta {
- FileData.FILE_PATH_KEY put path.toString()
- }
- directory(path, ignoreExtensions, formatResolver)
- }
-}
-
-@OptIn(DFInternal::class)
-@DFExperimental
-public inline fun <reified T : Any> IOPlugin.readDataDirectory(
- path: Path,
- ignoreExtensions: Set<String> = DEFAULT_IGNORE_EXTENSIONS,
- noinline formatResolver: FileFormatResolver<T>,
-): DataTree<T> = readDataDirectory(typeOf<T>(), path, ignoreExtensions, formatResolver)
-
-/**
- * Read raw binary data tree from the directory. All files are read as-is (save for meta files).
- */
-@DFExperimental
-public fun IOPlugin.readRawDirectory(
- path: Path,
- ignoreExtensions: Set<String> = emptySet(),
-): DataTree<Binary> = readDataDirectory(path, ignoreExtensions) { _, _ -> IOReader.binary }
-
-
-private fun Path.toName() = Name(map { NameToken.parse(it.nameWithoutExtension) })
-
-@DFInternal
-@DFExperimental
-public fun <T : Any> IOPlugin.monitorDataDirectory(
- type: KType,
- path: Path,
- ignoreExtensions: Set<String> = DEFAULT_IGNORE_EXTENSIONS,
- formatResolver: FileFormatResolver<T>,
-): DataSource<T> {
- if (path.fileName.toString().endsWith(".zip")) error("Monitoring not supported for ZipFS")
- if (!Files.isDirectory(path)) error("Provided path $path is not a directory")
- return DataSource(type, context) {
- directory(path, ignoreExtensions, formatResolver)
- launch(Dispatchers.IO) {
- val watchService = path.fileSystem.newWatchService()
-
- path.register(
- watchService,
- StandardWatchEventKinds.ENTRY_DELETE,
- StandardWatchEventKinds.ENTRY_MODIFY,
- StandardWatchEventKinds.ENTRY_CREATE
- )
-
- do {
- val key = watchService.take()
- if (key != null) {
- for (event: WatchEvent<*> in key.pollEvents()) {
- val eventPath = event.context() as Path
- if (event.kind() == StandardWatchEventKinds.ENTRY_DELETE) {
- remove(eventPath.toName())
- } else {
- val fileName = eventPath.fileName.toString()
- if (fileName.startsWith(IOPlugin.META_FILE_NAME)) {
- meta(readMetaFile(eventPath))
- } else if (!fileName.startsWith("@")) {
- file(eventPath, ignoreExtensions, formatResolver)
- }
- }
- }
- key.reset()
- }
- } while (isActive && key != null)
- }
- }
-}
-
-
-/**
- * Start monitoring given directory ([path]) as a [DataSource].
- */
-@OptIn(DFInternal::class)
-@DFExperimental
-public inline fun <reified T : Any> IOPlugin.monitorDataDirectory(
- path: Path,
- ignoreExtensions: Set<String> = DEFAULT_IGNORE_EXTENSIONS,
- noinline formatResolver: FileFormatResolver<T>,
-): DataSource<T> = monitorDataDirectory(typeOf<T>(), path, ignoreExtensions, formatResolver)
-
-/**
- * Read and monitor raw binary data tree from the directory. All files are read as-is (save for meta files).
- */
-@DFExperimental
-public fun IOPlugin.monitorRawDirectory(
- path: Path,
- ignoreExtensions: Set<String> = DEFAULT_IGNORE_EXTENSIONS,
-): DataSource<Binary> = monitorDataDirectory(path, ignoreExtensions) { _, _ -> IOReader.binary }
-
-/**
- * Write data tree to existing directory or create a new one using default [java.nio.file.FileSystem] provider
- */
-@DFExperimental
-public suspend fun <T : Any> IOPlugin.writeDataDirectory(
- path: Path,
- tree: DataTree<T>,
- format: IOWriter<T>,
- envelopeFormat: EnvelopeFormat? = null,
-) {
- withContext(Dispatchers.IO) {
- if (!Files.exists(path)) {
- Files.createDirectories(path)
- } else if (!Files.isDirectory(path)) {
- error("Can't write a node into file")
- }
- tree.items.forEach { (token, item) ->
- val childPath = path.resolve(token.toString())
- when (item) {
- is DataTreeItem.Node -> {
- writeDataDirectory(childPath, item.tree, format, envelopeFormat)
- }
-
- is DataTreeItem.Leaf -> {
- val envelope = item.data.toEnvelope(format)
- if (envelopeFormat != null) {
- writeEnvelopeFile(childPath, envelope, envelopeFormat)
- } else {
- writeEnvelopeDirectory(childPath, envelope)
- }
- }
- }
- }
- val treeMeta = tree.meta
- writeMetaFile(path, treeMeta)
- }
-}
-
-/**
- * Add file/directory-based data tree item
- *
- * @param ignoreExtensions a list of file extensions for which extension should be cut from the resulting item name
- */
-context(IOPlugin)
-@OptIn(DFInternal::class)
-@DFExperimental
-public fun <T : Any> DataSetBuilder<T>.file(
- path: Path,
- ignoreExtensions: Set<String> = DEFAULT_IGNORE_EXTENSIONS,
- formatResolver: FileFormatResolver<out T>,
-) {
-
- fun defaultPath() = if (path.extension in ignoreExtensions) path.nameWithoutExtension else path.name
-
- try {
- //If path is a single file or a special directory, read it as single datum
- if (!Files.isDirectory(path) || Files.list(path).allMatch { it.fileName.toString().startsWith("@") }) {
- val data = readDataFile(path, formatResolver)
- if (data == null) {
- logger.warn { "File format is not resolved for $path. Skipping." }
- return
- }
- val name: String = data.meta[Envelope.ENVELOPE_NAME_KEY].string ?: defaultPath()
- data(name.asName(), data)
- } else {
- //otherwise, read as directory
- val data: DataTree<T> = readDataDirectory(dataType, path, ignoreExtensions, formatResolver)
- val name = data.meta[Envelope.ENVELOPE_NAME_KEY].string ?: defaultPath()
- node(name.asName(), data)
- }
- } catch (ex: Exception) {
- logger.error { "Failed to read file or directory at $path: ${ex.message}" }
- }
-}
-
diff --git a/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/workspaceJvm.kt b/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/workspaceJvm.kt
index ea6ffb85..d9bc3e84 100644
--- a/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/workspaceJvm.kt
+++ b/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/workspaceJvm.kt
@@ -1,7 +1,6 @@
package space.kscience.dataforge.workspace
-import space.kscience.dataforge.data.DataSet
-import space.kscience.dataforge.data.filterByType
+import space.kscience.dataforge.data.DataTree
import space.kscience.dataforge.meta.Meta
import space.kscience.dataforge.misc.DFExperimental
import space.kscience.dataforge.names.Name
@@ -15,15 +14,14 @@ import space.kscience.dataforge.names.matches
* Select the whole data set from the workspace filtered by type.
*/
@OptIn(DFExperimental::class)
-public inline fun <reified T : Any> TaskResultBuilder<*>.dataByType(namePattern: Name? = null): DataSelector<T> =
- object : DataSelector<T> {
- override suspend fun select(workspace: Workspace, meta: Meta): DataSet<T> =
- workspace.data.filterByType { name, _ ->
- namePattern == null || name.matches(namePattern)
- }
+public inline fun <reified T : Any> TaskResultScope<*>.dataByType(namePattern: Name? = null): DataSelector<T> =
+ DataSelector<T> { workspace, _ ->
+ workspace.data.filterByType { name, _, _ ->
+ namePattern == null || name.matches(namePattern)
+ }
}
-public suspend inline fun <reified T : Any> TaskResultBuilder<*>.fromTask(
+public suspend inline fun <reified T : Any> TaskResultScope<*>.fromTask(
task: Name,
taskMeta: Meta = Meta.EMPTY,
-): DataSet<T> = workspace.produce(task, taskMeta).filterByType()
\ No newline at end of file
+): DataTree<T> = workspace.produce(task, taskMeta).filterByType()
\ No newline at end of file
diff --git a/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/writeFileData.kt b/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/writeFileData.kt
new file mode 100644
index 00000000..c65570df
--- /dev/null
+++ b/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/writeFileData.kt
@@ -0,0 +1,74 @@
+package space.kscience.dataforge.workspace
+
+import kotlinx.coroutines.Dispatchers
+import kotlinx.coroutines.withContext
+import space.kscience.dataforge.data.DataTree
+import space.kscience.dataforge.data.forEach
+import space.kscience.dataforge.data.meta
+import space.kscience.dataforge.io.*
+import space.kscience.dataforge.misc.DFExperimental
+import space.kscience.dataforge.names.Name
+import java.nio.file.Files
+import java.nio.file.Path
+import java.nio.file.spi.FileSystemProvider
+import kotlin.io.path.Path
+import kotlin.io.path.createDirectories
+import kotlin.io.path.exists
+import kotlin.io.path.extension
+
+
+/**
+ * Write the data tree to existing directory or create a new one using default [java.nio.file.FileSystem] provider
+ *
+ * @param nameToPath a [Name] to [Path] converter used to create
+ */
+@DFExperimental
+public suspend fun <T : Any> IOPlugin.writeDataDirectory(
+ path: Path,
+ dataSet: DataTree<T>,
+ format: IOWriter<T>,
+ envelopeFormat: EnvelopeFormat? = null,
+): Unit = withContext(Dispatchers.IO) {
+ if (!Files.exists(path)) {
+ Files.createDirectories(path)
+ } else if (!Files.isDirectory(path)) {
+ error("Can't write a node into file")
+ }
+ dataSet.forEach { data ->
+ val childPath = path.resolve(data.name.tokens.joinToString("/") { token -> token.toStringUnescaped() })
+ childPath.parent.createDirectories()
+ val envelope = data.toEnvelope(format)
+ if (envelopeFormat != null) {
+ writeEnvelopeFile(childPath, envelope, envelopeFormat)
+ } else {
+ writeEnvelopeDirectory(childPath, envelope)
+ }
+ }
+ dataSet.meta?.let { writeMetaFile(path, it) }
+
+}
+
+/**
+ * Write this [DataTree] as a zip archive
+ */
+@DFExperimental
+public suspend fun <T : Any> IOPlugin.writeZip(
+ path: Path,
+ dataSet: DataTree<T>,
+ format: IOWriter<T>,
+ envelopeFormat: EnvelopeFormat? = null,
+): Unit = withContext(Dispatchers.IO) {
+ if (path.exists()) error("Can't override existing zip data file $path")
+ val actualFile = if (path.extension == "zip") {
+ path
+ } else {
+ path.resolveSibling(path.fileName.toString() + ".zip")
+ }
+ val fsProvider = FileSystemProvider.installedProviders().find { it.scheme == "jar" }
+ ?: error("Zip file system provider not found")
+ //val fs = FileSystems.newFileSystem(actualFile, mapOf("create" to true))
+ val fs = fsProvider.newFileSystem(actualFile, mapOf("create" to true))
+ fs.use {
+ writeDataDirectory(fs.rootDirectories.first(), dataSet, format, envelopeFormat)
+ }
+}
\ No newline at end of file
diff --git a/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/zipData.kt b/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/zipData.kt
deleted file mode 100644
index 466552c2..00000000
--- a/dataforge-workspace/src/jvmMain/kotlin/space/kscience/dataforge/workspace/zipData.kt
+++ /dev/null
@@ -1,73 +0,0 @@
-package space.kscience.dataforge.workspace
-
-import kotlinx.coroutines.Dispatchers
-import kotlinx.coroutines.withContext
-import space.kscience.dataforge.data.DataTree
-import space.kscience.dataforge.data.DataTreeItem
-import space.kscience.dataforge.io.*
-import space.kscience.dataforge.misc.DFExperimental
-import java.nio.file.Files
-import java.nio.file.Path
-import java.nio.file.StandardOpenOption
-import java.util.zip.ZipEntry
-import java.util.zip.ZipOutputStream
-
-
-private suspend fun <T : Any> ZipOutputStream.writeNode(
- name: String,
- treeItem: DataTreeItem<T>,
- dataFormat: IOFormat<T>,
- envelopeFormat: EnvelopeFormat = TaggedEnvelopeFormat,
-): Unit = withContext(Dispatchers.IO) {
- when (treeItem) {
- is DataTreeItem.Leaf -> {
- //TODO add directory-based envelope writer
- val envelope = treeItem.data.toEnvelope(dataFormat)
- val entry = ZipEntry(name)
- putNextEntry(entry)
-
- //TODO remove additional copy
- val bytes = ByteArray {
- writeWith(envelopeFormat, envelope)
- }
- write(bytes)
-
- }
-
- is DataTreeItem.Node -> {
- val entry = ZipEntry("$name/")
- putNextEntry(entry)
- closeEntry()
- treeItem.tree.items.forEach { (token, item) ->
- val childName = "$name/$token"
- writeNode(childName, item, dataFormat, envelopeFormat)
- }
- }
- }
-}
-
-/**
- * Write this [DataTree] as a zip archive
- */
-@DFExperimental
-public suspend fun <T : Any> DataTree<T>.writeZip(
- path: Path,
- format: IOFormat<T>,
- envelopeFormat: EnvelopeFormat = TaggedEnvelopeFormat,
-): Unit = withContext(Dispatchers.IO) {
- val actualFile = if (path.toString().endsWith(".zip")) {
- path
- } else {
- path.resolveSibling(path.fileName.toString() + ".zip")
- }
- val fos = Files.newOutputStream(
- actualFile,
- StandardOpenOption.WRITE,
- StandardOpenOption.CREATE,
- StandardOpenOption.TRUNCATE_EXISTING
- )
- val zos = ZipOutputStream(fos)
- zos.use {
- it.writeNode("", DataTreeItem.Node(this@writeZip), format, envelopeFormat)
- }
-}
\ No newline at end of file
diff --git a/dataforge-workspace/src/jvmTest/kotlin/space/kscience/dataforge/workspace/CachingWorkspaceTest.kt b/dataforge-workspace/src/jvmTest/kotlin/space/kscience/dataforge/workspace/CachingWorkspaceTest.kt
index 4e1923bc..eb705e56 100644
--- a/dataforge-workspace/src/jvmTest/kotlin/space/kscience/dataforge/workspace/CachingWorkspaceTest.kt
+++ b/dataforge-workspace/src/jvmTest/kotlin/space/kscience/dataforge/workspace/CachingWorkspaceTest.kt
@@ -1,18 +1,16 @@
package space.kscience.dataforge.workspace
-import kotlinx.coroutines.ExperimentalCoroutinesApi
import kotlinx.coroutines.coroutineScope
import kotlinx.coroutines.test.runTest
import org.junit.jupiter.api.Test
-import space.kscience.dataforge.data.startAll
-import space.kscience.dataforge.data.static
+import space.kscience.dataforge.data.value
import space.kscience.dataforge.meta.Meta
import space.kscience.dataforge.meta.boolean
import space.kscience.dataforge.meta.get
import space.kscience.dataforge.misc.DFExperimental
import kotlin.test.assertEquals
-@OptIn(ExperimentalCoroutinesApi::class, DFExperimental::class)
+@OptIn(DFExperimental::class)
internal class CachingWorkspaceTest {
@Test
@@ -24,25 +22,24 @@ internal class CachingWorkspaceTest {
data {
//statically initialize data
repeat(5) {
- static("myData[$it]", it)
+ value("myData[$it]", it)
}
}
inMemoryCache()
val doFirst by task<Any> {
- pipeFrom(allData) { _, name, _ ->
+ transformEach(allData) {
firstCounter++
println("Done first on $name with flag=${taskMeta["flag"].boolean}")
}
}
- @Suppress("UNUSED_VARIABLE")
val doSecond by task<Any> {
- pipeFrom(
+ transformEach(
doFirst,
- dependencyMeta = if(taskMeta["flag"].boolean == true) taskMeta else Meta.EMPTY
- ) { _, name, _ ->
+ dependencyMeta = if (taskMeta["flag"].boolean == true) taskMeta else Meta.EMPTY
+ ) {
secondCounter++
println("Done second on $name with flag=${taskMeta["flag"].boolean ?: false}")
}
@@ -53,13 +50,15 @@ internal class CachingWorkspaceTest {
val secondA = workspace.produce("doSecond")
val secondB = workspace.produce("doSecond", Meta { "flag" put true })
val secondC = workspace.produce("doSecond")
+ //use coroutineScope to wait for the result
coroutineScope {
- first.startAll(this)
- secondA.startAll(this)
- secondB.startAll(this)
+ first.launchIn(this)
+ secondA.launchIn(this)
+ secondB.launchIn(this)
//repeat to check caching
- secondC.startAll(this)
+ secondC.launchIn(this)
}
+
assertEquals(10, firstCounter)
assertEquals(10, secondCounter)
}
diff --git a/dataforge-workspace/src/jvmTest/kotlin/space/kscience/dataforge/workspace/DataPropagationTest.kt b/dataforge-workspace/src/jvmTest/kotlin/space/kscience/dataforge/workspace/DataPropagationTest.kt
index 18086902..9cb040be 100644
--- a/dataforge-workspace/src/jvmTest/kotlin/space/kscience/dataforge/workspace/DataPropagationTest.kt
+++ b/dataforge-workspace/src/jvmTest/kotlin/space/kscience/dataforge/workspace/DataPropagationTest.kt
@@ -20,14 +20,12 @@ class DataPropagationTestPlugin : WorkspacePlugin() {
val result: Data<Int> = selectedData.foldToData(0) { result, data ->
result + data.value
}
- data("result", result)
+ result(result)
}
val singleData by task<Int> {
- workspace.data.filterByType<Int>()["myData[12]"]?.let {
- data("result", it)
- }
+ result(workspace.data.filterByType<Int>()["myData[12]"]!!)
}
@@ -47,7 +45,7 @@ class DataPropagationTest {
}
data {
repeat(100) {
- static("myData[$it]", it)
+ value("myData[$it]", it)
}
}
}
@@ -55,12 +53,12 @@ class DataPropagationTest {
@Test
fun testAllData() = runTest {
val node = testWorkspace.produce("Test.allData")
- assertEquals(4950, node.asSequence().single().await())
+ assertEquals(4950, node.content.data?.await())
}
@Test
fun testSingleData() = runTest {
val node = testWorkspace.produce("Test.singleData")
- assertEquals(12, node.asSequence().single().await())
+ assertEquals(12, node.content.data?.await())
}
}
\ No newline at end of file
diff --git a/dataforge-workspace/src/jvmTest/kotlin/space/kscience/dataforge/workspace/FileDataTest.kt b/dataforge-workspace/src/jvmTest/kotlin/space/kscience/dataforge/workspace/FileDataTest.kt
index 451c76f4..c9e5ea5a 100644
--- a/dataforge-workspace/src/jvmTest/kotlin/space/kscience/dataforge/workspace/FileDataTest.kt
+++ b/dataforge-workspace/src/jvmTest/kotlin/space/kscience/dataforge/workspace/FileDataTest.kt
@@ -1,6 +1,5 @@
package space.kscience.dataforge.workspace
-import kotlinx.coroutines.runBlocking
import kotlinx.coroutines.test.runTest
import kotlinx.io.Sink
import kotlinx.io.Source
@@ -9,38 +8,33 @@ import kotlinx.io.writeString
import space.kscience.dataforge.context.Context
import space.kscience.dataforge.context.Global
import space.kscience.dataforge.data.*
-import space.kscience.dataforge.io.Envelope
-import space.kscience.dataforge.io.IOFormat
-import space.kscience.dataforge.io.io
-import space.kscience.dataforge.io.readEnvelopeFile
+import space.kscience.dataforge.io.*
import space.kscience.dataforge.io.yaml.YamlPlugin
import space.kscience.dataforge.meta.get
import space.kscience.dataforge.misc.DFExperimental
import java.nio.file.Files
+import kotlin.io.path.deleteExisting
import kotlin.io.path.fileSize
import kotlin.io.path.toPath
-import kotlin.reflect.KType
-import kotlin.reflect.typeOf
import kotlin.test.Test
import kotlin.test.assertEquals
class FileDataTest {
- val dataNode = DataTree<String> {
+ val dataNode = DataTree.static<String> {
node("dir") {
- static("a", "Some string") {
+ value("a", "Some string") {
"content" put "Some string"
}
}
- static("b", "root data")
- meta {
- "content" put "This is root meta node"
- }
+ value("b", "root data")
+// meta {
+// "content" put "This is root meta node"
+// }
}
object StringIOFormat : IOFormat<String> {
- override val type: KType get() = typeOf<String>()
override fun writeTo(sink: Sink, obj: String) {
sink.writeString(obj)
@@ -51,29 +45,34 @@ class FileDataTest {
@Test
@DFExperimental
- fun testDataWriteRead() = with(Global.io) {
+ fun testDataWriteRead() = runTest {
+ val io = Global.io
val dir = Files.createTempDirectory("df_data_node")
- runBlocking {
- writeDataDirectory(dir, dataNode, StringIOFormat)
- println(dir.toUri().toString())
- val reconstructed = readDataDirectory(dir) { _, _ -> StringIOFormat }
- assertEquals(dataNode["dir.a"]?.meta?.get("content"), reconstructed["dir.a"]?.meta?.get("content"))
- assertEquals(dataNode["b"]?.await(), reconstructed["b"]?.await())
+ io.writeDataDirectory(dir, dataNode, StringIOFormat)
+ println(dir.toUri().toString())
+ val data = io.readDirectory(dir)
+ val reconstructed = data.transformEach(this) { (_, value) ->
+ value.toByteArray().decodeToString()
}
+ assertEquals(dataNode["dir.a"]?.meta?.get("content"), reconstructed["dir.a"]?.meta?.get("content"))
+ assertEquals(dataNode["b"]?.await(), reconstructed["b"]?.await())
}
@Test
@DFExperimental
fun testZipWriteRead() = runTest {
- with(Global.io) {
- val zip = Files.createTempFile("df_data_node", ".zip")
- dataNode.writeZip(zip, StringIOFormat)
- println(zip.toUri().toString())
- val reconstructed = readDataDirectory(zip) { _, _ -> StringIOFormat }
- assertEquals(dataNode["dir.a"]?.meta?.get("content"), reconstructed["dir.a"]?.meta?.get("content"))
- assertEquals(dataNode["b"]?.await(), reconstructed["b"]?.await())
+ val io = Global.io
+ val zip = Files.createTempFile("df_data_node", ".zip")
+ zip.deleteExisting()
+ io.writeZip(zip, dataNode, StringIOFormat)
+ println(zip.toUri().toString())
+ val reconstructed = io.readDirectory(zip).transformEach(this) { (_, value) ->
+ value.toByteArray().decodeToString()
}
+ assertEquals(dataNode["dir.a"]?.meta?.get("content"), reconstructed["dir.a"]?.meta?.get("content"))
+ assertEquals(dataNode["b"]?.await(), reconstructed["b"]?.await())
+
}
@OptIn(DFExperimental::class)
diff --git a/dataforge-workspace/src/jvmTest/kotlin/space/kscience/dataforge/workspace/FileWorkspaceCacheTest.kt b/dataforge-workspace/src/jvmTest/kotlin/space/kscience/dataforge/workspace/FileWorkspaceCacheTest.kt
index 00ca67cb..7aa1fb0e 100644
--- a/dataforge-workspace/src/jvmTest/kotlin/space/kscience/dataforge/workspace/FileWorkspaceCacheTest.kt
+++ b/dataforge-workspace/src/jvmTest/kotlin/space/kscience/dataforge/workspace/FileWorkspaceCacheTest.kt
@@ -3,12 +3,11 @@ package space.kscience.dataforge.workspace
import kotlinx.coroutines.ExperimentalCoroutinesApi
import kotlinx.coroutines.test.runTest
import org.junit.jupiter.api.Test
-import space.kscience.dataforge.data.startAll
-import space.kscience.dataforge.data.static
+import space.kscience.dataforge.data.value
import space.kscience.dataforge.misc.DFExperimental
import java.nio.file.Files
-@OptIn(ExperimentalCoroutinesApi::class,DFExperimental::class)
+@OptIn(ExperimentalCoroutinesApi::class, DFExperimental::class)
class FileWorkspaceCacheTest {
@Test
@@ -17,18 +16,17 @@ class FileWorkspaceCacheTest {
data {
//statically initialize data
repeat(5) {
- static("myData[$it]", it)
+ value("myData[$it]", it)
}
}
fileCache(Files.createTempDirectory("dataforge-temporary-cache"))
- @Suppress("UNUSED_VARIABLE")
val echo by task<String> {
- pipeFrom(dataByType<String>()) { arg, _, _ -> arg }
+ transformEach(dataByType<String>()) { value }
}
}
- workspace.produce("echo").startAll(this)
+ workspace.produce("echo").launchIn(this)
}
}
\ No newline at end of file
diff --git a/dataforge-workspace/src/jvmTest/kotlin/space/kscience/dataforge/workspace/SimpleWorkspaceTest.kt b/dataforge-workspace/src/jvmTest/kotlin/space/kscience/dataforge/workspace/SimpleWorkspaceTest.kt
index 7bfe0927..eb1c17b7 100644
--- a/dataforge-workspace/src/jvmTest/kotlin/space/kscience/dataforge/workspace/SimpleWorkspaceTest.kt
+++ b/dataforge-workspace/src/jvmTest/kotlin/space/kscience/dataforge/workspace/SimpleWorkspaceTest.kt
@@ -6,7 +6,6 @@ package space.kscience.dataforge.workspace
import kotlinx.coroutines.ExperimentalCoroutinesApi
import kotlinx.coroutines.runBlocking
import kotlinx.coroutines.test.runTest
-import org.junit.jupiter.api.Timeout
import space.kscience.dataforge.context.*
import space.kscience.dataforge.data.*
import space.kscience.dataforge.meta.*
@@ -16,6 +15,7 @@ import space.kscience.dataforge.names.plus
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertTrue
+import kotlin.time.Duration.Companion.milliseconds
/**
@@ -27,8 +27,8 @@ public fun <P : Plugin> P.toFactory(): PluginFactory<P> = object : PluginFactory
override val tag: PluginTag = this@toFactory.tag
}
-public fun Workspace.produceBlocking(task: String, block: MutableMeta.() -> Unit = {}): DataSet<Any> = runBlocking {
- produce(task, block)
+public fun Workspace.produceBlocking(task: String, block: MutableMeta.() -> Unit = {}): DataTree<*> = runBlocking {
+ produce(task, block).content
}
@OptIn(DFExperimental::class)
@@ -37,9 +37,9 @@ internal object TestPlugin : WorkspacePlugin() {
val test by task {
// type is inferred
- pipeFrom(dataByType<Int>()) { arg, _, _ ->
- logger.info { "Test: $arg" }
- arg
+ transformEach(dataByType<Int>()) {
+ logger.info { "Test: $value" }
+ value
}
}
@@ -62,42 +62,42 @@ internal class SimpleWorkspaceTest {
data {
//statically initialize data
repeat(100) {
- static("myData[$it]", it)
+ value("myData[$it]", it)
}
}
val filterOne by task<Int> {
val name by taskMeta.string { error("Name field not defined") }
- from(testPluginFactory) { test }.getByType<Int>(name)?.let { source ->
- data(source.name, source.map { it })
- }
+ result(from(testPluginFactory) { test }[name]!!)
}
val square by task<Int> {
- pipeFrom(dataByType<Int>()) { arg, name, meta ->
+ transformEach(dataByType<Int>()) {
if (meta["testFlag"].boolean == true) {
println("Side effect")
}
workspace.logger.info { "Starting square on $name" }
- arg * arg
+ value * value
}
}
val linear by task<Int> {
- pipeFrom(dataByType<Int>()) { arg, name, _ ->
+ transformEach(dataByType<Int>()) {
workspace.logger.info { "Starting linear on $name" }
- arg * 2 + 1
+ value * 2 + 1
}
}
val fullSquare by task<Int> {
val squareData = from(square)
val linearData = from(linear)
- squareData.forEach { data ->
- val newData: Data<Int> = data.combine(linearData[data.name]!!) { l, r ->
- l + r
+ result {
+ squareData.forEach { data ->
+ val newData: Data<Int> = data.combine(linearData[data.name]!!) { l, r ->
+ l + r
+ }
+ data(data.name, newData)
}
- data(data.name, newData)
}
}
@@ -106,23 +106,25 @@ internal class SimpleWorkspaceTest {
val res = from(square).foldToData(0) { l, r ->
l + r.value
}
- data("sum", res)
+ result(res)
}
val averageByGroup by task<Int> {
- val evenSum = workspace.data.filterByType<Int> { name, _ ->
+ val evenSum = workspace.data.filterByType<Int> { name, _, _ ->
name.toString().toInt() % 2 == 0
}.foldToData(0) { l, r ->
l + r.value
}
- data("even", evenSum)
- val oddSum = workspace.data.filterByType<Int> { name, _ ->
+ val oddSum = workspace.data.filterByType<Int> { name, _, _ ->
name.toString().toInt() % 2 == 1
}.foldToData(0) { l, r ->
l + r.value
}
- data("odd", oddSum)
+ result {
+ data("even", evenSum)
+ data("odd", oddSum)
+ }
}
val delta by task<Int> {
@@ -132,15 +134,17 @@ internal class SimpleWorkspaceTest {
val res = even.combine(odd) { l, r ->
l - r
}
- data("res", res)
+ result(res)
}
val customPipe by task<Int> {
- workspace.data.filterByType<Int>().forEach { data ->
- val meta = data.meta.toMutableMeta().apply {
- "newValue" put 22
+ result {
+ workspace.data.filterByType<Int>().forEach { data ->
+ val meta = data.meta.toMutableMeta().apply {
+ "newValue" put 22
+ }
+ data(data.name + "new", data.transform { (data.meta["value"].int ?: 0) + it })
}
- data(data.name + "new", data.map { (data.meta["value"].int ?: 0) + it })
}
}
@@ -148,18 +152,16 @@ internal class SimpleWorkspaceTest {
}
@Test
- @Timeout(1)
- fun testWorkspace() = runTest {
+ fun testWorkspace() = runTest(timeout = 200.milliseconds) {
val node = workspace.produce("sum")
- val res = node.asSequence().single()
- assertEquals(328350, res.await())
+ val res = node.data
+ assertEquals(328350, res?.await())
}
@Test
- @Timeout(1)
- fun testMetaPropagation() = runTest {
+ fun testMetaPropagation() = runTest(timeout = 200.milliseconds) {
val node = workspace.produce("sum") { "testFlag" put true }
- val res = node.asSequence().single().await()
+ val res = node.data?.await()
}
@Test
@@ -170,20 +172,25 @@ internal class SimpleWorkspaceTest {
}
@Test
- fun testFullSquare() {
- runBlocking {
- val node = workspace.produce("fullSquare")
- println(node.toMeta())
+ fun testFullSquare() = runTest {
+ val result = workspace.produce("fullSquare")
+ result.forEach {
+ println(
+ """
+ Name: ${it.name}
+ Meta: ${it.meta}
+ Data: ${it.await()}
+ """.trimIndent()
+ )
}
}
@Test
- fun testFilter() {
- runBlocking {
- val node = workspace.produce("filterOne") {
- "name" put "myData[12]"
- }
- assertEquals(12, node.single().await())
+ fun testFilter() = runTest {
+ val node = workspace.produce("filterOne") {
+ "name" put "myData[12]"
}
+ assertEquals(12, node.data?.await())
}
+
}
\ No newline at end of file
diff --git a/docs/templates/README-TEMPLATE.md b/docs/templates/README-TEMPLATE.md
index 762e5d7e..f868eb59 100644
--- a/docs/templates/README-TEMPLATE.md
+++ b/docs/templates/README-TEMPLATE.md
@@ -1,6 +1,69 @@
[](https://confluence.jetbrains.com/display/ALL/JetBrains+on+GitHub)
[](https://zenodo.org/badge/latestdoi/148831678)
-
+## Publications
+
+* [A general overview](https://doi.org/10.1051/epjconf/201817705003)
+* [An application in "Troitsk nu-mass" experiment](https://doi.org/10.1088/1742-6596/1525/1/012024)
+
+## Video
+
+* [A presentation on application of DataForge (legacy version) to Troitsk nu-mass analysis.](https://youtu.be/OpWzLXUZnLI?si=3qn7EMruOHMJX3Bc)
+
+## Questions and Answers
+
+In this section, we will try to cover DataForge main ideas in the form of questions and answers.
+
+### General
+
+**Q**: I have a lot of data to analyze. The analysis process is complicated, requires a lot of stages, and data flow is not always obvious. Also, the data size is huge, so I don't want to perform operation I don't need (calculate something I won't need or calculate something twice). I need it to be performed in parallel and probably on remote computer. By the way, I am sick and tired of scripts that modify other scripts that control scripts. Could you help me?
+
+**A**: Yes, that is precisely the problem DataForge was made to solve. It allows performing some automated data manipulations with optimization and parallelization. The important thing that data processing recipes are made in the declarative way, so it is quite easy to perform computations on a remote station. Also, DataForge guarantees reproducibility of analysis results.
+
+**Q**: How does it work?
+
+**A**: At the core of DataForge lies the idea of metadata processor. It utilizes the fact that to analyze something you need data itself and some additional information about what does that data represent and what does user want as a result. This additional information is called metadata and could be organized in a regular structure (a tree of values similar to XML or JSON). The important thing is that this distinction leaves no place for user instructions (or scripts). Indeed, the idea of DataForge logic is that one does not need imperative commands. The framework configures itself according to input meta-data and decides what operations should be performed in the most efficient way.
+
+**Q**: But where does it take algorithms to use?
+
+**A**: Of course algorithms must be written somewhere. No magic here. The logic is written in specialized modules. Some modules are provided out of the box at the system core, some need to be developed for a specific problem.
+
+**Q**: So I still need to write the code? What is the difference then?
+
+**A**: Yes, someone still needs to write the code. But not necessary you. Simple operations could be performed using provided core logic. Also, your group can have one programmer writing the logic and all other using it without any real programming expertise. The framework organized in a such way that one writes some additional logic, they do not need to think about complicated thing like parallel computing, resource handling, logging, caching, etc. Most of the things are done by the DataForge.
+
+### Platform
+
+**Q**: Which platform does DataForge use? Which operating system is it working on?
+
+**A**: The DataForge is mostly written in Kotlin-multiplatform and could be used on JVM, JS and native targets. Some modules and functions are supported only on JVM
+
+**Q**: Can I use my C++/Fortran/Python code in DataForge?
+
+**A**: Yes, as long as the code could be called from Java. Most common languages have a bridge for Java access. There are completely no problems with compiled C/Fortran libraries. Python code could be called via one of existing python-java interfaces. It is also planned to implement remote method invocation for common languages, so your Python, or, say, Julia, code could run in its native environment. The metadata processor paradigm makes it much easier to do so.
+
+### Features
+
+**Q**: What other features does DataForge provide?
+
+**A**: Alongside metadata processing (and a lot of tools for metadata manipulation and layering), DataForge has two additional important concepts:
+
+* **Modularisation**. Contrary to lot other frameworks, DataForge is intrinsically modular. The mandatory part is a rather tiny core module. Everything else could be customized.
+
+* **Context encapsulation**. Every DataForge task is executed in some context. The context isolates environment for the task and also works as dependency injection base and specifies interaction of the task with the external world.
+
+### Misc
+
+**Q**: So everything looks great, can I replace my ROOT / other data analysis framework with DataForge?
+
+**A**: One must note that DataForge is made for analysis, not for visualization. The visualization and user interaction capabilities of DataForge are rather limited compared to frameworks like ROOT, JAS3 or DataMelt. The idea is to provide reliable API and core functionality. [VisionForge](https://git.sciprog.center/kscience/visionforge) project aims to provide tools for both 2D and 3D visualization both locally and remotely.
+
+**Q**: How does DataForge compare to cluster computation frameworks like Apache Spark?
+
+**A**: It is not the purpose of DataForge to replace cluster computing software. DataForge has some internal parallelism mechanics and implementations, but they are most certainly worse than specially developed programs. Still, DataForge is not fixed on one single implementation. Your favourite parallel processing tool could be still used as a back-end for the DataForge. With full benefit of configuration tools, integrations and no performance overhead.
+
+**Q**: Is it possible to use DataForge in notebook mode?
+
+**A**: [Kotlin jupyter](https://github.com/Kotlin/kotlin-jupyter) allows using any JVM program in a notebook mode. The dedicated module for DataForge is work in progress.
${modules}
diff --git a/gradle.properties b/gradle.properties
index 31ef2f9a..015d2c52 100644
--- a/gradle.properties
+++ b/gradle.properties
@@ -1,10 +1,10 @@
+kotlin.code.style=official
+
org.gradle.parallel=true
org.gradle.jvmargs=-Xmx4096m
-kotlin.code.style=official
kotlin.mpp.stability.nowarn=true
-kotlin.incremental.js.ir=true
kotlin.native.ignoreDisabledTargets=true
+org.jetbrains.dokka.experimental.gradle.pluginMode=V2Enabled
-toolsVersion=0.15.1-kotlin-1.9.21
-#kotlin.experimental.tryK2=true
\ No newline at end of file
+toolsVersion=0.16.1-kotlin-2.1.0
\ No newline at end of file
diff --git a/gradle/wrapper/gradle-wrapper.properties b/gradle/wrapper/gradle-wrapper.properties
index e411586a..d6e308a6 100644
--- a/gradle/wrapper/gradle-wrapper.properties
+++ b/gradle/wrapper/gradle-wrapper.properties
@@ -1,5 +1,5 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
-distributionUrl=https\://services.gradle.org/distributions/gradle-8.4-bin.zip
+distributionUrl=https\://services.gradle.org/distributions/gradle-8.12-bin.zip
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
diff --git a/settings.gradle.kts b/settings.gradle.kts
index ca872038..35eae74e 100644
--- a/settings.gradle.kts
+++ b/settings.gradle.kts
@@ -43,6 +43,7 @@ include(
":dataforge-meta",
":dataforge-io",
":dataforge-io:dataforge-io-yaml",
+ ":dataforge-io:dataforge-io-proto",
":dataforge-context",
":dataforge-data",
":dataforge-workspace",