Call remote tasks of service workspace #75
@ -1,9 +1,18 @@
|
||||
package space.kscience.dataforge.data
|
||||
|
||||
import kotlinx.coroutines.coroutineScope
|
||||
import kotlinx.coroutines.flow.collect
|
||||
import space.kscience.dataforge.misc.DFExperimental
|
||||
import space.kscience.dataforge.names.*
|
||||
import space.kscience.dataforge.names.Name
|
||||
import space.kscience.dataforge.names.NameToken
|
||||
import space.kscience.dataforge.names.asName
|
||||
import space.kscience.dataforge.names.cutFirst
|
||||
import space.kscience.dataforge.names.cutLast
|
||||
import space.kscience.dataforge.names.firstOrNull
|
||||
import space.kscience.dataforge.names.isEmpty
|
||||
import space.kscience.dataforge.names.lastOrNull
|
||||
import space.kscience.dataforge.names.length
|
||||
import space.kscience.dataforge.names.plus
|
||||
import kotlin.collections.set
|
||||
import kotlin.reflect.KType
|
||||
import kotlin.reflect.typeOf
|
||||
|
||||
@ -61,6 +70,12 @@ internal class StaticDataTree<T : Any>(
|
||||
}
|
||||
}
|
||||
|
||||
@Suppress("FunctionName")
|
||||
public fun <T : Any> DataTree(dataType: KType): DataTree<T> = StaticDataTree(dataType)
|
||||
|
||||
@Suppress("FunctionName")
|
||||
public inline fun <reified T : Any> DataTree(): DataTree<T> = DataTree(typeOf<T>())
|
||||
|
||||
@Suppress("FunctionName")
|
||||
public suspend fun <T : Any> DataTree(
|
||||
dataType: KType,
|
||||
@ -73,6 +88,6 @@ public suspend inline fun <reified T : Any> DataTree(
|
||||
): DataTree<T> = DataTree(typeOf<T>(), block)
|
||||
|
||||
@OptIn(DFExperimental::class)
|
||||
public suspend fun <T : Any> DataSet<T>.seal(): DataTree<T> = DataTree(dataType){
|
||||
public suspend fun <T : Any> DataSet<T>.seal(): DataTree<T> = DataTree(dataType) {
|
||||
populate(this@seal)
|
||||
}
|
@ -1,12 +1,10 @@
|
||||
|
||||
package space.kscience.dataforge.distributed
|
||||
|
||||
import io.ktor.utils.io.core.*
|
||||
import io.lambdarpc.coding.coders.JsonCoder
|
||||
DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now. DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now.
Need to think about better naming Need to think about better naming
Why not suspended? Why not suspended?
I thought about I thought about `WorkspaceNode` or `WorkerWorkspace`. There is also `DistributedWorkspace` but it is not truly distributed itself.
Also `ServiceWorkpace` is good enought to my opinion. "Service" here means that this workspace should be run on some endpoint to be available.
Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so. Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so.
|
||||
import io.lambdarpc.dsl.LibService
|
||||
import io.lambdarpc.dsl.def
|
||||
import io.lambdarpc.dsl.j
|
||||
import io.lambdarpc.utils.ServiceId
|
||||
import kotlinx.coroutines.runBlocking
|
||||
DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now. DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now.
Need to think about better naming Need to think about better naming
Why not suspended? Why not suspended?
I thought about I thought about `WorkspaceNode` or `WorkerWorkspace`. There is also `DistributedWorkspace` but it is not truly distributed itself.
Also `ServiceWorkpace` is good enought to my opinion. "Service" here means that this workspace should be run on some endpoint to be available.
Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so. Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so.
|
||||
import kotlinx.serialization.KSerializer
|
||||
import space.kscience.dataforge.context.Context
|
||||
import space.kscience.dataforge.context.Global
|
||||
@ -32,7 +30,7 @@ public class ServiceWorkspace(
|
||||
DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now. DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now.
DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now. DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now.
Need to think about better naming Need to think about better naming
Need to think about better naming Need to think about better naming
Why not suspended? Why not suspended?
Why not suspended? Why not suspended?
I thought about I thought about `WorkspaceNode` or `WorkerWorkspace`. There is also `DistributedWorkspace` but it is not truly distributed itself.
Also `ServiceWorkpace` is good enought to my opinion. "Service" here means that this workspace should be run on some endpoint to be available.
I thought about I thought about `WorkspaceNode` or `WorkerWorkspace`. There is also `DistributedWorkspace` but it is not truly distributed itself.
Also `ServiceWorkpace` is good enought to my opinion. "Service" here means that this workspace should be run on some endpoint to be available.
Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so. Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so.
Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so. Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so.
|
||||
port: Int? = null,
|
||||
override val context: Context = Global.buildContext("workspace".asName()),
|
||||
private val dataSerializer: KSerializer<Any>? = null,
|
||||
data: DataSet<*> = runBlocking { DataTree<Any> {} },
|
||||
DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now. DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now.
Need to think about better naming Need to think about better naming
Why not suspended? Why not suspended?
I thought about I thought about `WorkspaceNode` or `WorkerWorkspace`. There is also `DistributedWorkspace` but it is not truly distributed itself.
Also `ServiceWorkpace` is good enought to my opinion. "Service" here means that this workspace should be run on some endpoint to be available.
Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so. Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so.
|
||||
data: DataSet<*> = DataTree<Any>(),
|
||||
DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now. DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now.
Need to think about better naming Need to think about better naming
Why not suspended? Why not suspended?
I thought about I thought about `WorkspaceNode` or `WorkerWorkspace`. There is also `DistributedWorkspace` but it is not truly distributed itself.
Also `ServiceWorkpace` is good enought to my opinion. "Service" here means that this workspace should be run on some endpoint to be available.
Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so. Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so.
|
||||
override val targets: Map<String, Meta> = mapOf(),
|
||||
) : Workspace, Closeable {
|
||||
private val _port: Int? = port
|
||||
@ -113,9 +111,6 @@ public class ServiceWorkspace(
|
||||
DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now. DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now.
DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now. DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now.
Need to think about better naming Need to think about better naming
Need to think about better naming Need to think about better naming
Why not suspended? Why not suspended?
Why not suspended? Why not suspended?
I thought about I thought about `WorkspaceNode` or `WorkerWorkspace`. There is also `DistributedWorkspace` but it is not truly distributed itself.
Also `ServiceWorkpace` is good enought to my opinion. "Service" here means that this workspace should be run on some endpoint to be available.
I thought about I thought about `WorkspaceNode` or `WorkerWorkspace`. There is also `DistributedWorkspace` but it is not truly distributed itself.
Also `ServiceWorkpace` is good enought to my opinion. "Service" here means that this workspace should be run on some endpoint to be available.
Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so. Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so.
Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so. Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so.
|
||||
|
||||
public companion object {
|
||||
internal val serviceId = ServiceId("d41b95b1-828b-4444-8ff0-6f9c92a79246")
|
||||
internal val execute by serviceId.def(
|
||||
DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now. DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now.
Need to think about better naming Need to think about better naming
Why not suspended? Why not suspended?
I thought about I thought about `WorkspaceNode` or `WorkerWorkspace`. There is also `DistributedWorkspace` but it is not truly distributed itself.
Also `ServiceWorkpace` is good enought to my opinion. "Service" here means that this workspace should be run on some endpoint to be available.
Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so. Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so.
|
||||
JsonCoder(Name.serializer()), JsonCoder(MetaSerializer), j<TaskRegistry>(),
|
||||
DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now. DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now.
Need to think about better naming Need to think about better naming
Why not suspended? Why not suspended?
I thought about I thought about `WorkspaceNode` or `WorkerWorkspace`. There is also `DistributedWorkspace` but it is not truly distributed itself.
Also `ServiceWorkpace` is good enought to my opinion. "Service" here means that this workspace should be run on some endpoint to be available.
Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so. Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so.
|
||||
j<DataSetPrototype>()
|
||||
DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now. DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now.
Need to think about better naming Need to think about better naming
Why not suspended? Why not suspended?
I thought about I thought about `WorkspaceNode` or `WorkerWorkspace`. There is also `DistributedWorkspace` but it is not truly distributed itself.
Also `ServiceWorkpace` is good enought to my opinion. "Service" here means that this workspace should be run on some endpoint to be available.
Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so. Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so.
|
||||
)
|
||||
DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now. DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now.
Need to think about better naming Need to think about better naming
Why not suspended? Why not suspended?
I thought about I thought about `WorkspaceNode` or `WorkerWorkspace`. There is also `DistributedWorkspace` but it is not truly distributed itself.
Also `ServiceWorkpace` is good enought to my opinion. "Service" here means that this workspace should be run on some endpoint to be available.
Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so. Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so.
|
||||
internal val execute by serviceId.def(j<Name>(), j(MetaSerializer), j<TaskRegistry>(), j<DataSetPrototype>())
|
||||
DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now. DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now.
Need to think about better naming Need to think about better naming
Why not suspended? Why not suspended?
I thought about I thought about `WorkspaceNode` or `WorkerWorkspace`. There is also `DistributedWorkspace` but it is not truly distributed itself.
Also `ServiceWorkpace` is good enought to my opinion. "Service" here means that this workspace should be run on some endpoint to be available.
Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so. Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so.
|
||||
}
|
||||
}
|
||||
|
||||
DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now. DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now.
DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now. DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now.
Need to think about better naming Need to think about better naming
Need to think about better naming Need to think about better naming
Why not suspended? Why not suspended?
Why not suspended? Why not suspended?
I thought about I thought about `WorkspaceNode` or `WorkerWorkspace`. There is also `DistributedWorkspace` but it is not truly distributed itself.
Also `ServiceWorkpace` is good enought to my opinion. "Service" here means that this workspace should be run on some endpoint to be available.
I thought about I thought about `WorkspaceNode` or `WorkerWorkspace`. There is also `DistributedWorkspace` but it is not truly distributed itself.
Also `ServiceWorkpace` is good enought to my opinion. "Service" here means that this workspace should be run on some endpoint to be available.
Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so. Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so.
Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so. Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so.
|
@ -24,15 +24,13 @@ internal class RemoteCallTest {
|
||||
switch to runTest switch to runTest
switch to runTest switch to runTest
This function is not available in dataforge, shall I add some dependency to the project? This function is not available in dataforge, shall I add some dependency to the project?
This function is not available in dataforge, shall I add some dependency to the project? This function is not available in dataforge, shall I add some dependency to the project?
|
||||
private lateinit var workspace: Workspace
|
||||
|
||||
@BeforeAll
|
||||
fun before() {
|
||||
switch to runTest switch to runTest
This function is not available in dataforge, shall I add some dependency to the project? This function is not available in dataforge, shall I add some dependency to the project?
|
||||
fun before() = runBlocking {
|
||||
switch to runTest switch to runTest
This function is not available in dataforge, shall I add some dependency to the project? This function is not available in dataforge, shall I add some dependency to the project?
|
||||
worker1 = ServiceWorkspace(
|
||||
context = Global.buildContext("worker1".asName()) {
|
||||
This is outdated, use This is outdated, use `Context()` factory function instead.
|
||||
plugin(MyPlugin1)
|
||||
},
|
||||
data = runBlocking {
|
||||
switch to runTest switch to runTest
This function is not available in dataforge, shall I add some dependency to the project? This function is not available in dataforge, shall I add some dependency to the project?
|
||||
DataTree<Any> {
|
||||
switch to runTest switch to runTest
This function is not available in dataforge, shall I add some dependency to the project? This function is not available in dataforge, shall I add some dependency to the project?
|
||||
static("int", 42)
|
||||
switch to runTest switch to runTest
This function is not available in dataforge, shall I add some dependency to the project? This function is not available in dataforge, shall I add some dependency to the project?
|
||||
}
|
||||
switch to runTest switch to runTest
This function is not available in dataforge, shall I add some dependency to the project? This function is not available in dataforge, shall I add some dependency to the project?
|
||||
data = DataTree<Any> {
|
||||
switch to runTest switch to runTest
This function is not available in dataforge, shall I add some dependency to the project? This function is not available in dataforge, shall I add some dependency to the project?
|
||||
static("int", 42)
|
||||
switch to runTest switch to runTest
This function is not available in dataforge, shall I add some dependency to the project? This function is not available in dataforge, shall I add some dependency to the project?
|
||||
},
|
||||
)
|
||||
worker1.start()
|
||||
|
||||
switch to runTest switch to runTest
switch to runTest switch to runTest
This function is not available in dataforge, shall I add some dependency to the project? This function is not available in dataforge, shall I add some dependency to the project?
This function is not available in dataforge, shall I add some dependency to the project? This function is not available in dataforge, shall I add some dependency to the project?
|
DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now.
DataTree builder will be non-suspending in the next version, but it should be possible to create it without suspending now.
Need to think about better naming
Need to think about better naming
Why not suspended?
Why not suspended?
I thought about
WorkspaceNode
orWorkerWorkspace
. There is alsoDistributedWorkspace
but it is not truly distributed itself.Also
ServiceWorkpace
is good enought to my opinion. "Service" here means that this workspace should be run on some endpoint to be available.I thought about
WorkspaceNode
orWorkerWorkspace
. There is alsoDistributedWorkspace
but it is not truly distributed itself.Also
ServiceWorkpace
is good enought to my opinion. "Service" here means that this workspace should be run on some endpoint to be available.Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so.
Because it is blocking in the google gRPC implementation. It can be made suspend and block separate thread but I do not see any reasons to do so.