trait BatchStream[+I, S, C] extends AnyRef
A functional stateful stream of items
lamp's training loops work from data presented in BatchStreams.
An instance of BatchStream is an description of the data stream, it does not by itself allocates or stores any data. The stream needs to be driven by an interpreter. lamp.data.IOLoops and the companion object BatchStream contain those interpreters to make something useful with a BatchStream.
See the abstract members and the companion object for more documentation.
- I
the item type , the stream will yield items of this type
- S
the state type, the stream will carry over and accumulate state of this type
- C
type of accessory resources (e.g. buffers), the stream might need an instance of this type for its working. The intended use for fixed, pre-allocated pinned buffer pairs to facilitate host-device copies. See lamp.Device.toBatched and lamp.BufferPair.
- Self Type
- BatchStream[I, S, C]
- Alphabetic
- By Inheritance
- BatchStream
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Abstract Value Members
- abstract def allocateBuffers(target: Device): Resource[IO, C]
Allocation of a resource needed during the lifetime of the stream The intended use is for transfer buffer pairs
- abstract def init: S
Initial value of the State
- abstract def nextBatch(device: Device, buffers: C, state: S): IO[(S, Resource[IO, StreamControl[I]])]
Returns the resource of the next item and the next state suspended in an effect.
Returns the resource of the next item and the next state suspended in an effect.
Returned values are wrapped in StreamControl type which signals the interpreter whether the stream is finished.
May be called from different threads, but always in serial State should be carried over in the state parameter and return type
Concrete Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- def concat[I2 >: I, S2](other: BatchStream[I2, S2, C]): BatchStream[I2, (Boolean, Either[S, S2]), C] { def init: (Boolean, scala.util.Left[S,Nothing]) }
Returns a stream which is the concatenation of this stream and an other.
- def drainIntoSeq(device: Device): Resource[IO, Vector[I]]
Drives the stream and returns all the items from it in a scala collection.
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def everyNth(n: Int, offset: Int): BatchStream[I, (Long, S), C]
Takes only batches where (i % n == offset), i being the number of batch counted from 0
- def foldLeft[B](zero: B, device: Device, stateZero: S)(f: (B, I) => IO[B]): IO[B]
Folds a function from an initial value over this stream
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def map[I2](f: (I) => Resource[IO, StreamControl[I2]]): BatchStream[I2, S, C]
Maps f over the elements of this stream
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- def repeatOrTake(requiredLength: Long): BatchStream[I, (Long, S), C] { val init0: S }
ensures the length of the stream is fixed.
ensures the length of the stream is fixed. Either repeats an element or truncates
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def take(n: Long): BatchStream[I, (Long, S), C]
Returns a stream wich contains only the first n elements of this stream
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- def withoutEmptyBatches: BatchStream[I, S, C]
Returns a new stream with EmptyBatches filtered out
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)