This post was created using the information given in “Stream processing with Apache Flink”, the book written by Vasiliki Kalavri and Fabian Hueske.
There are few types of window operations that are most commonly used in stream processing. They are: tumbling windowing, sliding windowing, and session windowing. Today we will take a closer look at each of them.
Windowing in stream processing
Some operations in stream processing must collect and buffer records to deliver final results. Consider a join operation on an unbounded stream. In this case, we definitely need to limit the number of records this operation maintains.
The basic principle all window operations are based on is to create finite sets of records called buckets from the unbounded stream and to perform computations on these sets. Records are usually assigned to buckets based on their time or data properties. To define a window operator correctly we need to know how do the events assign to buckets and the period of how often the window operator pushes the result to the evaluation function that applies the computation logic on the bucket.
So let’s describe the semantics of window types:
Tumbling windows
Tumbling windows collect events into non-overlapping buckets of the fixed size. When the window border is passed all events from the bucket are sent to the evaluation function. Tumbling windows that are based on the events count trigger the evaluation after its size reaches some pre-defined value. Time-based tumbling windows define a time interval during which events are buffered in the bucket. Immediately after the period is over the events are sent to the evaluation.
Sliding windows
Another type of window in stream processing is the sliding window. Sliding windows assign events to overlap windows of fixed size. Therefore any event may belong to multiple buckets. Sliding windows are defined by providing length and slide. The sliding value shows an interval at which new buckets are created. In the example below the sliding count-based window has a length of four and a slide of three events.
Session windows
There are some use cases where neither tumbling nor sliding windows are applicable. A good example can be a case where we need to aggregate some events that originate from the same user session. These events can be separated by some arbitrary periods of inactivity. The length of the session is not pre-defined so sliding and tumbling windows obviously can not satisfy our requirements.
Practical usage
In practice, we usually combine different window policies to achieve a needed behavior. Therefore we usually use parallel windows to partition the full stream into a bunch of sub-streams.
