Support rate family aggregation functions in table model - #18334
Open
CoollZzz wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Related issue: #17976
This PR adds four built-in aggregation functions to the IoTDB table model:
rate(),increase(),irate(), anddelta(). Their calculation semantics are based on the corresponding Prometheus range-vector functions, while window boundaries follow IoTDB's[window_start, window_end)convention.Function semantics
raterate(value, time, window_start, window_end)increaseincrease(value, time, window_start, window_end)irateirate(value, time)deltadelta(value, time, window_start, window_end)The supported value types are
INT32,INT64,FLOAT, andDOUBLE. Time and window arguments acceptTIMESTAMPandINT64, withINT64interpreted using the currenttimestamp_precision. All four functions returnDOUBLE.rate(),increase(), anddelta()use all valid samples in the aggregation group and extrapolate the result to the window boundaries. Extrapolation uses the average sampling interval, a1.1threshold for distant boundaries, and counter zero-point protection where applicable.irate()only uses the last two valid samples and does not perform boundary extrapolation.Accumulator and planner design
Both
TableAccumulatorandGroupedAccumulatorexecution paths are supported.Each function has its own abstract accumulator class for shared function-specific validation, window handling, and result calculation. Ordered and naive accumulators are implemented as separate concrete classes because their state structures and lifecycle behavior differ significantly.
SINGLEaggregation step when the physical planner proves that the function'stimeargument is ascending within every SQL aggregation group. They consume samples incrementally without buffering and sorting all input rows.BLOBintermediate state containing the complete sample state and window boundaries.The ordered-input property is propagated through
AggregationNodeserialization so that distributed execution selects the same accumulator implementation as the coordinator plan.Validation and edge cases
The implementation handles the following cases:
NULLvalues are ignored.NULL.rate(),increase(), andirate().delta().NaNand positive or negative infinity, are rejected.NULLfor a valid sample.window_start < window_end.window_start <= time < window_end.Tests
Integration tests were added to
IoTDBTableAggregationITfor:The four target test methods were verified with both the
TableSimpleITandTableClusterITprofiles. The full reactor test compilation also passed for both the default and Chinese locales.This PR has:
Key changed/added classes (or packages if there are too many classes) in this PR
org.apache.iotdb.calc.execution.operator.source.relational.aggregation.rateorg.apache.iotdb.calc.execution.operator.source.relational.aggregation.grouped.rateAccumulatorFactoryTableMetadataImplTableDistributedPlanGeneratorAggregationNodePushAggregationIntoTableScanIoTDBTableAggregationIT