Skip to content

Conversation

@Jzjsnow
Copy link
Contributor

@Jzjsnow Jzjsnow commented Dec 19, 2025

Why are the changes needed?

Close #4018

Brief change log

  • Add an endpoint /api/ams/v1/refresh/catalog/{catalog}/database/{database}/table/{table} for triggering table refresh
  • Add an ExternalEventService to receive and process posted external events
  • Add support for specifying tables/catalogs that can trigger refreshes via events in TableRuntimeRefreshExecutor:
    • Two new variables added to TableRuntimeRefreshExecutor:
      • managedEventTriggerTables: All tables configured using event-triggered refreshes
      • pendingRefreshTables: Tables awaiting refresh execution in the next periodic schedule

In this PR, the event-triggered refresh process is as follows:

  • If a table is configured for event-triggered refresh (self-optimizing.refresh.event-triggered=true), its tableIdentifier is added to managedEventTriggerTables
  • If an event triggers refresh for a table in managedEventTriggerTables (e.g., table commit event or manual trigger), its tableIdentifier is added to pendingRefreshTables
  • During the next scheduled execution of the execute() method for the tableRuntime (at the fixed refresh interval refresh-tables.interval, default 1 minute), if the table remains in pendingRefreshTables, it undergoes loadTable() and evaluation processes before being removed from the list; otherwise, loadTable() and subsequent steps are skipped.
  • If the table remains untriggered by events and reaches the fallback interval (i.e., the time since the last refreshTime exceeds maxInterval), a forced refresh and subsequent operations are executed.

How was this patch tested?

  • Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • Add screenshots for manual tests if appropriate

  • Run test locally before making a pull request

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@Jzjsnow Jzjsnow force-pushed the add_support_for_ExternalEventService branch from 4b656b8 to cf42823 Compare December 19, 2025 13:32
@github-actions github-actions bot added the type:docs Improvements or additions to documentation label Dec 19, 2025
@Jzjsnow Jzjsnow changed the title [AMORO-4018] Add support for ExternalEventService [AMORO-4018] Add support for ExternalEventService to trigger table refresh Dec 19, 2025
@Jzjsnow Jzjsnow force-pushed the add_support_for_ExternalEventService branch from cf42823 to 5632660 Compare December 29, 2025 15:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module:ams-server Ams server module module:common type:build type:docs Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Subtask]: Add an ExternalEventService to support refreshing tableRuntime based on external events.

1 participant