-
Notifications
You must be signed in to change notification settings - Fork 830
Open
Description
Now that we have a benchmarks branch that will run nightly, we can build upon this. It is not uncommon for someone to introduce a change, and for us to want to validate the benchmarks before merging. We should create automation where you can add a tag to a PR, and benchmarks will run, followed by a bot that will publish a comment on the PR with the results and much they deviate from the baseline on the benchmarks branch.
We will have to see how the final results look in terms of how accurate they will be, since we don't have much control over the hosts running the tests.
Requirements:
- When a "Run benchmarks" label is added to a PR, a github action job is triggered
- The action will run the "ci" benchmark job for the particular branch
- The action will diff the results with the latest info on the
benchmarksbranch - The action will leave a comment on the PR with a summary of the results and their delta compared to the baseline
zeitlinger
Metadata
Metadata
Assignees
Labels
No labels