With this, we should be able to set the image tag (`latest` or `nightly`
for example), to be used for the TED run for Cypress tests.
Unless we already have a different method to run Cypress with nightly
TED, this should give us that flexibility.
In testing we saw that updating `ci-test-result` has a chance of failing
if the correct check is not found.
When we are creating the `ci-test-result` check via the Github Api, it
successfully marked the PR check with the result. It did not seem to
have any issue even if a `ci-test-result` job already existed
By removing the logic of finding an existing check of the same name, and
always creating the check, we assume this will improve the PR check
updates
fixes: #31601
The current `TestResultLoggerExtension` extension only finds failed
tests from the server module, and not from any of the plugin or other
modules.
To fix this, this PR collects failed tests using the Surefire XML test
reports already generated by the `mvn test` command.
## Description
- This PR does some clean up of comments - in DeployHelper.ts
- This PR replaced external image api with TED mock api in
ServerSide/OnLoadTests/OnLoadActions_Spec.ts
- Extrenal example api to TED in Widgets/Image/Image_spec.js
- Extrenal api links in
app/client/cypress/e2e/Regression/ServerSide/ApiTests/API_Bugs_Spec.js
to TED api's
- Flaky fixes /ServerSide/LoginTests/LoginFailure_spec.js - for EE CI
run failure (Added multiple Dynamic checks)
- support command.js method LoginUser() - added Dynamic check
- Encapsulated Datasources/Arango_Basic_Spec.ts into conditional check
to run in CE alone, instead of entirely skipping, since its failing in
EE due to chrome crash
#### Media
#### Type of change
- Script fix (non-breaking change which fixes an issue)
## Testing
#### How Has This Been Tested?
- [X] Cypress CI runs
## Checklist:
#### QA activity:
- [X] Added `Test Plan Approved` label after Cypress tests were reviewed
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Tests**
- Updated Cypress regression tests to use flower images instead of cat
images for API calls.
- Modified limited test configurations to run with different spec files.
- Adjusted GitHub Actions workflow to change `matrix_count` values based
on the event type and retrieve PR number for manual triggers.
- Updated Docker image tag in CI tests to `nightly`.
- **Refactor**
- Enhanced `DataManager` class with new properties for flower image
URLs.
- Improved `EntityExplorer` class method to ensure more reliable
triggering of events.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Description
#### PR fixes following issue(s)
Fixes # (issue number)
#### Media
#### Type of change
- New feature (non-breaking change which adds functionality)
## Testing
#### How Has This Been Tested?
- [X] Manual
- [ ] JUnit
- [ ] Jest
- [ ] Cypress
#### Test Plan
## Checklist:
#### Dev activity
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] PR is being merged under a feature flag
#### QA activity:
- [ ] [Speedbreak
features](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#speedbreakers-)
have been covered
- [ ] Test plan covers all impacted features and [areas of
interest](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#areas-of-interest-)
- [ ] Test plan has been peer reviewed by project stakeholders and other
QA members
- [ ] Manually tested functionality on DP
- [ ] We had an implementation alignment call with stakeholders post QA
Round 2
- [ ] Cypress test cases have been added and approved by SDET/manual QA
- [ ] Added `Test Plan Approved` label after Cypress tests were reviewed
- [ ] Added `Test Plan Approved` label after JUnit tests were reviewed
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Introduced a new step in the CI/CD pipeline to trigger an automated
analysis workflow.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This should show the failed test list in the GitHub actions summary so
we don't have to load the _extremely large_ full log file of the server
tests, just to see the list of tests that failed.
Show the commit SHA in the version popup, instead of a snapshot version
number like `v1.11-SNAPSHOT`. But if the version number doesn't have a
`-SNAPSHOT` at the end, we show the version number as is. So if it's
`v1.12`, we show that instead of the commit SHA.
## Description
> Modified the message thrown when no tests ran from
`It seems like there are some failures 😔. We are not able to recognize
it, please check this manually`
to
` It seems like no tests ran 😔. We are not able to recognize it, please
check workflow`
#### Type of change
- Chore (housekeeping or task changes that don't impact user perception)
#### How Has This Been Tested?
> Please describe the tests that you ran to verify your changes. Also
list any relevant details for your test configuration.
> Delete anything that is not relevant
- [ ] Manual
## Checklist:
#### Dev activity
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] PR is being merged under a feature flag
#### QA activity:
- [ ] [Speedbreak
features](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#speedbreakers-)
have been covered
- [ ] Test plan covers all impacted features and [areas of
interest](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#areas-of-interest-)
- [ ] Test plan has been peer reviewed by project stakeholders and other
QA members
- [ ] Manually tested functionality on DP
- [ ] We had an implementation alignment call with stakeholders post QA
Round 2
- [ ] Cypress test cases have been added and approved by SDET/manual QA
- [ ] Added `Test Plan Approved` label after Cypress tests were reviewed
- [ ] Added `Test Plan Approved` label after JUnit tests were reviewed
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Improved messaging for test failure scenarios to clarify when no tests
have been run.
- Updated the link for checking the workflow status.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixing minor syntax issues when notifying on Slack on failure of the Test build Docker workflow.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Improved Slack notification messages by adjusting link formatting.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Description
- This PR includes the flaky fixes of the below specs:
- jsEditor.NavigateToNewJSEditor() using TS helpers (Addressing
flakyness in specs /BugTests/JS_Bug29131_spec.ts,
BugTests/invalidLintError_Spec.ts,
ClientSide/BugTests/JS_Bug28764_Spec.ts,
/ClientSide/BugTests/Bug29566_Spec.ts,
ServerSide/JsFunctionExecution/JSFunctionExecution_spec.ts)
- /Widgets/ListV2/Listv2_onItemClick_spec.js (toast validation updated
to ts helper)
- /Apps/MongoDBShoppingCart_spec.ts (Flaky fix 2nd case)
- /Regression/Apps/PromisesApp_spec.js (1st test)
- /ClientSide/OtherUIFeatures/ApplicationURL_spec.js (whole spec for
static wait removal + flaky fixes)
- /ClientSide/Templates/Fork_Template_spec.js (complete spec fixes,
cypress exception fix)
- /Sanity/Datasources/Arango_Basic_Spec.ts (3rd test)
- /ServerSide/OnLoadTests/JSOnLoad2_Spec.ts (8th case flaky fix)
- /ClientSide/MobileResponsiveTests/AutoFillWidgets_Reflow_spec.ts (2nd
testcase, flow update)
- /ClientSide/OtherUIFeatures/Resize_spec.js (import update)
- /ClientSide/Templates/Fork_Template_spec.js (1st & 3rd for flaky fix,
removed 2nd - redundant check)
- /Git/GitSync/RepoLimitExceededErrorModal_spec.js flaky fix for EE repo
failure
- /ServerSide/ApiTests/API_Bugs_Spec.js (3rd case, removed waits, moved
to TED Api)
- /Apps/EchoApiCMS_spec.js (removed redundant action)
- /AppNavigation/Sidebar_spec.ts, /AppNavigation/TopInline_spec.ts,
/AppNavigation/TopStacked_spec.ts - added refresh calls for CI flakyness
- homePage.RenameWorkspace() & homePage.NavigateToHome() &
homePage.Signout() - removed static wait, added dynamic check
- homePage.CreateNewApplication() - removed static wait
- homePage.CreateAppInWorkspace() - removed commented code
- homePage.CreateNewWorkspace() - logic improved
- agHelper.AssertURL() - added timeout, removed static wait, calling
assertHelper.AssertDocumentReady()
- Few of agHelper methods timeout & error mesg text update
- cy.Signup() - updated validations
- agHelper.CheckForErrorToast() to agHelper.FailIfErrorToast() - method
name
- /ClientSide/BugTests/DSDiscardBugs_spec.ts - removed unnecessary
method call
#### Type of change
- Script fix (non-breaking change which fixes an issue)
## Testing
#### How Has This Been Tested?
- [X] Cypress CI runs
## Checklist:
#### QA activity:
- [X] Added `Test Plan Approved` label after all changes were reviewed
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Refactor**
- Enhanced timeout handling across various helper methods for improved
stability.
- Streamlined interactions with UI elements using updated helper
methods.
- Adjusted test suites to utilize configuration values for consistency.
- **Tests**
- Temporarily disabled specific test suites to address current issues.
- Updated test assertions and interaction patterns for clarity and
reliability.
- Added new test cases and assertions to cover additional
functionalities.
- **Chores**
- Updated workflow variables for more efficient CI processes.
- Curated the list of limited tests to focus on critical test scenarios.
- **Documentation**
- Clarified test descriptions and intentions for better understanding.
- **Bug Fixes**
- Fixed UI element interactions to prevent test flakiness and improve
user experience.
- Implemented page refresh before application imports to ensure clean
state for tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Updated GitHub workflow to include PostgreSQL client installation
steps.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Description
- Scheduled TBP workflow to run 4 times a day with 3hrs interval
- Made changes in TBP workflow to run all the steps only while running
with schedule or manual workflow trigger
- Added condition to run ci-test and ci-test-result only during schedule
or manual workflow run.
#### Type of change
- Workflow changes
## Testing
>
#### How Has This Been Tested?
- [x] CI Run
>
>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Updated the workflow schedule for Docker image build tests to run at
new times: 7:00 AM, 10:00 AM, 1:00 PM, and 4:00 PM IST on weekdays.
- Modified conditional triggers in the CI pipeline to exclude jobs from
running on push events.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Description
> Added: **step** - Add suggestion in the PR on using @tag.All in
workflow `.github/workflows/integration-tests-command.yml`
> This will post a comment in the PR to use specific tags.
#### Type of change
- Chore (housekeeping or task changes that don't impact user perception)
## Testing
> tested with tbp
#### How Has This Been Tested?
- [ ] Manual
## Checklist:
#### Dev activity
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] PR is being merged under a feature flag
#### QA activity:
- [ ] [Speedbreak
features](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#speedbreakers-)
have been covered
- [ ] Test plan covers all impacted features and [areas of
interest](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#areas-of-interest-)
- [ ] Test plan has been peer reviewed by project stakeholders and other
QA members
- [ ] Manually tested functionality on DP
- [ ] We had an implementation alignment call with stakeholders post QA
Round 2
- [ ] Cypress test cases have been added and approved by SDET/manual QA
- [ ] Added `Test Plan Approved` label after Cypress tests were reviewed
- [ ] Added `Test Plan Approved` label after JUnit tests were reviewed
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Integration tests now suggest using specific tags for more efficient
testing.
- Docker image build tests prompt for using `@tag.All` when appropriate,
with a link to documentation for guidance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix server test rerun
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Updated workflow to ensure test extraction step runs when certain
tests fail.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Description
> Added step to check if @tag.All mentioned is as expected, if not throw
a message to use `/ok-to-test tags="@tag.All"` and exit the run
#### Type of change
- Chore (housekeeping or task changes that don't impact user perception)
## Testing
> TBP with tags
#### How Has This Been Tested?
- [x] Manual
- [x] TBP
>
## Checklist:
#### Dev activity
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] PR is being merged under a feature flag
#### QA activity:
- [ ] [Speedbreak
features](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#speedbreakers-)
have been covered
- [ ] Test plan covers all impacted features and [areas of
interest](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#areas-of-interest-)
- [ ] Test plan has been peer reviewed by project stakeholders and other
QA members
- [ ] Manually tested functionality on DP
- [ ] We had an implementation alignment call with stakeholders post QA
Round 2
- [ ] Cypress test cases have been added and approved by SDET/manual QA
- [ ] Added `Test Plan Approved` label after Cypress tests were reviewed
- [ ] Added `Test Plan Approved` label after JUnit tests were reviewed
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Summary by CodeRabbit
The existing bullet-point list is still valid and does not need to be
updated.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix empty commit failures for server build
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Updated the server build workflow to allow commits with empty
messages.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Description
Added restore cache from git for client build
#### Type of change
- New feature (non-breaking change which adds functionality)
## Testing
Tested in local fork
#### How Has This Been Tested?
- [X] Manual
- [ ] JUnit
- [ ] Jest
- [ ] Cypress
## Checklist:
#### Dev activity
- [X] My code follows the style guidelines of this project
- [X] I have performed a self-review of my own code
- [X] I have commented my code, particularly in hard-to-understand areas
- [X] I have made corresponding changes to the documentation
- [X] My changes generate no new warnings
- [X] I have added tests that prove my fix is effective or that my
feature works
- [X] New and existing unit tests pass locally with my changes
- [ ] PR is being merged under a feature flag
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Refactor**
- Optimized the client build workflow to trigger only on relevant
changes or specific events, ensuring resource-efficient operations.
- **Chores**
- Updated workflow conditions to improve the CI/CD process, enhancing
overall development efficiency.
- **Documentation**
- Provided triage instructions to streamline the handling of workflow
changes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Description
Add Caching support for Client Build
#### PR fixes following issue(s)
#### Type of change
- New feature (non-breaking change which adds functionality)
## Testing
#### How Has This Been Tested?
Tested in a fork
## Checklist:
#### Dev activity
- [X] My code follows the style guidelines of this project
- [X] I have performed a self-review of my own code
- [X] I have commented my code, particularly in hard-to-understand areas
- [X] I have made corresponding changes to the documentation
- [X] My changes generate no new warnings
- [X] I have added tests that prove my fix is effective or that my
feature works
- [X] New and existing unit tests pass locally with my changes
- [ ] PR is being merged under a feature flag
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Implemented caching for release builds to enhance performance.
- **Chores**
- Updated workflow to ensure successful status updates during build
processes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Description
> Added excludeForAirgap tgs
#### Type of change
- Chore (housekeeping or task changes that don't impact user perception)
## Testing
> running tbp
> /ok-to-test tags="@tag.excludeForAirgap"
#### How Has This Been Tested?
- [ ] Manual
- [ ] Cypress
>
>
## Checklist:
#### Dev activity
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] PR is being merged under a feature flag
#### QA activity:
- [ ] [Speedbreak
features](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#speedbreakers-)
have been covered
- [ ] Test plan covers all impacted features and [areas of
interest](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#areas-of-interest-)
- [ ] Test plan has been peer reviewed by project stakeholders and other
QA members
- [ ] Manually tested functionality on DP
- [ ] We had an implementation alignment call with stakeholders post QA
Round 2
- [ ] Cypress test cases have been added and approved by SDET/manual QA
- [ ] Added `Test Plan Approved` label after Cypress tests were reviewed
- [ ] Added `Test Plan Approved` label after JUnit tests were reviewed
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Test Enhancements**
- Improved test descriptions and added relevant tags for better
categorization and filtering across various test suites.
- Consolidated test logic and descriptions for clarity and coherence in
test suites.
- **Bug Fixes**
- Updated test cases to reflect accurate scenarios for bugs and feature
testing.
- Adjusted test sequences and assertions for enhanced reliability and
coverage.
- **Refactor**
- Restructured test suites for a more logical flow and clearer
objectives.
- Enhanced test setup steps and assertions for comprehensive validation.
- **Chores**
- Implemented a new tagging functionality for organizing test suites and
cases.
- Added new tags to the tagging system for improved test management.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Added checks for scheduled changes
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Refactor**
- Updated the workflow to trigger certain actions on a scheduled basis
as well as other specified events.
- **Documentation**
- No changes in public API documentation required.
- **Chores**
- Triage process remains unchanged.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Updated the build process to use a new self-hosted deployment runner
with enhanced resources.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Description
added changes to skip server incase of no changes
#### PR fixes following issue(s)
#### Type of change
- New feature (non-breaking change which adds functionality)
## Checklist:
#### Dev activity
- [X] My code follows the style guidelines of this project
- [X] I have performed a self-review of my own code
- [X] I have commented my code, particularly in hard-to-understand areas
- [X] I have made corresponding changes to the documentation
- [X] My changes generate no new warnings
- [X] I have added tests that prove my fix is effective or that my
feature works
- [X] New and existing unit tests pass locally with my changes
- [ ] PR is being merged under a feature flag
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Improved the efficiency of the build process by only running
server-related jobs when there are changes in the server folder.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
