## Description
> Modified the message thrown when no tests ran from
`It seems like there are some failures 😔. We are not able to recognize
it, please check this manually`
to
` It seems like no tests ran 😔. We are not able to recognize it, please
check workflow`
#### Type of change
- Chore (housekeeping or task changes that don't impact user perception)
#### How Has This Been Tested?
> Please describe the tests that you ran to verify your changes. Also
list any relevant details for your test configuration.
> Delete anything that is not relevant
- [ ] Manual
## Checklist:
#### Dev activity
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] PR is being merged under a feature flag
#### QA activity:
- [ ] [Speedbreak
features](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#speedbreakers-)
have been covered
- [ ] Test plan covers all impacted features and [areas of
interest](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#areas-of-interest-)
- [ ] Test plan has been peer reviewed by project stakeholders and other
QA members
- [ ] Manually tested functionality on DP
- [ ] We had an implementation alignment call with stakeholders post QA
Round 2
- [ ] Cypress test cases have been added and approved by SDET/manual QA
- [ ] Added `Test Plan Approved` label after Cypress tests were reviewed
- [ ] Added `Test Plan Approved` label after JUnit tests were reviewed
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Improved messaging for test failure scenarios to clarify when no tests
have been run.
- Updated the link for checking the workflow status.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Description
> Added: **step** - Add suggestion in the PR on using @tag.All in
workflow `.github/workflows/integration-tests-command.yml`
> This will post a comment in the PR to use specific tags.
#### Type of change
- Chore (housekeeping or task changes that don't impact user perception)
## Testing
> tested with tbp
#### How Has This Been Tested?
- [ ] Manual
## Checklist:
#### Dev activity
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] PR is being merged under a feature flag
#### QA activity:
- [ ] [Speedbreak
features](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#speedbreakers-)
have been covered
- [ ] Test plan covers all impacted features and [areas of
interest](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#areas-of-interest-)
- [ ] Test plan has been peer reviewed by project stakeholders and other
QA members
- [ ] Manually tested functionality on DP
- [ ] We had an implementation alignment call with stakeholders post QA
Round 2
- [ ] Cypress test cases have been added and approved by SDET/manual QA
- [ ] Added `Test Plan Approved` label after Cypress tests were reviewed
- [ ] Added `Test Plan Approved` label after JUnit tests were reviewed
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Integration tests now suggest using specific tags for more efficient
testing.
- Docker image build tests prompt for using `@tag.All` when appropriate,
with a link to documentation for guidance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Description
> Added step to check if @tag.All mentioned is as expected, if not throw
a message to use `/ok-to-test tags="@tag.All"` and exit the run
#### Type of change
- Chore (housekeeping or task changes that don't impact user perception)
## Testing
> TBP with tags
#### How Has This Been Tested?
- [x] Manual
- [x] TBP
>
## Checklist:
#### Dev activity
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] PR is being merged under a feature flag
#### QA activity:
- [ ] [Speedbreak
features](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#speedbreakers-)
have been covered
- [ ] Test plan covers all impacted features and [areas of
interest](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#areas-of-interest-)
- [ ] Test plan has been peer reviewed by project stakeholders and other
QA members
- [ ] Manually tested functionality on DP
- [ ] We had an implementation alignment call with stakeholders post QA
Round 2
- [ ] Cypress test cases have been added and approved by SDET/manual QA
- [ ] Added `Test Plan Approved` label after Cypress tests were reviewed
- [ ] Added `Test Plan Approved` label after JUnit tests were reviewed
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Summary by CodeRabbit
The existing bullet-point list is still valid and does not need to be
updated.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Adding logs to debug ci-test-result
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Refactor**
- Improved error handling and logging in integration test workflows.
- **Documentation**
- Fixed typographical error in the name of the test workflow.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Description
- Added logic to handle the runner count based on the tags
#### Type of change
- Workflow files
## Testing
- Workflow run
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a `tag-check` job to validate the presence of tags in
workflow commands.
- Workflows now comment on PRs with documentation links for tags when
tags are missing.
- **Enhancements**
- Enhanced CI workflows to conditionally run jobs based on tag presence.
- Added matrix inputs to improve job configuration flexibility.
- **Documentation**
- Added comments to PRs with links to workflow runs when tags are
specified.
- **Refactor**
- Centralized tag and matrix setup across multiple workflows for
consistency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Chores**
- Updated `umani/changed-files` action to v4.1.0 across various workflow
files.
- Updated `peter-evans/create-or-update-comment` action to v3 in
multiple workflow files.
- Minor adjustments to workflow syntax for consistency and accuracy.
- **Documentation**
- Corrected URLs in Slack notification messages to ensure accuracy.
- **Refactor**
- Removed redundant job configurations and steps related to caching and
storing run results.
- Streamlined environment variable usage by sourcing values from
secrets.
- **Style**
- Standardized quote usage in workflow files for file matching patterns.
- **Bug Fixes**
- Fixed URLs in echo statements within `test-build-docker-image.yml` to
point to the correct "cypress-dashboard" path segment.
- Updated `slack_color` and `slack_icon` values to better reflect
success and failure states in notifications.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Description
If the user types `ok-to-test tags=Git`, Cypress should only run the
specs marked with the Git tag
Related to #28943
#### Type of change
- New feature (non-breaking change which adds functionality)
## Testing
#### How Has This Been Tested?
- [ ] Manual
- [ ] JUnit
- [ ] Jest
- [ ] Cypress
#### QA activity:
- [ ] [Speedbreak
features](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#speedbreakers-)
have been covered
- [ ] Test plan covers all impacted features and [areas of
interest](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#areas-of-interest-)
- [ ] Test plan has been peer reviewed by project stakeholders and other
QA members
- [ ] Manually tested functionality on DP
- [ ] We had an implementation alignment call with stakeholders post QA
Round 2
- [ ] Cypress test cases have been added and approved by SDET/manual QA
- [ ] Added `Test Plan Approved` label after Cypress tests were reviewed
- [ ] Added `Test Plan Approved` label after JUnit tests were reviewed
## Description
- This PR includes [SetWidget property
validation](https://github.com/appsmithorg/TestSmith/issues/2409) script
- Identified flaky test path updation to point to Cypress dashboard
flaky test list
**Flaky fixes for below:**
- Api URL enter - added some settling time
- ClientSide/BugTests/AbortAction_Spec.ts - with new url
- ClientSide/Binding/Button_Text_WithRecaptcha_spec.js
- Updated from agHelper.SelectDropdownList to
propPane.SelectPropertiesDropDown
- /ClientSide/Widgets/Datepicker/DatePicker3_spec.ts
- /ClientSide/Widgets/ListV2/DataIdentifierProperty_spec.ts - split
- ServerSide/Postgres_DataTypes/Array_Spec.ts
- Radio/Radio2_spec.ts
- ListV2/DataIdentifier_spec.ts
- Postgres_DataTypes/UUID_Spec.ts
- Widgets/Multiselect/Multi_Select_Tree_spec.js
#### Type of change
- Script fix (non-breaking change which fixes an issue)
## Testing
>
#### How Has This Been Tested?
- [X] Cypress CI runs
## Checklist:
#### QA activity:
- [X] Added `Test Plan Approved` label after Cypress tests were reviewed
## Description
- This PR adds tests for validating the set properties for widget across
all objects
- Covers only some scenarios from [Test
#2409](https://github.com/appsmithorg/TestSmith/issues/2409)
- Adds the failure cypress dashboard link directly in case of CI run
failures
- Flaky fix - Sanity/Datasources/SMTPDatasource_spec.js
- Cypress.on - exception & fail handling updated
- InvokeDispatchOnStore() created for logout/login Internal server
errors - trial fix for RBACFunctionalTests/OthersTabPermission_spec.js
- EditAppFromAppHover() improved to take app name also - trial fix
PropertyPane_Connections_Error_spec.js
- Flaky fix - TableV2_Property_ToggleJs_With_Binding_spec.js
- Flaky fix - Radio2_spec.ts
- Flaky fix - API_TestExecuteWithDynamicBindingInUrl_spec.ts
- Flaky fix - Button_with_API_spec.js
- AssertTableInVirtuosoList() improved to handle the DS name not present
in EE scenario also
- Flaky fix - Button_with_API_spec.js
- Flaky fix - Table_Style_ToggleJS_spec.js
- Flaky fix - Tablev2_Style_ToggleJS_spec
#### Type of change
- Script update (non-breaking change which fixes an issue)
## Testing
#### How Has This Been Tested?
- [X] Cypress CI runs
## Checklist:
#### QA activity:
- [X] Added `Test Plan Approved` label after Cypress tests were reviewed
> Pull Request Template
>
> Use this template to quickly create a well written pull request.
Delete all quotes before creating the pull request.
>
## Description
> Add a TL;DR when description is extra long (helps content team)
>
> Please include a summary of the changes and which issue has been
fixed. Please also include relevant motivation
> and context. List any dependencies that are required for this change
>
> Links to Notion, Figma or any other documents that might be relevant
to the PR
>
>
#### PR fixes following issue(s)
Fixes # (issue number)
> if no issue exists, please create an issue and ask the maintainers
about this first
>
>
#### Media
> A video or a GIF is preferred. when using Loom, don’t embed because it
looks like it’s a GIF. instead, just link to the video
>
>
#### Type of change
> Please delete options that are not relevant.
- Bug fix (non-breaking change which fixes an issue)
- New feature (non-breaking change which adds functionality)
- Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- Chore (housekeeping or task changes that don't impact user perception)
- This change requires a documentation update
>
>
>
## Testing
>
#### How Has This Been Tested?
> Please describe the tests that you ran to verify your changes. Also
list any relevant details for your test configuration.
> Delete anything that is not relevant
- [ ] Manual
- [ ] JUnit
- [ ] Jest
- [ ] Cypress
>
>
#### Test Plan
> Add Testsmith test cases links that relate to this PR
>
>
#### Issues raised during DP testing
> Link issues raised during DP testing for better visiblity and tracking
(copy link from comments dropped on this PR)
>
>
>
## Checklist:
#### Dev activity
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] PR is being merged under a feature flag
#### QA activity:
- [ ] [Speedbreak
features](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#speedbreakers-)
have been covered
- [ ] Test plan covers all impacted features and [areas of
interest](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#areas-of-interest-)
- [ ] Test plan has been peer reviewed by project stakeholders and other
QA members
- [ ] Manually tested functionality on DP
- [ ] We had an implementation alignment call with stakeholders post QA
Round 2
- [ ] Cypress test cases have been added and approved by SDET/manual QA
- [ ] Added `Test Plan Approved` label after Cypress tests were reviewed
- [ ] Added `Test Plan Approved` label after JUnit tests were reviewed
## Description
- This PR does below:
- Oracle - Create new table, insert data & select. tc #
[2358](https://github.com/appsmithorg/TestSmith/issues/2358) tc #
[2360](https://github.com/appsmithorg/TestSmith/issues/2358)
- Oracle - Alter the new table for 2 new columns, insert multiple rows
into table
- Oracle - Select * from table & select with condition tc #
[2359](https://github.com/appsmithorg/TestSmith/issues/2359)
- Oracle - Add table widget from Suggested widget section, Deploy the
app & verify table data load
- Oracle - Back to Editor, Delete all queries, Delete DS
- Failure fix - ClientSide/Git/GitSync/Connection_spec.js fo CI runs
#### Type of change
- Script fix + update (non-breaking change which fixes an issue)
## Testing
#### How Has This Been Tested?
- [X] Cypress CI runs
## Checklist:
#### QA activity:
- [X] Added `Test Plan Approved` label after Cypress tests were reviewed
## Description
- This PR fixes the flakyness with error message in
ConnectionErrors_spec
- This PR adds validations for SMTP plugin (added documentation
verification + email sent successfully)
- Updating the Cypress Dashboard for new links
- Improve Widgets/Image/Image2_Spec.ts
- Flaky fix BugTests/Bug26716_Spec.ts
- ShowAllDatasources() added for direct navigation to Action DS list
- Fix ServerSide/GenerateCRUD/Postgres2_Spec.ts
- DeleteDatasouceFromActiveTab() improved
- Airtable - added documentation verification
- Airtable - Added widget binding & deploy App verification of Api
records data
- Datasources/MongoDatasource_spec.js - js to ts definition
- /EmbedSettings/EmbedSettings_spec.js - server restart timeout
increased
#### Type of change
- Script update (non-breaking change which fixes an issue)
## Testing
#### How Has This Been Tested?
- [X] Cypress CI runs
## Checklist:
#### QA activity:
- [X] Added `Test Plan Approved` label after Cypress tests were reviewed
## Description
- Fix cypress dashboard url for rerun for Git messages that comes in PR
runs
#### Type of change
- Workflow changes
## Testing
>
#### How Has This Been Tested?
- workflow run
## Description
- Fix for comments in case ci-test failure due to other steps
- Fixed the ci-test-result issue in ci-test-limited
#### Type of change
- Workflow file
## Testing
- Workflow run
## Description
- Fix to use cypress dashboard for ci test
#### Type of change
- Workflow file changes
## Testing
>
#### How Has This Been Tested?
- workflow run
## Description
- Fix comment issue in PR for failed specs
#### Type of change
- Workflow file changes
## Testing
>
#### How Has This Been Tested?
- Workflow run
## Description
- Fix cypress dashboard link in pr comment
#### Type of change
- Workflow file changes
## Testing
>
#### How Has This Been Tested?
- Workflow run
## Description
- Fix extracting the cypress dashboard url in ci-test-result
#### Type of change
- Workflow file changes
## Testing
>
#### How Has This Been Tested?
- Workflow file changes
## Description
- Get the newly added files from the PR branch instead of commits for ts
check
- CI improvements, now added cypress dashboard link with the comments in
PR
- CI improvements removed the steps which saves un-necessary cache
#### Type of change
- Workflow file changes
## Testing
>
#### How Has This Been Tested?
- Workflow run
## Description
- Skipping the ts check for test files if not required, introduced a
input variable for the client build to control this
#### Type of change
- Workflow file
## Testing
>
#### How Has This Been Tested?
- Workflow run
This PR fixes incorrect diff calculation between base and head of the
pull request raised. This diff is used for determining if server tests
should run or not.
Related PR to this is :
https://github.com/appsmithorg/appsmith/pull/24731
This PR prints github event when ok-to-test command is run. This is
needed for debugging optimization changes to skip server tests if there
are no server side changes.
Fixes#24700
These changes will allow the CI to skip tests if there are no changes in
server folder when using ok-to-test command. This will save CI minutes
## Description
- This PR updates the text message that gets commented after a PR run
## Type of change
- Yml file update
## Checklist:
### QA activity:
- [X] Added Test Plan Approved label after reviewing all changes
## Description
- This fixes the Firestore spec that fails sometimes in CI due to all
runs accessing the same cloud collection for validation.
- This PR also adds comments to PR - if all cypress cases passed, right
now there is no comment
## Type of change
- Script update
## How Has This Been Tested?
- Cypress CI runs
## Checklist:
### QA activity:
- [X] Added Test Plan Approved label after reviewing all Cypress test
## Description
- Added 'Workflow run' and 'Commit' for the failed spec comments
## Type of change
- YML Changes
## How Has This Been Tested?
- CI run
## Checklist:
### QA activity:
- [ ] Test plan has been approved by relevant developers
- [ ] Test plan has been peer reviewed by QA
- [ ] Cypress test cases have been added and approved by either SDET or
manual QA
- [ ] Organized project review call with relevant stakeholders after
Round 1/2 of QA
- [ ] Added Test Plan Approved label after reveiwing all Cypress test
---------
Co-authored-by: Aishwarya UR <aishwarya@appsmith.com>
## Description
- This PR adds the Flaky list application link into the git comments
section on completion of the run, so the PR owners have easy access to
it.
## Type of change
- Yml file update
## Checklist:
### QA activity:
- [X] Added Test Plan Approved label after reviewing all changes
## Description
- Upload an empty failed-spec-ci file if all the specs are passed
## Type of change
- YML file changes
## How Has This Been Tested?
- github actions
## Checklist:
### QA activity:
- [ ] Test plan has been approved by relevant developers
- [ ] Test plan has been peer reviewed by QA
- [ ] Cypress test cases have been added and approved by either SDET or
manual QA
- [ ] Organized project review call with relevant stakeholders after
Round 1/2 of QA
- [ ] Added Test Plan Approved label after reveiwing all Cypress test
## Description
- Added dub points for failed-spec comment
## Type of change
- YML File changes
## How Has This Been Tested?
- Manual
## Checklist:
### QA activity:
- [ ] Test plan has been approved by relevant developers
- [ ] Test plan has been peer reviewed by QA
- [ ] Cypress test cases have been added and approved by either SDET or
manual QA
- [ ] Organized project review call with relevant stakeholders after
Round 1/2 of QA
- [ ] Added Test Plan Approved label after reveiwing all Cypress test
## Description
- Fix for combined-failed-spec-ci cache not found issue
## Type of change
- YML file changes
## How Has This Been Tested?
- Manual
## Checklist:
### QA activity:
- [ ] Test plan has been approved by relevant developers
- [ ] Test plan has been peer reviewed by QA
- [ ] Cypress test cases have been added and approved by either SDET or
manual QA
- [ ] Organized project review call with relevant stakeholders after
Round 1/2 of QA
- [ ] Added Test Plan Approved label after reveiwing all Cypress test
## Description
- Updated the workflow files to use actions/cache@v3
## Type of change
- yml file changes
## How Has This Been Tested?
- Manual
## Checklist:
### QA activity:
- [ ] Test plan has been approved by relevant developers
- [ ] Test plan has been peer reviewed by QA
- [ ] Cypress test cases have been added and approved by either SDET or
manual QA
- [ ] Organized project review call with relevant stakeholders after
Round 1/2 of QA
- [ ] Added Test Plan Approved label after reveiwing all Cypress test
## Description
- Reverting the use of combined-failed-spec-ci artifact instead of cache
## Type of change
- yml file changes
## How Has This Been Tested?
- Manual
## Checklist:
### QA activity:
- [ ] Test plan has been approved by relevant developers
- [ ] Test plan has been peer reviewed by QA
- [ ] Cypress test cases have been added and approved by either SDET or
manual QA
- [ ] Organized project review call with relevant stakeholders after
Round 1/2 of QA
- [ ] Added Test Plan Approved label after reveiwing all Cypress test
## Description
- Deleting the combined-failed-spec-ci in ci-test-result
## Type of change
- Yml file changes
## How Has This Been Tested?
- Manual
## Checklist:
### QA activity:
- [ ] Test plan has been approved by relevant developers
- [ ] Test plan has been peer reviewed by QA
- [ ] Cypress test cases have been added and approved by either SDET or
manual QA
- [ ] Organized project review call with relevant stakeholders after
Round 1/2 of QA
- [ ] Added Test Plan Approved label after reveiwing all Cypress test
## Description
- Deleting the failed-spec-ci once it is downloaded in ci-test-result to
avoid getting specs from older runs
## Type of change
- yml file changes
## How Has This Been Tested?
- Manual
## Checklist:
### QA activity:
- [ ] Test plan has been approved by relevant developers
- [ ] Test plan has been peer reviewed by QA
- [ ] Cypress test cases have been added and approved by either SDET or
manual QA
- [ ] Organized project review call with relevant stakeholders after
Round 1/2 of QA
- [ ] Added Test Plan Approved label after reveiwing all Cypress test
## Description
- Removed the artifact deleting as we need the artifact in case of rerun
on cancel
## Type of change
- Workflow yml files
## How Has This Been Tested?
- Manual
## Checklist:
### QA activity:
- [ ] Test plan has been approved by relevant developers
- [ ] Test plan has been peer reviewed by QA
- [ ] Cypress test cases have been added and approved by either SDET or
manual QA
- [ ] Organized project review call with relevant stakeholders after
Round 1/2 of QA
- [ ] Added Test Plan Approved label after reveiwing all Cypress test
## Description
- Removing the old combined failed spec file
## Type of change
- workflow files
## How Has This Been Tested?
- Manual
## Checklist:
### QA activity:
- [ ] Test plan has been approved by relevant developers
- [ ] Test plan has been peer reviewed by QA
- [ ] Cypress test cases have been added and approved by either SDET or
manual QA
- [ ] Organized project review call with relevant stakeholders after
Round 1/2 of QA
- [ ] Added Test Plan Approved label after reveiwing all Cypress test
## Description
- This PR includes tests for Airtable ds - along with connection &
querying & usage of various fields in the query page
## Type of change
- New script (non-breaking change which test functionality of Airtable
data-source)
## How Has This Been Tested?
- Cypress CI runs
## Checklist:
### QA activity:
- [X] Added Test Plan Approved label after reviewing all changes
## Description
- This PR includes basic tests for connecting to a MsSQL ds & running
simple queries on it
## Type of change
- New script (non-breaking change which functional tests the MsSql ds)
## How Has This Been Tested?
- Cypress local & CI run
## Checklist:
### QA activity:
- [X] Added Test Plan Approved label after reviewing all Cypress test
## Description
- Deleting duplicate specs while commenting failed specs in the pr
## Type of change
- integration-tests-command.yml
## How Has This Been Tested?
- Manual
## Checklist
### QA activity:
- [X] Added Test Plan Approved label after reviewing all changes
