PromucFlow_constructor/.github/workflows/ci-test-custom-script.yml
Aishwarya-U-R 30300bb6a4
test: Cypress | Replacing some external api's with TED + Flaky fix (#31118)
## Description
- This PR does some clean up of comments - in DeployHelper.ts
- This PR replaced external image api with TED mock api in
ServerSide/OnLoadTests/OnLoadActions_Spec.ts
- Extrenal example api to TED in Widgets/Image/Image_spec.js
- Extrenal api links in
app/client/cypress/e2e/Regression/ServerSide/ApiTests/API_Bugs_Spec.js
to TED api's
- Flaky fixes /ServerSide/LoginTests/LoginFailure_spec.js - for EE CI
run failure (Added multiple Dynamic checks)
- support command.js method LoginUser() - added Dynamic check
- Encapsulated Datasources/Arango_Basic_Spec.ts into conditional check
to run in CE alone, instead of entirely skipping, since its failing in
EE due to chrome crash

#### Media
#### Type of change
- Script fix (non-breaking change which fixes an issue)

## Testing
#### How Has This Been Tested?
- [X] Cypress CI runs

## Checklist:
#### QA activity:
- [X] Added `Test Plan Approved` label after Cypress tests were reviewed

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Tests**
- Updated Cypress regression tests to use flower images instead of cat
images for API calls.
- Modified limited test configurations to run with different spec files.
- Adjusted GitHub Actions workflow to change `matrix_count` values based
on the event type and retrieve PR number for manual triggers.
	- Updated Docker image tag in CI tests to `nightly`.

- **Refactor**
- Enhanced `DataManager` class with new properties for flower image
URLs.
- Improved `EntityExplorer` class method to ensure more reliable
triggering of events.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-02-23 13:18:01 +05:30

423 lines
19 KiB
YAML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

name: Appsmith CI Test Workflow
on:
# This line enables manual triggering of this workflow.
workflow_dispatch:
workflow_call:
inputs:
pr:
description: "This is the PR number in case the workflow is being called in a pull request"
required: false
type: number
tags:
description: "These are the optional tags a developer can specify in order to run a subset of all the tests"
required: false
type: string
default: ""
matrix:
description: "Matrix jobs"
required: false
type: string
default: "[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59]"
jobs:
ci-test:
runs-on: ubuntu-latest
if: |
github.event.pull_request.head.repo.full_name == github.repository ||
github.event_name == 'push' ||
github.event_name == 'workflow_dispatch' ||
github.event_name == 'repository_dispatch' ||
github.event_name == 'schedule'
defaults:
run:
shell: bash
strategy:
fail-fast: false
matrix:
job: ${{ fromJson(inputs.matrix) }}
# Service containers to run with this job. Required for running tests
services:
# Label used to access the service container
redis:
# Docker Hub image for Redis
image: redis
ports:
# Opens tcp port 6379 on the host and service container
- 6379:6379
mongo:
image: mongo
ports:
- 27017:27017
steps:
- name: Set up Depot CLI
uses: depot/setup-action@v1
- name: Login to DockerHub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
# Check out merge commit
- name: Fork based /ok-to-test checkout
if: inputs.pr != 0
uses: actions/checkout@v4
with:
ref: "refs/pull/${{ inputs.pr }}/merge"
# Checkout the code in the current branch in case the workflow is called because of a branch push event
- name: Checkout the head commit of the branch
if: inputs.pr == 0
uses: actions/checkout@v4
# Timestamp will be used to create cache key
- id: timestamp
run: echo "timestamp=$(date +'%Y-%m-%dT%H:%M:%S')" >> $GITHUB_OUTPUT
# In case this is second attempt try restoring status of the prior attempt from cache
- name: Restore the previous run result
id: cache-appsmith
uses: actions/cache@v3
with:
path: |
~/run_result
key: ${{ github.run_id }}-${{ github.job }}-${{ matrix.job }}
restore-keys: |
${{ github.run_id }}-${{ github.job }}-${{ matrix.job }}
- name: Get the previous run result
if: steps.cache-appsmith.outputs.cache-hit == 'true'
id: run_result
run: |
run_result_env=$(cat ~/run_result)
echo "run_result=$run_result_env" >> $GITHUB_OUTPUT
if [[ "$run_result_env" == "failedtest" ]]; then
echo "rerun=true" >> $GITHUB_OUTPUT
else
echo "rerun=false" >> $GITHUB_OUTPUT
fi
- name: Dump steps context
env:
STEPS_CONTEXT: ${{ toJson(steps) }}
run: echo "$STEPS_CONTEXT"
- if: steps.run_result.outputs.run_result != 'success' && steps.run_result.outputs.run_result != 'failedtest'
run: echo "Starting full run" && exit 0
- if: steps.run_result.outputs.run_result == 'failedtest'
run: echo "Rerunning failed tests" && exit 0
- name: cat run_result
run: echo ${{ steps.run_result.outputs.run_result }}
- name: Restore the docker image cache
uses: actions/cache@v3
with:
path: cicontainer.tar.gz
key: docker-image-${{github.run_id}}
- name: Load Docker image from tar file
run: |
gunzip cicontainer.tar.gz
docker load -i cicontainer.tar
- name: Create folder
if: steps.run_result.outputs.run_result != 'success'
working-directory: "."
run: |
mkdir -p cicontainerlocal/stacks/configuration/
- name: Run Appsmith & TED docker image
if: steps.run_result.outputs.run_result != 'success'
env:
LAUNCHDARKLY_BUSINESS_FLAGS_SERVER_KEY: ${{ secrets.LAUNCHDARKLY_BUSINESS_FLAGS_SERVER_KEY }}
working-directory: "."
run: |
sudo /etc/init.d/ssh stop ;
mkdir -p ~/git-server/keys
mkdir -p ~/git-server/repos
docker run --name test-event-driver -d -p 22:22 -p 5001:5001 -p 3306:3306 \
-p 5432:5432 -p 28017:27017 -p 25:25 -p 5000:5000 -p 3001:3000 -p 6001:6001 -p 8001:8000 --privileged --pid=host --ipc=host --volume /:/host -v ~/git-server/keys:/git-server/keys \
-v ~/git-server/repos:/git-server/repos appsmith/test-event-driver:latest
docker run --name cloud-services -d -p 8000:80 -p 8090:8090 \
--privileged --pid=host --ipc=host --add-host=host.docker.internal:host-gateway\
-e APPSMITH_CLOUD_SERVICES_MONGODB_URI=mongodb://host.docker.internal:27017 \
-e APPSMITH_CLOUD_SERVICES_MONGODB_DATABASE=cs \
-e APPSMITH_CLOUD_SERVICES_MONGODB_AUTH_DATABASE=admin \
-e APPSMITH_REDIS_URL=redis://host.docker.internal:6379/ \
-e APPSMITH_APPS_API_KEY=dummy-api-key \
-e APPSMITH_REMOTE_API_KEY=dummy-api-key \
-e APPSMITH_GITHUB_API_KEY=dummy-appsmith-gh-api-key \
-e APPSMITH_JWT_SECRET=appsmith-cloud-services-jwt-secret-dummy-key \
-e APPSMITH_ENCRYPTION_SALT=encryption-salt \
-e APPSMITH_ENCRYPTION_PASSWORD=encryption-password \
-e APPSMITH_CLOUD_SERVICES_URL=https://cs-dev.appsmith.com \
-e APPSMITH_CUSTOMER_PORTAL_URL=https://dev.appsmith.com \
-e APPSMITH_CLOUD_SERVICES_BASE_URL=https://cs-dev.appsmith.com \
-e APPSMITH_CLOUD_SERVER_BASE_URL=https://release.app.appsmith.com \
-e AUTH0_ISSUER_URL=https://login.release-customer.appsmith.com/ \
-e AUTH0_CLIENT_ID=dummy-client-id \
-e AUTH0_CLIENT_SECRET=dummy-secret-id \
-e AUTH0_AUDIENCE_URL=https://login.local-customer.appsmith.com/ \
-e CLOUDSERVICES_URL=cs-dev.appsmith.com \
-e CUSTOMER_URL=dev.appsmith.com \
-e ENTERPRISE_USER_NAME=ent-user@appsmith.com \
-e ENTERPRISE_USER_PASSWORD=ent_user_password \
-e ENTERPRISE_ADMIN_NAME=ent-admin@appsmith.com \
-e ENTERPRISE_ADMIN_PASSWORD=ent_admin_password \
-e FLAGSMITH_URL=http://host.docker.internal:5001/flagsmith \
-e FLAGSMITH_SERVER_KEY=dummykey \
-e FLAGSMITH_SERVER_KEY_BUSINESS_FEATURES=dummykeybusinessfeatures \
-e LAUNCHDARKLY_BUSINESS_FLAGS_SERVER_KEY=$LAUNCHDARKLY_BUSINESS_FLAGS_SERVER_KEY \
appsmith/cloud-services:release
cd cicontainerlocal
docker run -d --name appsmith -p 80:80 \
-v "$PWD/stacks:/appsmith-stacks" \
-e APPSMITH_DISABLE_TELEMETRY=true \
-e APPSMITH_INTERCOM_APP_ID=DUMMY_VALUE \
-e APPSMITH_CLOUD_SERVICES_BASE_URL=http://host.docker.internal:5001 \
-e APPSMITH_CLOUD_SERVICES_SIGNATURE_BASE_URL=http://host.docker.internal:8090 \
--add-host=host.docker.internal:host-gateway --add-host=api.segment.io:host-gateway --add-host=t.appsmith.com:host-gateway \
cicontainer
- name: Use Node.js
if: steps.run_result.outputs.run_result != 'success'
uses: actions/setup-node@v3
with:
node-version-file: app/client/package.json
# actions/setup-node@v3 doesnt work properly with Yarn 3
# when the project lives in a subdirectory: https://github.com/actions/setup-node/issues/488
# Restoring the cache manually instead
- name: Restore Yarn cache
if: steps.run_result.outputs.run_result != 'success'
uses: actions/cache@v3
with:
path: app/client/.yarn/cache
key: v1-yarn3-${{ hashFiles('app/client/yarn.lock') }}
# Install all the dependencies
- name: Install dependencies
if: steps.run_result.outputs.run_result != 'success'
working-directory: app/client
run: |
yarn install --immutable
- name: Set cypress tags to exclude
if: steps.run_result.outputs.run_result != 'success'
run: |
echo 'tags_to_exclude=airgap' >> "$GITHUB_ENV"
echo "CYPRESS_grepTags=${{ inputs.tags }}" >> $GITHUB_ENV
- name: Print the tags
run: |
echo "tags_to_exclude: ${{ env.tags_to_exclude }}"
echo "_grepTags: ${{ env.CYPRESS_grepTags }}"
- name: Setting up the cypress tests
if: steps.run_result.outputs.run_result != 'success'
shell: bash
run: |
cd app/client
chmod a+x ./cypress/setup-test-ci.sh
./cypress/setup-test-ci.sh
- uses: browser-actions/setup-chrome@latest
with:
chrome-version: stable
- run: |
echo "BROWSER_PATH=$(which chrome)" >> $GITHUB_ENV
- name: Save Git values
# pass env variables from this step to other steps
# using GitHub Actions environment file
# https://docs.github.com/en/actions/learn-github-actions/workflow-commands-for-github-actions#environment-files
run: |
PR_NUMBER=${{ inputs.pr }}
echo COMMIT_INFO_BRANCH=$(git rev-parse --abbrev-ref HEAD) >> $GITHUB_ENV
echo COMMIT_INFO_MESSAGE=OkToTest run on PR# ${{ inputs.pr }} >> $GITHUB_ENV
echo COMMIT_INFO_EMAIL=$(git show -s --pretty=%ae) >> $GITHUB_ENV
echo COMMIT_INFO_AUTHOR=$(git show -s --pretty=%an) >> $GITHUB_ENV
echo COMMIT_INFO_SHA=$(git show -s --pretty=%H) >> $GITHUB_ENV
echo COMMIT_INFO_TIMESTAMP=$(git show -s --pretty=%ct) >> $GITHUB_ENV
echo COMMIT_INFO_REMOTE=$(git config --get remote.origin.url) >> $GITHUB_ENV
# delete the .git folder afterwords to use the environment values
rm -rf .git
- name: Show Git values
run: |
echo Branch $COMMIT_INFO_BRANCH
echo Message $COMMIT_INFO_MESSAGE
echo Email $COMMIT_INFO_EMAIL
echo Author $COMMIT_INFO_AUTHOR
echo SHA $COMMIT_INFO_SHA
echo Timestamp $COMMIT_INFO_TIMESTAMP
echo Remote $COMMIT_INFO_REMOTE
- name: Set Commit Message
env:
EVENT_COMMITS: ${{ toJson(github.event.commits[0].message) }}
run: |
eventCommit=$(echo ${{env.EVENT_COMMITS}} | sed "s/'//g")
if [[ ${{ inputs.pr }} -ne 0 ]]; then
echo "COMMIT_INFO_MESSAGE=${{ env.COMMIT_INFO_MESSAGE }}" >> $GITHUB_ENV
else
if [[ '$eventCommit' == 'null' ]]; then
echo "COMMIT_INFO_MESSAGE=$(echo ${{ github.event_name }} | perl -pe 's/(^|_)./uc($&)/ge;s/_//g')" >> $GITHUB_ENV
echo "COMMIT_INFO_MESSAGE=${{ env.COMMIT_INFO_MESSAGE}} by ${{ env.COMMIT_INFO_AUTHOR }}" >> $GITHUB_ENV
else
echo "COMMIT_INFO_MESSAGE=$(echo "$eventCommit" | awk -F '\\\\n' '{print $1}' | sed 's/^"//;s/"$//')" >> $GITHUB_ENV
fi
fi
- name: Run the cypress test
uses: cypress-io/github-action@v6
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
CYPRESS_USERNAME: ${{ secrets.CYPRESS_USERNAME }}
CYPRESS_PASSWORD: ${{ secrets.CYPRESS_PASSWORD }}
CYPRESS_TESTUSERNAME1: ${{ secrets.CYPRESS_TESTUSERNAME1 }}
CYPRESS_TESTPASSWORD1: ${{ secrets.CYPRESS_TESTPASSWORD1 }}
CYPRESS_TESTUSERNAME2: ${{ secrets.CYPRESS_TESTUSERNAME2 }}
CYPRESS_TESTPASSWORD2: ${{ secrets.CYPRESS_TESTPASSWORD1 }}
CYPRESS_TESTUSERNAME3: ${{ secrets.CYPRESS_TESTUSERNAME3 }}
CYPRESS_TESTPASSWORD3: ${{ secrets.CYPRESS_TESTPASSWORD3 }}
CYPRESS_TESTUSERNAME4: ${{ secrets.CYPRESS_TESTUSERNAME4 }}
CYPRESS_TESTPASSWORD4: ${{ secrets.CYPRESS_TESTPASSWORD4 }}
CYPRESS_S3_ACCESS_KEY: ${{ secrets.CYPRESS_S3_ACCESS_KEY }}
CYPRESS_S3_SECRET_KEY: ${{ secrets.CYPRESS_S3_SECRET_KEY }}
CYPRESS_GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
CYPRESS_AIRTABLE_BEARER: ${{ secrets.AIRTABLE_BEARER }}
CYPRESS_ORACLE_HOST: ${{ secrets.ORACLE_HOST }}
CYPRESS_ORACLE_SERVICE: ${{ secrets.ORACLE_SERVICE }}
CYPRESS_ORACLE_USERNAME: ${{ secrets.ORACLE_USERNAME }}
CYPRESS_ORACLE_PASSWORD: ${{ secrets.ORACLE_PASSWORD }}
CYPRESS_FIRESTORE_PRIVATE_KEY: ${{ secrets.FIRESTORE_PRIVATE_KEY }}
CYPRESS_GITHUB_PERSONAL_ACCESS_TOKEN: ${{ secrets.CYPRESS_GITHUB_PERSONAL_ACCESS_TOKEN }}
CYPRESS_TEST_GITHUB_USER_NAME: ${{ secrets.CYPRESS_TEST_GITHUB_USER_NAME }}
CYPRESS_APPSMITH_OAUTH2_GOOGLE_CLIENT_ID: ${{ secrets.CYPRESS_APPSMITH_OAUTH2_GOOGLE_CLIENT_ID }}
CYPRESS_APPSMITH_OAUTH2_GOOGLE_CLIENT_SECRET: ${{ secrets.CYPRESS_APPSMITH_OAUTH2_GOOGLE_CLIENT_SECRET }}
CYPRESS_APPSMITH_OAUTH2_GITHUB_CLIENT_ID: ${{ secrets.CYPRESS_APPSMITH_OAUTH2_GITHUB_CLIENT_ID }}
CYPRESS_APPSMITH_OAUTH2_GITHUB_CLIENT_SECRET: ${{ secrets.CYPRESS_APPSMITH_OAUTH2_GITHUB_CLIENT_SECRET }}
CYPRESS_OAUTH_SAML_EMAIL: ${{ secrets.CYPRESS_OAUTH_SAML_EMAIL }}
CYPRESS_OAUTH_SAML_ENTITY_ID: ${{ secrets.CYPRESS_OAUTH_SAML_ENTITY_ID }}
CYPRESS_OAUTH_SAML_METADATA_URL: ${{ secrets.CYPRESS_OAUTH_SAML_METADATA_URL }}
CYPRESS_OAUTH_SAML_METADATA_XML: ${{ secrets.CYPRESS_OAUTH_SAML_METADATA_XML }}
CYPRESS_OAUTH_SAML_PUB_CERT: ${{ secrets.CYPRESS_OAUTH_SAML_PUB_CERT }}
CYPRESS_OAUTH_SAML_SSO_URL: ${{ secrets.CYPRESS_OAUTH_SAML_SSO_URL }}
CYPRESS_OAUTH_SAML_REDIRECT_URL: ${{ secrets.CYPRESS_OAUTH_SAML_REDIRECT_URL }}
CYPRESS_APPSMITH_OAUTH2_OIDC_CLIENT_ID: ${{ secrets.CYPRESS_APPSMITH_OAUTH2_OIDC_CLIENT_ID }}
CYPRESS_APPSMITH_OAUTH2_OIDC_CLIENT_SECRET: ${{ secrets.CYPRESS_APPSMITH_OAUTH2_OIDC_CLIENT_SECRET }}
CYPRESS_APPSMITH_OAUTH2_OIDC_AUTH_URL: ${{ secrets.CYPRESS_APPSMITH_OAUTH2_OIDC_AUTH_URL }}
CYPRESS_APPSMITH_OAUTH2_OIDC_TOKEN_URL: ${{ secrets.CYPRESS_APPSMITH_OAUTH2_OIDC_TOKEN_URL }}
CYPRESS_APPSMITH_OAUTH2_OIDC_USER_INFO: ${{ secrets.CYPRESS_APPSMITH_OAUTH2_OIDC_USER_INFO }}
CYPRESS_APPSMITH_OAUTH2_OIDC_JWKS_URL: ${{ secrets.CYPRESS_APPSMITH_OAUTH2_OIDC_JWKS_URL }}
CYPRESS_APPSMITH_OAUTH2_OIDC_OKTA_PASSWORD: ${{ secrets.CYPRESS_APPSMITH_OAUTH2_OIDC_OKTA_PASSWORD }}
CYPRESS_APPSMITH_OAUTH2_OIDC_DIRECT_URL: ${{ secrets.CYPRESS_APPSMITH_OAUTH2_OIDC_DIRECT_URL }}
CYPRESS_EXCLUDE_TAGS: ${{ env.tags_to_exclude }}
CYPRESS_AIRGAPPED: false
APPSMITH_DISABLE_TELEMETRY: true
APPSMITH_GOOGLE_MAPS_API_KEY: ${{ secrets.APPSMITH_GOOGLE_MAPS_API_KEY }}
POSTGRES_PASSWORD: postgres
CYPRESS_VERIFY_TIMEOUT: 100000
COMMIT_INFO_MESSAGE: ${{ env.COMMIT_INFO_MESSAGE }}
THIS_RUNNER: ${{ strategy.job-index }}
TOTAL_RUNNERS: ${{ strategy.job-total }}
RUNID: ${{ github.run_id }}
ATTEMPT_NUMBER: ${{ github.run_attempt }}
REPOSITORY: ${{ github.repository }}
COMMITTER: ${{ env.COMMIT_INFO_AUTHOR }}
TAG: ${{ github.event_name }}
BRANCH: ${{ env.COMMIT_INFO_BRANCH }}
CYPRESS_RERUN: ${{steps.run_result.outputs.rerun}}
CYPRESS_DB_USER: ${{ secrets.CYPRESS_DB_USER }}
CYPRESS_DB_HOST: ${{ secrets.CYPRESS_DB_HOST }}
CYPRESS_DB_NAME: ${{ secrets.CYPRESS_DB_NAME }}
CYPRESS_DB_PWD: ${{ secrets.CYPRESS_DB_PWD }}
CYPRESS_S3_ACCESS: ${{ secrets.CYPRESS_S3_ACCESS }}
CYPRESS_S3_SECRET: ${{ secrets.CYPRESS_S3_SECRET }}
CYPRESS_grepTags: ${{ env.CYPRESS_grepTags }} # This is a comma separated list of tags to run a subset of the suite
CYPRESS_SKIP_FLAKY: true
CYPRESS_STATIC_ALLOCATION: true
DEBUG: ${{secrets.CYPRESS_GREP_DEBUG }} # This is derived from secrets so that we can toggle it without having to change any workflow. Only acceptable value is: @cypress/grep
with:
browser: ${{ env.BROWSER_PATH }}
install: false
config-file: cypress_ci_custom.config.ts
working-directory: app/client
env: "NODE_ENV=development"
- name: Trim number of cypress log files
if: failure()
run: |
find ${{ github.workspace }}/app/client/cypress/cypress-logs -name '*.json' -type f | tail -n +11 | xargs -I {} rm -- {}
- name: Upload failed test cypress logs artifact
if: failure()
uses: actions/upload-artifact@v3
with:
name: cypress-console-logs
path: ${{ github.workspace }}/app/client/cypress/cypress-logs
- name: Collect CI container logs
if: failure()
working-directory: "."
run: |
mkdir -p ~/dockerlogs
docker logs appsmith 2>&1 > ~/dockerlogs/dockerlogs-${{ matrix.job }}.txt
# Upload docker logs
- name: Upload failed test list artifact
if: failure()
uses: actions/upload-artifact@v3
with:
name: dockerlogs
path: ~/dockerlogs
- name: Rename reports
if: failure()
run: |
mkdir -p ~/results
mv ${{ github.workspace }}/app/client/results ~/results/${{ matrix.job }}
- name: Upload cypress report
if: failure()
uses: actions/upload-artifact@v3
with:
name: results-${{github.run_attempt}}
path: ~/results
# Set status = failedtest
- name: Set fail if there are test failures
if: failure()
run: |
echo "run_result=failedtest" >> $GITHUB_OUTPUT
echo "failedtest" > ~/run_result
# Force store previous run result to cache
- name: Store the previous run result
if: failure()
uses: actions/cache/save@v3
with:
path: |
~/run_result
key: ${{ github.run_id }}-${{ github.job }}-${{ matrix.job }}
# Upload the log artifact so that it can be used by the test & deploy job in the workflow
- name: Upload server logs bundle on failure
uses: actions/upload-artifact@v3
if: failure()
with:
name: server-logs-${{ matrix.job }}
path: app/server/server-logs.log
# Set status = success
- name: Save the status of the run
run: |
echo "run_result=success" >> $GITHUB_OUTPUT
echo "success" > ~/run_result