chore: Move appsmithctl to RTS (#37531)

Move the `appsmithctl` code to RTS.

RTS' own build system will build `appsmithctl` as well. We're adding two
command scripts, `ctl` and `appsmithctl` to `/opt/bin`, which will be
the entrypoints for this. The `appsmithctl` is now just an alias to the
much shorter and non-redundancy-inducing `ctl`.

We aren't migrating to TypeScript in this PR so we're ignoring the new
`ctl` folder in both `tsconfig.json` and `.eslintrc`. That's temporary,
the next PR will fix that.


## Automation

/test sanity

### 🔍 Cypress test results
<!-- This is an auto-generated comment: Cypress test results  -->
> [!TIP]
> 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉
> Workflow run:
<https://github.com/appsmithorg/appsmith/actions/runs/11930931528>
> Commit: 90b5f97b801ac8d4b4b0126d85edff3dccc050bd
> <a
href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11930931528&attempt=1"
target="_blank">Cypress dashboard</a>.
> Tags: `@tag.Sanity`
> Spec:
> <hr>Wed, 20 Nov 2024 10:36:02 UTC
<!-- end of auto-generated comment: Cypress test results  -->


## Communication
Should the DevRel and Marketing teams inform users about this change?
- [ ] Yes
- [x] No


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

- **New Features**
  - Introduced the `appsmithctl` command for easier command execution.
  - Updated build process to include additional entry points.

- **Bug Fixes**
- Streamlined Docker build process, enhancing efficiency and reducing
complexity.

- **Documentation**
- Added a new section in the README for `appsmithctl` command
description.

- **Chores**
  - Updated dependencies in `package.json`.
  - Removed obsolete files and workflows to simplify project structure.

- **Style**
  - Added a new ESLint configuration for specific project needs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
Shrikant Sharat Kandula 2024-11-20 20:58:38 +05:30 committed by GitHub
parent f21ec5e36b
commit 81b7727817
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
29 changed files with 203 additions and 8802 deletions

View File

@ -1,99 +0,0 @@
# This workflow is responsible for building, testing & packaging the Appsmithctl CLI util
name: Build Appsmithctl CLI util Workflow
on:
# This line enables manual triggering of this workflow.
workflow_dispatch:
workflow_call:
inputs:
pr:
description: "This is the PR number in case the workflow is being called in a pull request"
required: false
type: number
pull_request:
branches: [release, master]
paths:
- "deploy/docker/fs/opt/appsmith/utils/**"
# Change the working directory for all the jobs in this workflow
defaults:
run:
working-directory: deploy/docker/fs/opt/appsmith/utils/
jobs:
build:
runs-on: ubuntu-latest
# Only run this workflow for internally triggered events
if: |
github.event.pull_request.head.repo.full_name == github.repository ||
github.event_name == 'push' ||
github.event_name == 'workflow_dispatch' ||
github.event_name == 'repository_dispatch'
steps:
# The checkout steps MUST happen first because the default directory is set according to the code base.
# GitHub Action expects all future commands to be executed in the code directory. Hence, we need to check out
# the code before doing anything else.
# Check out merge commit with the base branch in case this workflow is invoked via pull request
- name: Checkout the merged commit from PR and base branch
if: inputs.pr != 0
uses: actions/checkout@v4
with:
ref: refs/pull/${{ inputs.pr }}/merge
# Checkout the code in the current branch in case the workflow is called because of a branch push event
- name: Checkout the head commit of the branch
if: inputs.pr == 0
uses: actions/checkout@v4
- name: Figure out the PR number
run: echo ${{ inputs.pr }}
- name: Print the Github event
run: echo ${{ github.event_name }}
# In case this is second attempt try restoring status of the prior attempt from cache
- name: Restore the previous run result
uses: actions/cache@v4
with:
path: |
~/appsmithctl_run_result
key: ${{ github.run_id }}-${{ github.job }}-appsmithctl-util
# Fetch prior run result
- name: Get the previous run result
id: appsmithctl_run_result
run: cat ~/appsmithctl_run_result 2>/dev/null || echo 'default'
# In case of prior failure run the job
- if: steps.appsmithctl_run_result.outputs.appsmithctl_run_result != 'success'
run: echo "I'm alive!" && exit 0
- name: Use Node.js
if: steps.appsmithctl_run_result.outputs.appsmithctl_run_result != 'success'
uses: actions/setup-node@v4
with:
node-version-file: app/client/package.json
# Install all the dependencies
- name: Install dependencies
if: steps.appsmithctl_run_result.outputs.appsmithctl_run_result != 'success'
run: yarn install --immutable
# Run the Jest tests only if the workflow has been invoked in a PR
- name: Run the jest tests
if: steps.appsmithctl_run_result.outputs.appsmithctl_run_result != 'success'
run: yarn run test
# Set status = failure
- name: Set result as failed if there are build failures
if: failure()
run: |
echo "appsmithctl_run_result=failed" >> $GITHUB_OUTPUT > ~/appsmithctl_run_result
exit 1;
# Set status = success
- name: Save the status of the run
run: echo "appsmithctl_run_result=success" >> $GITHUB_OUTPUT > ~/appsmithctl_run_result

View File

@ -22,10 +22,6 @@ The `client-build` job builds the client-side codebase. It uses the configuratio
The `rts-build` job builds the "rts" (real-time suggestions) package of the client-side codebase. It uses the configuration defined in the `.github/workflows/rts-build.yml` file.
### `test-appsmithctl`
The `test-appsmithctl` job runs tests for the `appsmithctl` utility. It uses the configuration from the `.github/workflows/appsmithctl.yml` file.
### `build-docker-image`
The `build-docker-image` job builds and pushes the Docker image for the application. It depends on the successful completion of the `client-build`, `server-build`, and `rts-build` jobs. The Docker image is built with support for both `linux/arm64` and `linux/amd64` platforms.

View File

@ -45,14 +45,6 @@ jobs:
with:
pr: ${{ github.event.number }}
test-appsmithctl:
if: success()
name: appsmithctl
uses: ./.github/workflows/appsmithctl.yml
secrets: inherit
with:
pr: ${{ github.event.number }}
build-docker-image:
needs: [client-build, server-build, rts-build]
# Only run if the build step is successful

View File

@ -195,6 +195,7 @@ app/server/appsmith-server/src/main/java/com/appsmith/server/migrations/**/* @sh
# DevOps & Shri
deploy/**/* @sharat87 @pratapaprasanna
.github/workflows/*.yml @sharat87
app/client/packages/ctl/**/* @sharat87 @pratapaprasanna
app/server/**/pom.xml @sharat87
app/server/appsmith-server/src/main/java/com/appsmith/server/repositories/ce/BaseAppsmithRepositoryCEImpl.java @sharat87
app/server/appsmith-server/src/main/java/com/appsmith/server/repositories/ce/params/QueryAllParams.java @sharat87

View File

@ -29,17 +29,11 @@ COPY ./app/client/build editor/
# Add RTS - Application Layer
COPY ./app/client/packages/rts/dist rts/
ENV PATH /opt/bin:/opt/appsmith/utils/node_modules/.bin:/opt/java/bin:/opt/node/bin:$PATH
ENV PATH /opt/bin:/opt/java/bin:/opt/node/bin:$PATH
RUN <<END
set -o errexit
cd ./utils
npm install --only=prod
npm install --only=prod -g .
rm -rf utils/node_modules/resolve/test
cd -
# Make all `*.sh` files executable, excluding `node_modules`.
find . \( -name node_modules -prune \) -o \( -type f -name '*.sh' \) -exec chmod +x '{}' +

View File

@ -44,7 +44,7 @@ const getWorkflowDependencies = () => {
const bundle = async () => {
return esbuild
.build({
entryPoints: ["src/server.ts"],
entryPoints: ["src/server.ts", "src/ctl/index.js"],
bundle: true,
sourcemap: true,
platform: "node",

View File

@ -23,10 +23,14 @@
"@opentelemetry/semantic-conventions": "^1.27.0",
"@shared/ast": "workspace:^",
"axios": "^1.7.4",
"dotenv": "10.0.0",
"express": "^4.20.0",
"express-validator": "^6.14.2",
"http-status-codes": "^2.2.0",
"loglevel": "^1.8.1",
"mongodb": "^5.8.0",
"nodemailer": "6.9.9",
"readline-sync": "1.4.10",
"socket.io": "^4.6.2"
},
"devDependencies": {

View File

@ -0,0 +1,15 @@
{
"extends": ["../../../../.eslintrc.base.json"],
"rules": {
"@typescript-eslint/prefer-nullish-coalescing": "off",
"@typescript-eslint/strict-boolean-expressions": "off",
"@typescript-eslint/no-explicit-any": "off",
"testing-library/no-debugging-utils": "off",
"@typescript-eslint/no-var-requires": "off",
"padding-line-between-statements": "off",
"no-console": "off",
"@typescript-eslint/promise-function-async": "off",
"@typescript-eslint/no-unused-vars": "off",
"sort-destructure-keys/sort-destructure-keys": "off"
}
}

View File

@ -0,0 +1,3 @@
# `appsmithctl`
This is the source for the `appsmithctl` command in Appsmith containers.

View File

@ -4547,6 +4547,15 @@ __metadata:
languageName: node
linkType: hard
"@mongodb-js/saslprep@npm:^1.1.0":
version: 1.1.9
resolution: "@mongodb-js/saslprep@npm:1.1.9"
dependencies:
sparse-bitfield: ^3.0.3
checksum: 6f13983e41c9fbd5273eeae9135e47e5b7a19125a63287bea69e33a618f8e034cfcf2258c77d0f5d6dcf386dfe2bb520bc01613afd1528c52f82c71172629242
languageName: node
linkType: hard
"@mswjs/cookies@npm:^0.1.4":
version: 0.1.4
resolution: "@mswjs/cookies@npm:0.1.4"
@ -11305,6 +11314,23 @@ __metadata:
languageName: node
linkType: hard
"@types/webidl-conversions@npm:*":
version: 7.0.3
resolution: "@types/webidl-conversions@npm:7.0.3"
checksum: 535ead9de4d3d6c8e4f4fa14e9db780d2a31e8020debc062f337e1420a41c3265e223e4f4b628f97a11ecf3b96390962cd88a9ffe34f44e159dec583ff49aa34
languageName: node
linkType: hard
"@types/whatwg-url@npm:^8.2.1":
version: 8.2.2
resolution: "@types/whatwg-url@npm:8.2.2"
dependencies:
"@types/node": "*"
"@types/webidl-conversions": "*"
checksum: 5dc5afe078dfa1a8a266745586fa3db9baa8ce7cc904789211d1dca1d34d7f3dd17d0b7423c36bc9beab9d98aa99338f1fc60798c0af6cbb8356f20e20d9f243
languageName: node
linkType: hard
"@types/ws@npm:^8.5.1":
version: 8.5.3
resolution: "@types/ws@npm:8.5.3"
@ -12715,11 +12741,15 @@ __metadata:
"@types/express": ^4.17.14
"@types/jest": ^29.2.3
axios: ^1.7.4
dotenv: 10.0.0
express: ^4.20.0
express-validator: ^6.14.2
http-status-codes: ^2.2.0
jest: ^29.3.1
loglevel: ^1.8.1
mongodb: ^5.8.0
nodemailer: 6.9.9
readline-sync: 1.4.10
socket.io: ^4.6.2
supertest: ^6.3.3
ts-jest: 29.1.0
@ -14336,6 +14366,13 @@ __metadata:
languageName: node
linkType: hard
"bson@npm:^5.5.0":
version: 5.5.1
resolution: "bson@npm:5.5.1"
checksum: f49730504e8362e2c8d1eb0c272e5e125392c41fb7196fc35ccbc39718ee62569a1d197bd2342c3334cd420073d3fd5dc7dea764a7f219dcd79e0ce473dd2772
languageName: node
linkType: hard
"buffer-crc32@npm:~0.2.3":
version: 0.2.13
resolution: "buffer-crc32@npm:0.2.13"
@ -17172,6 +17209,13 @@ __metadata:
languageName: node
linkType: hard
"dotenv@npm:10.0.0, dotenv@npm:^10.0.0":
version: 10.0.0
resolution: "dotenv@npm:10.0.0"
checksum: f412c5fe8c24fbe313d302d2500e247ba8a1946492db405a4de4d30dd0eb186a88a43f13c958c5a7de303938949c4231c56994f97d05c4bc1f22478d631b4005
languageName: node
linkType: hard
"dotenv@npm:8.2.0":
version: 8.2.0
resolution: "dotenv@npm:8.2.0"
@ -17179,13 +17223,6 @@ __metadata:
languageName: node
linkType: hard
"dotenv@npm:^10.0.0":
version: 10.0.0
resolution: "dotenv@npm:10.0.0"
checksum: f412c5fe8c24fbe313d302d2500e247ba8a1946492db405a4de4d30dd0eb186a88a43f13c958c5a7de303938949c4231c56994f97d05c4bc1f22478d631b4005
languageName: node
linkType: hard
"dotenv@npm:^8.1.0":
version: 8.6.0
resolution: "dotenv@npm:8.6.0"
@ -20994,10 +21031,13 @@ __metadata:
languageName: node
linkType: hard
"ip@npm:^2.0.0":
version: 2.0.1
resolution: "ip@npm:2.0.1"
checksum: d765c9fd212b8a99023a4cde6a558a054c298d640fec1020567494d257afd78ca77e37126b1a3ef0e053646ced79a816bf50621d38d5e768cdde0431fa3b0d35
"ip-address@npm:^9.0.5":
version: 9.0.5
resolution: "ip-address@npm:9.0.5"
dependencies:
jsbn: 1.1.0
sprintf-js: ^1.1.3
checksum: aa15f12cfd0ef5e38349744e3654bae649a34c3b10c77a674a167e99925d1549486c5b14730eebce9fea26f6db9d5e42097b00aa4f9f612e68c79121c71652dc
languageName: node
linkType: hard
@ -23026,6 +23066,13 @@ __metadata:
languageName: node
linkType: hard
"jsbn@npm:1.1.0":
version: 1.1.0
resolution: "jsbn@npm:1.1.0"
checksum: 944f924f2bd67ad533b3850eee47603eed0f6ae425fd1ee8c760f477e8c34a05f144c1bd4f5a5dd1963141dc79a2c55f89ccc5ab77d039e7077f3ad196b64965
languageName: node
linkType: hard
"jsbn@npm:~0.1.0":
version: 0.1.1
resolution: "jsbn@npm:0.1.1"
@ -24357,6 +24404,13 @@ __metadata:
languageName: node
linkType: hard
"memory-pager@npm:^1.0.2":
version: 1.5.0
resolution: "memory-pager@npm:1.5.0"
checksum: d1a2e684583ef55c61cd3a49101da645b11ad57014dfc565e0b43baa9004b743f7e4ab81493d8fff2ab24e9950987cc3209c94bcc4fc8d7e30a475489a1f15e9
languageName: node
linkType: hard
"merge-descriptors@npm:1.0.3":
version: 1.0.3
resolution: "merge-descriptors@npm:1.0.3"
@ -25107,6 +25161,48 @@ __metadata:
languageName: node
linkType: hard
"mongodb-connection-string-url@npm:^2.6.0":
version: 2.6.0
resolution: "mongodb-connection-string-url@npm:2.6.0"
dependencies:
"@types/whatwg-url": ^8.2.1
whatwg-url: ^11.0.0
checksum: 1d662f0ecfe96f7a400f625c244b2e52914c98f3562ee7d19941127578b5f8237624433bdcea285a654041b945b518803512989690c74548aec5860c5541c605
languageName: node
linkType: hard
"mongodb@npm:^5.8.0":
version: 5.9.2
resolution: "mongodb@npm:5.9.2"
dependencies:
"@mongodb-js/saslprep": ^1.1.0
bson: ^5.5.0
mongodb-connection-string-url: ^2.6.0
socks: ^2.7.1
peerDependencies:
"@aws-sdk/credential-providers": ^3.188.0
"@mongodb-js/zstd": ^1.0.0
kerberos: ^1.0.0 || ^2.0.0
mongodb-client-encryption: ">=2.3.0 <3"
snappy: ^7.2.2
dependenciesMeta:
"@mongodb-js/saslprep":
optional: true
peerDependenciesMeta:
"@aws-sdk/credential-providers":
optional: true
"@mongodb-js/zstd":
optional: true
kerberos:
optional: true
mongodb-client-encryption:
optional: true
snappy:
optional: true
checksum: fad5621f4a9764d71c48f2955be57925e506b14a4a63c5b9209ff733a58b2524cf9c87f216f2f02de092fab20eaa033ff5ade45c5cfa7f1c8c6bce114cb22e3f
languageName: node
linkType: hard
"moo-color@npm:^1.0.2":
version: 1.0.2
resolution: "moo-color@npm:1.0.2"
@ -25409,6 +25505,13 @@ __metadata:
languageName: node
linkType: hard
"nodemailer@npm:6.9.9":
version: 6.9.9
resolution: "nodemailer@npm:6.9.9"
checksum: 9af862b497273796ad6ba7ca9eac23a7d69737554f6dd51cfd3557f60902c56344158192672f1208daf00340a47a9e1805bd809965edc191011b35dd98a74be1
languageName: node
linkType: hard
"nopt@npm:^6.0.0":
version: 6.0.0
resolution: "nopt@npm:6.0.0"
@ -29412,6 +29515,13 @@ __metadata:
languageName: node
linkType: hard
"readline-sync@npm:1.4.10":
version: 1.4.10
resolution: "readline-sync@npm:1.4.10"
checksum: 4dbd8925af028dc4cb1bb813f51ca3479035199aa5224886b560eec8e768ab27d7ebf11d69a67ed93d5a130b7c994f0bdb77796326e563cf928bbfd560e3747e
languageName: node
linkType: hard
"recast@npm:^0.23.3, recast@npm:^0.23.5":
version: 0.23.6
resolution: "recast@npm:0.23.6"
@ -30987,13 +31097,13 @@ __metadata:
languageName: node
linkType: hard
"socks@npm:^2.6.2":
version: 2.7.1
resolution: "socks@npm:2.7.1"
"socks@npm:^2.6.2, socks@npm:^2.7.1":
version: 2.8.3
resolution: "socks@npm:2.8.3"
dependencies:
ip: ^2.0.0
ip-address: ^9.0.5
smart-buffer: ^4.2.0
checksum: 259d9e3e8e1c9809a7f5c32238c3d4d2a36b39b83851d0f573bfde5f21c4b1288417ce1af06af1452569cd1eb0841169afd4998f0e04ba04656f6b7f0e46d748
checksum: 7a6b7f6eedf7482b9e4597d9a20e09505824208006ea8f2c49b71657427f3c137ca2ae662089baa73e1971c62322d535d9d0cf1c9235cf6f55e315c18203eadd
languageName: node
linkType: hard
@ -31114,6 +31224,15 @@ __metadata:
languageName: node
linkType: hard
"sparse-bitfield@npm:^3.0.3":
version: 3.0.3
resolution: "sparse-bitfield@npm:3.0.3"
dependencies:
memory-pager: ^1.0.2
checksum: 174da88dbbcc783d5dbd26921931cc83830280b8055fb05333786ebe6fc015b9601b24972b3d55920dd2d9f5fb120576fbfa2469b08e5222c9cadf3f05210aab
languageName: node
linkType: hard
"spawn-wrap@npm:^2.0.0":
version: 2.0.0
resolution: "spawn-wrap@npm:2.0.0"
@ -31208,6 +31327,13 @@ __metadata:
languageName: node
linkType: hard
"sprintf-js@npm:^1.1.3":
version: 1.1.3
resolution: "sprintf-js@npm:1.1.3"
checksum: a3fdac7b49643875b70864a9d9b469d87a40dfeaf5d34d9d0c5b1cda5fd7d065531fcb43c76357d62254c57184a7b151954156563a4d6a747015cfb41021cad0
languageName: node
linkType: hard
"sprintf-js@npm:~1.0.2":
version: 1.0.3
resolution: "sprintf-js@npm:1.0.3"
@ -32455,6 +32581,15 @@ __metadata:
languageName: node
linkType: hard
"tr46@npm:^3.0.0":
version: 3.0.0
resolution: "tr46@npm:3.0.0"
dependencies:
punycode: ^2.1.1
checksum: 44c3cc6767fb800490e6e9fd64fd49041aa4e49e1f6a012b34a75de739cc9ed3a6405296072c1df8b6389ae139c5e7c6496f659cfe13a04a4bff3a1422981270
languageName: node
linkType: hard
"tr46@npm:~0.0.3":
version: 0.0.3
resolution: "tr46@npm:0.0.3"
@ -33804,6 +33939,13 @@ __metadata:
languageName: node
linkType: hard
"webidl-conversions@npm:^7.0.0":
version: 7.0.0
resolution: "webidl-conversions@npm:7.0.0"
checksum: f05588567a2a76428515333eff87200fae6c83c3948a7482ebb109562971e77ef6dc49749afa58abb993391227c5697b3ecca52018793e0cb4620a48f10bd21b
languageName: node
linkType: hard
"webpack-bundle-analyzer@npm:^3.6.0":
version: 3.9.0
resolution: "webpack-bundle-analyzer@npm:3.9.0"
@ -34031,6 +34173,16 @@ __metadata:
languageName: node
linkType: hard
"whatwg-url@npm:^11.0.0":
version: 11.0.0
resolution: "whatwg-url@npm:11.0.0"
dependencies:
tr46: ^3.0.0
webidl-conversions: ^7.0.0
checksum: ed4826aaa57e66bb3488a4b25c9cd476c46ba96052747388b5801f137dd740b73fde91ad207d96baf9f17fbcc80fc1a477ad65181b5eb5fa718d27c69501d7af
languageName: node
linkType: hard
"whatwg-url@npm:^5.0.0":
version: 5.0.0
resolution: "whatwg-url@npm:5.0.0"

File diff suppressed because it is too large Load Diff

View File

@ -1,32 +0,0 @@
{
"name": "appsmith_utils",
"version": "1.0.0",
"description": "appsmith utils tool",
"main": "bin/index.js",
"author": "",
"license": "Apache-2.0",
"private": true,
"repository": {
"type": "git",
"url": "https://github.com/appsmithorg/appsmith.git",
"directory": "deploy/docker"
},
"dependencies": {
"dotenv": "10.0.0",
"mongodb": "^5.8.0",
"nodemailer": "6.9.9",
"readline-sync": "1.4.10"
},
"devDependencies": {
"jest": "^29.7.0"
},
"bin": {
"appsmithctl": "./bin/index.js"
},
"scripts": {
"test": "jest"
},
"overrides": {
"fast-xml-parser": "4.4.1"
}
}

View File

@ -0,0 +1,4 @@
#!/usr/bin/env bash
# An alias to `ctl`.
exec ctl "$@"

View File

@ -0,0 +1,6 @@
#!/usr/bin/env bash
# Do NOT change working directory with `cd`, so that the command being run has access to the working directory where
# the command was invoked by the user.
exec node /opt/appsmith/rts/bundle/ctl/index.js "$@"